* hydra/berlin.scm (%rsync-modules-for-backup): New variable.
<services>: Add 'rsync-service-type' instance.
* hydra/website.scm (website-services): Add comment about the Cuirass
and GWL web sites.
This means that when the relevant AAAA records are added, these sites
will be available over IPv6.
* hydra/bayfront.scm (%hpc.guix.info-nginx-servers,
%guix-hpc.bordeaux.inria.fr-nginx-servers,
%coordinator.bayfront.guix.gnu.org-nginx-servers,
%bayfront.guix.gnu.org-nginx-servers,
%bordeaux.guix.gnu.org-nginx-servers): Listen on IPv6 as well as IPv4.
This changes were made last week.
This commit switches from serving narinfo files directly from the
disk, and instead has NGinx forward those requests to the
nar-herder. For nar requests, NGinx will check the local storage, and
if the nar isn't there, it'll forward the requests to
lakefront.cbaines.net. Additionally, the nars are now cached by
NGinx. The Guix Build Coordinator configuration is changed to call the
nar-herder to import new nars.
* hydra/bayfront.scm: Adapt the NGinx and Guix Build Coordinator
configuration for the introduction of the Nar Herder.
This increases the number of concurrent requests it can handle.
* hydra/bayfront.scm (services): Alter the nginx-configuration to run
8 worker processes.
Factorize website services in a single (website) module so that they can be
hosted by multiple nodes for redundancy.
* hydra/website.scm: New file.
* hydra/berlin.scm: Use it.
This is partially duplicating the behaviour of the NGinx configuration
for bordeaux.guix.gnu.org. For maintainability, I think it's worth now
switching to just having bordeaux.guix.gnu.org serve nars. This also
enables getting guix publish back running on bayfront.guix.gnu.org if
that's something that's wanted.
* hydra/bayfront.scm (%bayfront.guix.gnu.org-nginx-servers): Remove
substitute serving related configuration.
Use bordeaux.guix.gnu.org rather than bayfront.guix.gnu.org for non
derivation substiutes, as this is the proper domain to use.
* hydra/bayfront.scm (services): Change non-derivation-substitute-urls
within the guix-build-coordinator-agent-configuration to
https://bordeaux.guix.gnu.org.
Drop --max-jobs to 1, but increase --cores to 16. Since the
guix-build-coordinator-agent is only running 1 build at a time, this
might make things a little faster.
* hydra/bayfront.scm (services)[guix-configuration]: Change --max-jobs
to 1 and --cores to 16.
Fix the following error:
service networking provided more that once
by using a single static-networking record for both interfaces.
* hydra/berlin.scm: Fix the static networking service.
Only the keys from the berlin directory are authorized in the %build-node-keys
procedure.
* hydra/keys/guix/grunewald.rekado.pub: Move it to the berlin directory.
* hydra/keys/guix/kreuzberg.rekado.pub: Move it to the berlin directory.
* hydra/berlin.scm: Add wireguard-peer.
* hydra/deploy-honeycomb.scm: Add machine record for "grunewald".
* hydra/keys/guix/berlin/grunewald.rekado.pub: New file.
I wanted to keep this minimal. It didn't work: Wireguard clients use
10.0.0.0/8 IPs directly as well.
* hydra/nginx/berlin.scm (%berlin-servers): Match anything ending in a
digit, which, until the gTLD crowd goes truly bonkers, is an IP address.
And if it's not, the request wouldn't reach us anyway, right?
Reported by Ludovic Courtès <ludo@gnu.org>.
* hydra/nginx/berlin.scm (%berlin-servers): Add a default port-80 server
to redirect all requests to their HTTPS counterparts. Remove explicit
HTTP support for guix.gnu.org and issues.guix.gnu.org.
This makes the 'rdv-guix-maintainers' as well as the
'rdv-guix-sysadmin' Jami rendezvous points accounts available for live
conference calls (audio or video). The service runs in a Linux
container and can only be reached by the Jami contacts declared as
allowed.
* hydra/milano-guix-1.scm (operating-system)
[services]{jami-service-type}: New service.
With the recent OpenSSH 8.8p1 update, SHA1-signed RSA keys are
deprecated, which my previous key was.
* hydra/keys/ssh/maxim.pub: Update to a newer ed25519 key.
* hydra/bayfront.scm <nginx-configuration>: Remove 'nginx' field. The
extra modules are enabled by default in the 'nginx' package since Guix
commit 2d31eeecf06ab14732d374c75cdf6e7a55aa704e.
* hydra/modules/sysadmin/services.scm (<disarchive-configuration>): New
record type.
(disarchive-activation, disarchive-mcron-jobs): New procedures.
(disarchive-service-type): New variable.
* hydra/berlin.scm <services>: Use it.
This should avoid high load on bayfront from building things.
* hydra/bayfront.scm (operating-system)[services]: Have the
guix-build-coordinator-agent only perform one build at a time,
increase guix-daemon timeouts and reduce the --cores configuration to
6.
This has actually been in place for a week now, so
bordeaux.guix.gnu.org has good coverage for armhf-linux.
* hydra/bayfront.scm (operating-system)[services]: Change the
guix-build-coordinator-queue-builds-configuration to include
armhf-linux.
This has been discussed here: <https://issues.guix.gnu.org/48926>.
* hydra/modules/sysadmin/services.scm (frontend-services): Increase the
default publish nar ttl to 180 days.
That makes it consistent with the guix-daemon settings.
* hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os):
Use MAX-JOBS as the number of Cuirass workers.
* hydra/nginx/berlin.scm (languages-to-accept): New list. Move here
the languages list from ...
(%extra-content) ... here. Use languages from 'languages-to-accept'.
(guix.gnu.org-redirects-for-each-language): New procedure. Add
new video redirections.
(guix.gnu.org-redirect-locations): New list. Move here the
old redirections.
(guix.gnu.org-other-locations): New list. Move here the other nginx
locations from ...
(guix.gnu.org-locations): ... here. Reimplement in terms of the above.
* hydra/nginx/html/bordeaux/index.html: (Using these substitutes):
Fixed a typo, and added a how to code snippet for users of guix system.
Signed-off-by: Christopher Baines <mail@cbaines.net>
* hydra/bayfront.scm (operating-system)[services]: Change the
guix-build-coordinator-queue-builds-configuration to include i586-gnu
and cross builds to i586-pc-gnu.
* hydra/nginx/html/bordeaux/index.html: New file.
* hydra/bayfront.scm (%bayfront-nginx-service-extra-config): Remove
proxy config relevant for hydra.
(%bordeaux.guix.gnu.org-nginx-servers): Serve log files, and show log
files for /build/UUID requests. Also service an index.html page.
(operating-system)[services]: Use a custom nginx package built with a
couple of additional modules.
This is used when deciding whether the outputs from a build should be
submitted.
* hydra/bayfront.scm (%guix-build-coordinator-configuration)[hooks]:
Specify a build-submit-outputs-hook.
* hydra/milano-guix-1.scm (operating-system)[file-systems]: Mount /tmp
as a tmpfs, to hopefully speed up builds.
[swap-devices]: Add /dev/sdb2.
[services]: Disable SSH password authentication, and switch bayfront
to bordeaux in a couple of places.
* hydra/bayfront.scm (%bordeaux.guix.gnu.org-nginx-servers): New
variable.
(servers)[nginx-service-type]: Add
%bordeaux.guix.gnu.org-nginx-servers to the server blocks list.
Make the Guix Build Coordinator agent more gentle, reducing the max
parallel builds and decreasing the load average limit. Also reduce the
default max-jobs at the daemon level (for builds happening not through
the coordinator agent), and set the default cores to 24 to avoid one
build from loading all the cores.
* hydra/bayfront.scm (services)[guix-build-coordinator-agent-service-type]:
Tweak max-parallel-builds and max-1min-load-average.
[guix-service-type]: Tweak the extra-options.
This is to be used for substitutes, currently served from bayfront.
* hydra/modules/sysadmin/dns.scm (guix.gnu.org.zone): Add entry for
bordeaux.guix.gnu.org.
Berlin doesn't build much stuff anymore, and removing transient cache failures
require manual intervention.
* hydra/modules/sysadmin/services.scm (guix-daemon-config): Do not cache failures.
Those machines are unreachable and should be made available through Wireguard
when back online.
* hydra/machines-for-berlin (overdrive): Remove unreachable machines.
Also mention the CPU they use and the amount of physical memory they
have. Likewise for the Softiron Overdrive 1000.
* hydra/machines-for-berlin.scm (overdrive): Mention CPU model and RAM.
(armv7): Likewise for the BeagleBoard.
This is required for offloading.
* hydra/keys/guix/maxim-desktop-export.pub: New key.
* hydra/modules/sysadmin/overdrive.scm (%authorized-guix-keys):
Authorize it.
This is so that I can use overdrive1 as an offload machine.
* hydra/modules/sysadmin/overdrive.scm (%accounts) [maxim]: New
sysadmin account.
(overdrive-system) [service] <openssh-service-type>: Authorize my
public SSH key.
So that the guix-build-coordinator agent builds don't get stuck.
* hydra/milano-guix-1.scm (operating-system)[services]: Specify values
for max-silent-time and timeout.
This commit adapts milano-guix-1 to build things for the Guix Build
Coordinator instance running on bayfront, and removes the reminants of
the configuration related to running data.guix.gnu.org (since it's
hosted elsewhere now).
* hydra/milano-guix-1.scm (gc-job): Garbage collect 500G since 50G is
not much free space for performing builds.
(%nginx-configuration): Remove variable.
(operating-system)[packages]: Remove comment.
[services]: Comment out qemu-binfmt-service-type as it's currently
unused, remove PostgreSQL, add the Prometheus node exporter, add the
Guix Build Coordinator agent and adjust the Guix service to allow for
substitutes from both bayfront and data.guix.gnu.org.
Bayfront is now running the Guix Build Coordinator, and building
things for x86_64-linux. This has been useful for finding and fixing
some bugs in this area at least.
* hydra/bayfront.scm (operating-system)[swap-devices]: Add /swap since
it exists.
[services]: Add guix-build-coordinator-queue-builds-service-type, and
tweak the guix-build-coordinator agent configuration.
The p9.tobias.gr machine is a POWER9 box lent by OSUOSL.
* hydra/keys/guix/p9.tobias.gr-export.pub,
hydra/keys/guix/berlin/p9.tobias.gr.pub: New files.
Simon mentioned the machines no longer have an associated A record;
rename name, for clarity.
* hydra/machines-for-berlin.scm (armv7)
<guix-x15.sjd.se, guix-x15b.sjd.se>: Rename to guix-x15 and guix-x15b.
* hydra/berlin.scm (services) [wireguard]: Likewise.
* doc/cuirass.org (External machines): Likewise.
* hydra/modules/sysadmin/services.scm (cuirass-specs): Restrict system tests
to the x86_64-linux architecture. The situation on other architectures is for
now too problematic to provide a valuable information.
Both the coordinator component, and an agent running locally on
bayfront.
This commit also makes other changes to enable this.
* hydra/bayfront.scm (%nginx-deploy-hook): Fix location of the NGinx
pid file.
(%certbot-configuration): Add coordinator.bayfront.guix.gnu.org.
(%bayfront.guix.gnu.org-nginx-servers): Adjust for serving narinfos
and nars from /var/lib/nars.
(%coordinator.bayfront.guix.gnu.org-nginx-servers,
%guix-build-coordinator-configuration): New variables.
(operating-system)[packages]: Add guix-build-coordinator.
[services]: Adjust NGinx and Guix, add the Guix Build Coordinator and
Guix Build Coordinator agent.
This is helpful to deploy the Guix Build Coordinator on bayfront, this
domain will be used by agents to communicate with the coordinator.
* hydra/modules/sysadmin/dns.scm (guix.gnu.org.zone): Add
"coordinator.bayfront". Bump 'serial'.
The previous attempt contained a line from a previous version of the
file that caused a conflict.
* hydra/nginx/berlin.scm (%extra-content): Autoredirect 'eo', 'ko' and 'ru'
to the translated website.
This reverts commit 3d63e8d690 because it causes
the following nginx error:
2021/04/07 17:05:08 [emerg] 94058#0: variable already defined: "lang" in /gnu/store/ajvqgc205hvrfab7plbwds2a9wiqj52f-nginx.conf:4666
This should have been %build-node-keys, but was changed when
refactoring the config recently.
* hydra/bayfront.scm (operating-system)[services]: Adjust guix
authorized-keys.
This involves a few things. The Cuirass service is removed through
moving most of the frontend-services functionality directly in to the
configuration, and the NGinx configuration is transformed in to
various records. I'm hoping this will make the bayfront configuration
easier to change and maintain.
* hydra/bayfront.scm (%nginx-config): Remove variable.
(%bayfront-nginx-service-extra-config, %hpc.guix.info-nginx-servers,
%guix-hpc.bordeaux.inria.fr-nginx-servers,
%logs.guix.gnu.org-nginx-servers,
%bayfront.guix.gnu.org-nginx-servers): New variables.
(operating-system)[packages]: Remove comment relating to Cuirass.
[services]: Remove Cuirass dependencies, and incorporate most of the
services returned by frontend-services.
* hydra/nginx/bayfront-locations.conf: Delete file.
* hydra/nginx/bayfront.conf: Delete file.
* hydra/nginx/guix-hpc-inria-locations.conf: Delete file.
* hydra/nginx/guix-hpc-locations.conf: Delete file.
* hydra/modules/sysadmin/services.scm (guix-input): Remove it.
(cuirass-notifications): New procedure.
(cuirass-specs): Adapt it to use Cuirass new specification format.
Now web browsers requesting any kind of Chinese get the website in
mainland Chinese.
zh, zh-Hans, zh-Hans-CN all are synonymous with zh-CN now.
Fixes <https://bugs.gnu.org/46807>.
* hydra/nginx/berlin.scm (accept-languages): New procedure.
(%extra-content): Normalize $lang variable with it.
The previously configured certificate was for an obsolete domain name
(berlin.guixsd.org) & cannot be renewed, causing an outage on 2021-02-27.
* hydra/nginx/berlin.scm (%berlin-servers): Use an existing Let's Encrypt
certificate for "ci.guix.gnu.org".
* hydra/berlin.scm: Use "postgresql-service-type" instead of
"postgresql-service" to prevent a warning message.
Signed-off-by: Mathieu Othacehe <othacehe@gnu.org>
* hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os): Add
a systems argument. Start a Cuirass remote worker building substitutes for the
given systems.
* hydra/modules/sysadmin/services.scm (cuirass-without-fiber-tests): Remove
it.
(cuirass-service): Add a remote-server.
Signed-off-by: Mathieu Othacehe <othacehe@gnu.org>
* hydra/modules/sysadmin/web.scm (gwl-web-shepherd-service): Remove
inner wrapping with GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH; set
GUIX_EXTENSIONS_PATH instead; run "guix workflow web".
(gwl-web-service-type): Update description.
So that they have more swap, and thus can run more parallel builds
without running out of memory.
* hydra/fosshost1.scm: Add /swapfile to swap-devices list.
* hydra/fosshost2.scm: Likewise.
* hydra/nginx/berlin.scm (guix.info-locations): Remove variable.
(%berlin-servers): Move all legacy hostnames to their own server
configuration, and redirect to the current name.
These are virtual machines provided by Fosshost. They're situated in
the USA, and currently each have 6 cores + 8GB of RAM.
They're currently being uses to build things through the instance of
the Guix Build Coordinator I'm running to quality assurance and patch
review, but they can be used for other things as well.
* hydra/fosshost1.scm: New file.
* hydra/fosshost2.scm: New file.
* hydra/bayfront.scm (%certbot-configuration): Add logs.guix.gnu.org to the
DOMAINS.
* hydra/nginx/bayfront.conf: Add a Certbot webroot location to the
logs.guix.gnu.org HTTP server. Add an HTTPS (‘ssl’) server block for the
same domain using the newly-generated certificate.
Recently many machines runs out of disk space regularly.
* hydra/modules/sysadmin/build-machines.scm (berlin-new-build-machine-os):
Free 150GiB instead of 90 GiB.
This is in preparation of purely declarative ACLs:
https://issues.guix.gnu.org/39819#8
* hydra/berlin.scm (%build-node-key-directory, %build-node-keys): New
variables.
<top level>: Pass #:authorized-keys to 'frontend-services'.
Recently we have run out of build users on a few occasions so increase
the number.
* hydra/berlin.scm <services>: Pass #:build-accounts-to-max-jobs-ratio
to 'frontend-services'.
This essentially reverts commit
7463429bee, which was used for testing.
* hydra/berlin.scm (childhurd-gc-job, childhurd-os)
(childhurd-net-options): Remove.
<service>: Remove 'hurd-vm-service-type' instance.
* hydra/modules/sysadmin/services.scm (cuirass-without-fiber-tests)[inputs]:
Use "guile-3.0/libgc-7" instead of "guile" to workaround libgc memory issues.
