${arch}/${branch}/latest/${portdir}. We will use this in the
processfail script, so that the "new package build errors" webpages do
not have out-of-date links but instead link to the most recent copy of
the build error.
that it may be called by hand.
Support new portbuild.conf variables
client_user = user to connect to on the client (not necessarily
root). This user must have write permission to the
/var/portbuild tree if disconnected=1 (i.e. we're
going to run rsync).
rsync_gzip = set to "-z" to enable compression on low-bandwidth
disconnected clients.
Approved by: portmgr (self)
ssh times out)
* Support new portbuild.conf settings:
client_user = user to connect to on the client (not necessarily root)
sudo_cmd = If ssh'ing to a non-root user, run this command to gain
root privs (set to empty string for client_user=root,
or sudo for !root). Cannot require interactivity, of
course.
Approved by: portmgr (self)
because this file is a chronological history of port builds that have
failed, the files listed may not be present in the current set of
error logs, and we currently have no easy way to find the most recent
failure log to use instead.
i386-5-latest that are linked to from the index.html are symlinks to
dated directories (e.5.`date`), so the URLs in the error reports will
expire with the start of the next build when the symlink is repointed.
This change makes the URLs in the error reports use the realpath of
the target file, so they do not expire.
* Clients no longer have ssh access to the master, so we need to
push/pull everything on the client from here. This means we need to
know where the build took place so we can go in and get the files
after it finishes. Introduce the claim-chroot script which
atomically claims a free chroot directory on the host and returns
the name. This directory is later populated by the portbuild script
if it does not already contain an extracted bindist.
* Use the per-node portbuild.$(hostname) config file to decide where
in the filesystem to claim the chroot on the build host.
* If a port failed unexpectedly (i.e. is not marked BROKEN), or if
something strange happened when trying to pull in build results from
a client, then send me email (XXX should be configurable).
* Clean up after the build finishes and we have everything we need, by
dispatching the clean-chroot script on the client.
if requested (".keep" file in the port directory), no matter where
we fail.
* Add package dependencies before the corresponding build stage
(e.g. FETCH_DEPENDS before 'make fetch'), and remove them again
afterwards. This allows us to catch ports that list their
dependencies too early/late.
* No need to check for set[ug]id files here, the security-check target
in bsd.port.mk does it for us.
* Exclude some more directories and files from showing up in the mtree
before/after comparison, to trim down the false-positive in the
pkg-plist check.
* Other minor changes
it's done properly^Wbetter in makeparallel
* Script accepts new arguments:
-nodoccvs: skip cvs update of the doc tree
-trybroken: try to build BROKEN ports (off by default because the
i386 cluster is fast enough now that when doing incremental builds we
were spending most of the time rebuilding things we know are probably
going to fail anyway. Conversely, the other clusters are slow enough
that we also usually don't want to waste time on BROKEN ports).
-incremental: compare the interesting fields of the new INDEX with
the previous one, remove packages and log files for the old ports that
have changed, and rebuild the rest. This substantially cuts down on
build times since we don't rebuild ports that we know have not
changed. XXX checkpoint of work-in-progress, not yet working as
committed.
* When setting up the nodes, read in per-node config files
("portbuild.$(hostname)") before dispatching the setupnode script on
each node. For disconnected nodes (which don't mount the master via
NFS), we also rsync the interesting files required by the builds
(ports/src/doc trees, bindist tarballs, scripts) into place on the
client. They will be mounted locally via nullfs in the build chroots.
* Break out the restricted.sh generation into a makerestr script so it
can be called manually as needed.
* Remove the -nocvsup argument which has been unused for a long time.
* For now, don't prune the list of failed ports with prunefail,
since when -trybroken is not specified, every BROKEN port ends up in
the duds file (so the build is skipped), and as a result we would
prune almost everything from the list of failed ports. XXX
prunefailure should be run conditionally on -trybroken, or I should
find a way to prune in both cases.
* Don't run index in the background, it was thrashing against makeduds
and not saving any time by doing it concurrently.
* Build with 'make quickports all' to kick off the quickports builds
earlier.
* Delete restricted and/or cdrom distfiles *after* post-processing the
distfiles, otherwise the script doesn't remove any of them since
they're not in the expected place.
* Miscellaneous other minor changes and cleanups
tells us whether the node has NFS access to the master.
* Also copy the bindist-$(hostname).tar file to allow local
customization of the build chroots (e.g. resolv.conf and make.conf
files for disconnected systems)
* For disconnected hosts, we don't copy the bindist files from the
master, but just set up the local directories and let the server rsync
them into place later. Also set up dangling symlinks to the bindist
files in the build area, which will be filled in by the server too (in
the NFS case it makes sense to cache the bindist files locally to
avoid extra NFS traffic, but here we know the file is local so a
symlink is fine)
* Remove an apparently spurious 'killall fetch' that snuck in for what
were probably transient reasons.
* Forcibly clean up old chroot directories since we are preparing to
start another build and don't want old (possibly orphaned) builds to
skew the job scheduling or use up resources.
host), specified by disconnected=1 in portbuild.$(hostname) file.
These do not mount via NFS, so we need to maintain a local copy of
things needed by the build (like the ports/src/doc trees) on the build
host, which are mounted into the chroot by read-only nullfs. These
local files are maintained in the dopackages script via rsync.
* Download packages via http instead of NFS. Allow fetching via a
local http proxy (http_proxy variable in per-node
portbuild.$(hostname) file). Caching package dependencies saves about
85% of package fetches and similar reduction in package fetch traffic
by byte count.
* Support a per-node tarball (bindist-$(hostname).tar) to customize
the build chroots. This is used for things like local resolv.conf and
make.conf files on disconnected nodes.
* Make sure we don't use a chroot until it is finished extracting.
* Don't set '.' in PATH; this is bad practise, and fortunately nothing
seems to rely on it.
* Only try to build broken packages if requested
* Try harder to unmount leftover linprocfs mounts in the chroot, by copying
in the 5.x mount binary and supporting libraries from the host system.
The 5.x mount is able to unmount by FSID in situations where the 4.x umount
becomes confused.
* Don't clean up when we are signalled, that is done by the build
master from outside.
* Suppress some code relating to jail builds, which are not yet ready
for use.
* Don't push results of the build back to the master; the master now
pulls them from the client when the build completes. Clients no
longer need ssh access into the master; this is good for security as
well as significantly reducing the load on the master since it is not
thrashed by dozens of sshd processes.
advantage is that here we know the value of PKGSUFFIX (.tgz/.tbz) for
the build via buildenv.
* Add a list of 'quickports', which are ports with long dependency chains
that we should kick off straight away to try and avoid bottlenecks later
on when most of the cluster idles waiting for one or two ports to build.
Ideally we'd build dependencies of these ports exclusively first and only
build other ports when we run out (i.e. a build slot becomes free), but I
couldn't work out how to do this. As a compromise, we now do
'make -k -j<#> quickports all' which doesn't give quite as high a
priority to the quickports (i.e. we also build other ports from the
beginning while there are quickport dependencies still to build), but is
better than nothing.
* Pass in the FETCH/EXTRACT/PATCH/BUILD/RUN_DEPENDS separately via env
variables when dispatching a job. This allows us to add and remove
the dependencies at the corresponding build stage to catch ports
with dependencies listed too early/late.
sure we don't try and schedule jobs on it even if all other machines are
busy
* Remove sleep in outer loop, this isn't needed or worthwhile now that there
are so many machines being monitored
build locking, log files, and cleans things up if a build fails.
This script is the primary starting point for a package build. Symlinks
should be created in the form of dopackages.${branch} -> dopackages.wrapper
where ${branch} is currently one of 4, 4-exp, or 5. This script takes the
place of the unofficial (i.e. uncommitted) dopackages.steveX scripts.
Ok'd by: kris
Tested by: 4.10-RELEASE package build
used in 20 minutes, as well as directories listed as 'in use' that have not been touched
in 24 hours (corresponding to port builds that have timed out or shut down uncleanly)
and prunes them to reclaim space. This is intended to be run as a cron job.
more accurate:
* The tests for "Cannot stat", "can't cd", and "tar command failed" are moved
to the bottom. This will sacrifice a little bit of performance for a gain
in accuracy
* The tests for "configuration errors" needed to be combined and tweaked to
better identify errors that are architecture-specific (usuallly the
"all pointers are ints" assumption)
* The detection of chown was buggy
* There were some duplicate test cases
* As an experiment, another test for dependency problems is added
PR: 61914
Submitted by: linimon
encountered again, nad having it in there slows performance. However,
I'm leaving it in the file in case a bad bindist slips back in, and it
becomes convenient to uncomment this for purposes of seeing what build
failures are cluster-related.
Requested by: kris
processonelog script, that contains all the per-errorlog logic [1]
* Add new error checks, one for detecting CPU problems, and the other for
checking for broken libgnugetopt support [2]
* Add a new "munmap" error type to check for broken bindists
* Add imake and pthread detection to processonelog [3]
* Add errortype column to the bento "New Build Failures" report [4]
PR: 50258 56859 [1]
54406 [2]
57067 [3]
59272 [4]
Submitted by: linimon [1] [2] [3] [4]
is full
* The newfailure file is no longer maintained, but generated as-needed
by the processfail script
* Update format of failure file (records both initial failure date and
most recent failure date)
of stale entries (removed ports, ports listed in duds that were mistakenly
built, malformed entries) and refreshes the version of entries to the
latest in INDEX. This must be run under lockf (see the comments in the
script) to avoid racing with portbuild which also tries to write to
these files.
processing (e.g. transfer to ftp site). It was taking up too much disk
space to collect distfiles for 14 architecture/version combinations
(including the backup set from the previous run).
Put the cvsdone files in the arch/branch subdirectory, not arch/.
of having the clients scp their loads to bento every 10 seconds. Fix
some indentation and add some sleeps to make sure the startup script
doesn't run too early in the boot process.
Add some new command-line options:
* -continue : restart the build without rebuilding ports that have
already failed
* -nofinish : don't run the post-processing steps when the build finishes
* -finish : run the post-processing steps only
Reorder some code sections appropriately
- Set the UNAME_* variables to make use of the override support in uname(1)
This allows us to get rid of the uname shell script in the build chroots
- Collect some variables that are probably old and mouldy, for later GCing
- If FETCH_ORIGINAL is set in the environment, don't fetch from ftp-master
in portbuild.conf). [1]
* Build stage 2 builds in a jail instead of a chroot. This allows us to
detect ports that attempt IP communication during the build, as well as
allowing clean termination of the port build (packages built in a chroot
can leave processes hanging around after the build finishes). There
are some caveats with this approach which will be worked around in a
future commit.
Requested by: peter [1]
are now collected by polling a small server on the client (using netcat)
instead of having the client scp the load files to the master every 10
seconds (!!!)
- Call processfail to produce 'new port failures' page
- Comment out the 'comparelogs' calls for now until I make them aware
of architectures that don't have 4.x support
and 'buildfailure' and produce a HTML output listing ports with the date
they became broken. The output can surely be made more useful (e.g. it
always links to the ${branch}-latest logs, which may not exist when the
build-in-progress has not yet attempted the build of that port).
keep track of the number of times a port has failed to build, resetting
the counter when it builds successfully. There are still some bugs to be
worked out, but this will be used as the basis for maintainer notification
of port failures.
- Increase timeout to 2 hours (needs to be arch-specific)
- Mount nfs filesystems with nfsv3,intr (the latter so that clients do
not hang if bento panics)
- Run pnohang.${arch} on clients instead of pnohang
- Support builds as non-privileged user
- Increase timeout to 8 hours (this needs to be made per-arch so it
doesn't overly pessimize fast client machines)
- Support building as a non-privileged user
- Use $INDEXFILE
- Use makeparallel script instead of 'make parallel'
- Support building packages as a non-privileged user (needs root access
to client machines)
- Switch to using ${arch}/${branch} subdirectory instead of ${branch}
- export INDEXFILE, MACHINE_ARCH and ARCH variables
- Add (commented out) FETCH_BEFORE_ARGS setting
- Add MASTER_SITE_OVERRIDE to fetch from ftp://bento
- Zap some mouldy old unused variables
body tag nor a closing html tag. In general, browsers work without
these, but Konqueror in particular was displaying some artifacts.
Also make sure empty cells have an nbsp directive; eliminate a bogus
case for Aff.; and change Date build to Build date.
Submitted by: Mark Linimon <linimon@lonesome.com>
PR: www/45021
* Update list of quickports to include kde3
* Add ability to restart builds (-restart)
* Change default to build packages for non-cdrom use (associated option name
inverted from -nocdrom to -cdrom)
* Cope with .tgz or .tbz packages
* Use buildenv framework
* Use cvs update -PdA instead of -Pd when updating ports/docs tree
* Always delete restricted packages after a build run, since they are
published on the website.
* Minor cosmetic changes
* generate an interim .logs file containing the data to be presented in
the html files (also useful for other consumers, like fenner's
per-maintainer error logs)
* Generate 5 different summaries of the error data, sorted by port name,
maintainer, category, build error, builddate
* Improve the table html code so it renders faster
* Add a few new failure reasons
Submitted by: edwin (based on)
PR: ports/43927
uname script. Mount the docs from the correct location. Remove the
hack to pkg_add perl into the chroot environment, now that the scripts
no longer rely on it.
table row, instead of making a new row for every single file. This
cuts down the generated page size drastically, improving rendering times.
Submitted by: Edwin Groothuis <edwin@mavetju.org>
PR: ports/40962
* Add a trap handler to try and clean up the build if it is interrupted
by a signal (one problem with the previous version is that package builds
whice are interrupted by ptimeout because they are stuck, leave their
working files lying around in the chroot).
* Switch to NFS v3 mounts instead of v2
* Autogenerate the version string to report in uname within the chroot,
based on the version string in the head of the CVS branch being built.
* Copy packages via cp from the NFS mount, not scp.
* Require an additional <tmpdir> argument so the client knows where its
temp directory is.
* Mount the portbuild directory readonly via NFS, and copy files that
way instead of via scp, which has too much overhead
* Don't assume the script will be called from the ports directory
* Use buildenv to set environment variables
* Set LOCALBASE and X11BASE to dummy variables to prevent the ports tree
from picking up packages installed on the host system, and patch up
the generated index at the end
operations in one central place, instead of doing them piecemeal all over
the place. This also includes the ability to customize settings per
port branch (e.g. XFREE86_VERSION)
Also, instead of hardcoding values of OSVERSION and OSREL which are going
to get forgotten again, pull them out of the source tree in ${branch}/src
so they track the head of the branch.
tangled up.
* Sort the environment variables so that all of the commented out ones
are grouped together
* Reverse sense of NOPLISTCHECK -> PLISTCHECK, since it's not an option
we want enabled by default (it causes too many build failures). This
was too easy to forget when building packages 'by hand' using the parallel
makefile.
* Copy across UNAME_VERSION into the chroot area so that uname inside the
chroot reports the desired version
* Shorten timeout period from 12 hours to 4 hours to avoid delaying the builds
unnecessarily.
* Reverse sense of NOPLISTCHECK -> PLISTCHECK, since it's not an option
we want enabled by default (it causes too many build failures). This
was too easy to forget when building packages 'by hand' using the parallel
makefile.
we want enabled by default (it causes too many build failures). This
was too easy to forget when building packages 'by hand' using the parallel
makefile.
* Display correct pathnames in build progress messages
* Add a -nocdrom option to avoid the (time-consuming) clean-for-cdrom-list
target
* Remove x11/XFree86 from dummyports since we don't want to use an old
stale version of the package which never gets rebuilt
* Add comments noting that we should check for CVS conflicts in the cvs
update scripts and exit gracefully.
we want enabled by default (it causes too many build failures). This
was too easy to forget when building packages 'by hand' using the parallel
makefile.
* Don't require the script to be called from inside the branch directory;
follow standard practise and pass the branch to build as an argument.
* Populate the chroot with the BSD.local.dist mtree file
* Add whitespace for readability, and sprinkle liberally with comments
* Comment on some inscrutable parts of the script which were presumably
put there to work around a (perceived) problem, but which I can't
understand.
of just proclaiming "error(s) occurred". This should immediately identify
which port is causing the build to break (although it's a bit of a verbose
solution).
* Update the list of ports which takes a long time to build.
* Don't use a single ports collection for every port build, indirect through
${pb}/${branch}/ports so we can have different ports collections for
each branch. Add -noportscvs option to prevent cvs update of ports
collection.
* Remove 3.x as a valid branch, add 4-exp for the 4.x experimental builds
used for testing bsd.port.mk patches
* Update usage information
decide machines which should be handed new jobs. Also, at some point someone
added a note about a possible bug in the reportload script which causes
this one to fall over.
(2) (portbuild) Change no-output timeout from 1800 to 3600. Some ports
seem to actually take more than 30 minutes inside a command.
(3) (portbuild) Add -p flag to tar when extracting bindist.
(4) (makeindex) Remove the -j flag to make index, the argument was 1 anyway.
(1) The script now assumes make(1) knows how to handle long dependency
chains properly. quickports is a list of ports that take a
long time to build by thesmelves (not ports that have long
dependency chains). The script adds several extra dependency
levels to the generate Makefile for those ports to make their
dependency chains longer.
(2) Use -R flag to cvs. Comment out the cvsup part.
(3) Preserve error messages for ports that didn't build the first
time around but did build on the retry.
(4) Record the package's timestamps in the log directory by doing
an "ls -asFlrt" in the packages/All directory.
(5) Add a little sleep after two background jobs so outputs won't
be garbled.
ftp site (specified in portbuild.conf). (It used to be implied by
-norestr.)
(2) Remove x11-toolkits/Motif-dummy from list of dummy ports, since
open-motif is now the default. Also gone is the MOTIF_OPEN
variable.
(3) Run the tar -czf of the ports tree and generation of CHECKSUM.MD5
in the background.
(4) Print the amount of time it took for the entire process, as well
as phase 1 and phase 2 of the compilations. (It is done by date
-r, so it will only work modulo 24 hours. Hopefully the build
will never go over 24 hours again.)
(5) Make symlink of the form [ae].${branch}.YYYYMMDD ->
[ae].${branch}.YYYYMMDDHH so we can have the simple "date" form as
well as the new "date+hour" directories for logs.
(6) Remove temporary make status files from /tmp that are over 60
minutes since the last modification. It was filling up the root
filesystem.
(7) Print out an "ls -lrt" of packages/All into logs/ls-lrt. This is
going to be used to evaluate make's job dispatch policies.
mtree outputs.
Also change the title of the file from "List of extra files and
directories" to "List of files and directories that do not match their
mtree description", which is more accurate.
/var/portbuild is the new designated home of the portbuild setup, and
is expected to be a symlink to wherever you choose to put the stuff.
Also, change reportload to use /var/portbuild to store temporary files.
Seems there are some bugs in the null mount code that make the files
inaccessible if you are using an NFS root.
around now.
Make a symlink "Makefile" so I don't have to "make -f ../../Makefile"
when I'm restarting an aborted build.
Print out pnohang messages along with ptimeout messages.
Count the number of jobs at the beginning of the second phase too.
Check integrity of packages at end with gzip -t. Remove the ones that
don't pass the test.
Don't try to compare list of packages if there is no prior list to
compare with.
If the -norestr flag is given, copy the packages to the ftp site.
