Commit graph

720 commits

Author SHA1 Message Date
Kris Kennaway
687f043633 Support builds using zfs and tmpfs backing store
zfs:

* Enabled by use_zfs=1 in portbuild.conf

* Populate build chroots by cloning a zfs snapshot instead of maintaining
  many duplicate copies.  In principle this is very efficient since
  everything is copy-on-write and zfs snapshot creation is almost
  instantaneous.  There might be additional overheads from building on zfs
  though.  Currently the snapshot base is hard-wired to y/${branch}@base
  but should be parametrized.  This also must be populated beforehand, e.g.
  during machine startup

* Clean build chroots by just destroying the snapshot.

tmpfs:

* Enabled by use_tmpfs=1 and tmpfs_size in portbuild.conf

* The previous md strategy of mounting in used/, populating and then
  remounting (to avoid possible races from multiple builds claiming the
  same chroot) doesn't work here because tmpfs instances are destroyed at
  umount.  I am not entirely sure the simpler approach is free from races.
2007-07-29 20:27:54 +00:00
Kris Kennaway
8d84574a4a Refresh old sample portbuild.conf with the one currently used for i386
builds.  Probably this should be better documented and commented ;)
2007-07-29 20:06:05 +00:00
Kris Kennaway
d197960279 * Clean up hacks that had to copy in shared libraries from the host in
order to run certain host binaries that were kernel-dependent.  We
  now seem to be able to rely on the /rescue versions (and killall(1)
  seems to be unused).

* Allow for ccache directories to be shared over NFS via the ccache_dir_nfs
  portbuild.conf boolean

* Populate BSD.local.dist from ${PORTSDIR}/Templates and remove population
  of BSD.x11-4.dist and support for XFree86 3.x
2007-07-29 19:56:36 +00:00
Kris Kennaway
54783169b2 Correct typo ("singal") 2007-07-29 19:48:40 +00:00
Kris Kennaway
064bda8040 * Switch from netcat port to base system nc. Replace symbolic port name
by its numeric equivalent since nc(1) does not look up the former.
2007-07-29 19:45:43 +00:00
Kris Kennaway
5f9d4f6e71 * Retire 4 build and add 6-exp2 experimental build
* Don't populate BSD.local.dist, this file is maintained in the ports tree
  now and is populated at build time
2007-07-29 19:45:05 +00:00
Kris Kennaway
ca2e9e98c7 Retire support for 4 and 4-exp builds 2007-07-29 19:43:22 +00:00
Kris Kennaway
26965e6e9c * Increase index build concurrency to reflect the new, improved pointyhat
* Add support for 6-exp2 experimental builds
2007-07-29 19:42:35 +00:00
Kris Kennaway
cc0916c8a0 * Revise the machine selection algorithm. Previously it chose the first
machine with the lowest number of running jobs.  This worked when the
  clients were all roughly equivalent, but schedules poorly when there
  are some that are much more powerful (e.g. 8-core machines vs UP machines)

* We now compute the ratio of running jobs to maximum jobs and schedule on
  the machine with lowest occupation fraction.  This populates the machines
  to equal fractions of their capacity.
2007-07-29 19:41:52 +00:00
Kris Kennaway
13e1b025a1 Use rsync -L instead of rsync -l to deal with the fact that we like to
symlink ports and src trees elsewhere.  With -l this would turn a populated
tree into a dangling symlink.
2007-07-29 19:39:30 +00:00
Kris Kennaway
b3551fde44 * Remove support for 4 and 4-exp builds and add 6-exp2 experimental build
* Only hardlink the old log files instead of anything else that might be
  in the directories
* Add comment that old logfiles should be removed as well as packages, to
  avoid duplicate versions of the same port log
2007-07-29 19:38:08 +00:00
Pav Lucistnik
87b4c7c511 Introduce straslivy.py, a fancier showrunning type of thing, written in Python. 2007-07-29 19:36:55 +00:00
Kris Kennaway
3eb8ee9622 * Remove processing of 4 and 4-exp logs now that the builds are no longer
run
* Add a 6-exp2 build which is a second experimental tree
* Remove an archaic 4.x/5.x log comparison
2007-07-29 19:36:33 +00:00
Kris Kennaway
5d0b9b3789 * Remove vestiges of procfs mounting/umounting from here; we have to
do it in portbuild from outside the jail thesedays

* Ignore /var/db/fontconfig which does not get restored to pristine state

* Save copies of master.passwd and groups and check them after the build
  for changes, to look for user/group additions that may not be correctly
  registered in UIDs/GIDs.  Future work will hopefully automatically
  check against those files and make unregistered IDs a fatal condition

* Correct logic mistake that was keeping distfiles for collection when
  the checksum mismatched
2007-07-29 19:35:00 +00:00
Pav Lucistnik
98990a91c7 - Log the failure to copy dependant package from pointyhat 2007-07-29 19:27:48 +00:00
Kris Kennaway
fd3a8f51f6 Refactor common variables from case statement for various branches.
Update default X11BASE.  Retire PARALLEL_PACKAGE_BUILD and HAVE_MOTIF which
are now obsolete.
2007-07-29 19:27:22 +00:00
Kris Kennaway
0a0b1be73c Retire this obsolete instruction file dating back to last century. We
have a more modern and complete version in the doc collection.
2007-07-29 19:21:47 +00:00
Mark Linimon
2e85070f37 Drop 4.X statistics. They are half-a-year stale. 2007-07-22 04:36:49 +00:00
Mark Linimon
df852181a7 Add 9 more cases of gcc4; remove some false positives for autoconf; add
an 'f77' reason; flag 'warnings being treated as errors' as 'cc'.  This
reduces the unknown output on i386-7 by about half.
2007-07-02 10:22:47 +00:00
Erwin Lansing
afade85a8a ARCH may be set in the environment for cross building, so only
set it conditionally.

Submitted by:	kris
2007-06-12 15:14:29 +00:00
Erwin Lansing
87b364f5fd Set default architecture to i386. 2007-06-11 21:18:46 +00:00
Erwin Lansing
05ba2446ab Make it possible to override the templates by using an environement
variable TEMPLATES.

RT:		105108 (reworked from)
Submitted by:	Curtis Jewell<swordsman@csjewell.fastmail.us>
2007-05-24 22:23:37 +00:00
Florent Thoumie
d4f0d0048a - Welcome X.org 7.2 \o/.
- Set X11BASE to ${LOCALBASE} for recent ${OSVERSION}.
- Bump PORTREVISION for ports intalling files in ${X11BASE}.
2007-05-19 20:36:56 +00:00
Renato Botelho
75b87da473 Fix addport to work correctly with ports-mgmt category.
Reported by:	miwi
Tested by:	miwi
2007-04-20 12:45:01 +00:00
Vasil Dimov
101df1b468 Use echo instead of printf because external commands cannot be executed
with very long arguments (>400000 characters).

The problem reveals for example if
/usr/ports/Tools/scripts/rmport -d print/ghostscript-gnu
is executed - it does
printf "%s\n" "... 451109 chars ..."

Spotted by:	rafan
2007-03-28 20:39:18 +00:00
Mark Linimon
43ad8e0f95 Add a column for 'build logs'. This is larger than the number of the
packages due to packages being trimmed by RESTRICTED.

While here, note that the 'missing' column will be off by the number of
duplicates in the other columns.  This happens when partial builds are
restarted.
2007-03-23 05:49:49 +00:00
Mark Linimon
9d6f970760 Workaround for the fact that 4/ports is no longer populated on pointyhat. 2007-03-09 02:43:55 +00:00
Vasil Dimov
eed3828ab1 * Run cvs update just before cvs commit to sync a potentially changed files
in the repository. Suggested by: gabor@
* Fix typo in the output
2007-02-19 07:24:36 +00:00
Kris Kennaway
7379c7f034 Increase resource limits to 400MB filesize and 2 hours CPU limit per
process.  Thankyou openoffice!

Pass in the jail IP address in the JAIL_ADDR environment variable
2007-02-18 09:05:28 +00:00
Kris Kennaway
ae0d1702e9 Default to -nocopy when using -norsync 2007-02-18 09:03:52 +00:00
Kris Kennaway
f400f2bb03 Also filter out gconf leftover files from /usr/local as well as /usr/X11R6.
Set DISPLAY to the IP address instead of using the :<...> implied form.
This is required for jails.
2007-02-18 09:03:21 +00:00
Kris Kennaway
ac6a452879 Don't require arch to be passed in on the command line, query it using
uname
2007-02-18 08:59:51 +00:00
Kris Kennaway
b21c917ce8 Don't play silly games trying to guess the arch, we are passed it in on
the command line
2007-02-18 08:59:22 +00:00
Kris Kennaway
bc7eeeee96 Increase global build timeout to 100 hours 2007-02-18 08:58:46 +00:00
Kris Kennaway
487025d7c8 Improved sanitization of the build environment 2007-02-18 08:58:24 +00:00
Kris Kennaway
da57eeaeb0 Don't require an /etc/arch file, we can obtain the information from uname 2007-02-18 08:58:07 +00:00
Kris Kennaway
d48df213cd Also attempt to clean /root/.ccache in case it is in use. When cleaning
on a machine that has use_md_swap=1, allow for the possibility of reusing
a md between builds if md_persistent=1.  This requires a patch from pjd
to support BIO_DELETE in md devices, but it is a big optimization when
it can be used.
2007-02-18 08:57:20 +00:00
Kris Kennaway
21c449a7fa Index building needs to happen before prunefailures is run, otherwise it
will completely erase the list.

When using -continue, don't append to the duds.errors, we don't want to
preserve the old cruft from a previous build
2007-02-18 08:55:39 +00:00
Kris Kennaway
56b6a0b170 Increase cleanup timeout to 5 days from 48 hours, the openoffice builds
now regularly take longer than this :(
2007-02-18 08:53:21 +00:00
Mark Linimon
e472b6f28c Fix late typo in previous commit. 2007-02-14 08:24:23 +00:00
Mark Linimon
385bfc1533 Eliminate some false positives for missing_header, sometimes seen in the
configure step.
2007-02-14 04:16:30 +00:00
Mark Linimon
b0904b6955 Combine all like terms to get another 10% speedup in processing time.
There is no change in any of the individual terms; this is merely a
rearrangement.

This change undoes what I was trying to do back in 2004 of breaking up
each individual test into a grep, for readability.  The performance of
the script has continued to suffer as new greps were added over time,
to the point where this is now a bad tradeoff.
2007-02-13 21:58:21 +00:00
Mark Linimon
f95f03392e Move two more cases into the 'uncommon' section. 2007-02-10 09:19:24 +00:00
Mark Linimon
87143d71d6 Append one character to the gcc4 case (missed during cut-and-paste). 2007-02-10 06:59:13 +00:00
Mark Linimon
8bff98fdb9 Move the uncommon cases below the common ones. A 10% improvement on large
directories, but a 5% loss on smaller ones.

No code changes (yet) except for the deletion of one duplicate
("fetch: transfer timed out" -> "fetch_timeout".
2007-02-10 06:57:53 +00:00
Mark Linimon
31e52eb01c Remove the other 2 cases of apxs. 2007-02-09 21:49:11 +00:00
Mark Linimon
4dd06f27ed Speed up by removing the following obsolete tags:
alignment apxs assert CATEGORIES cgi-bin forbidden stl
2007-02-09 20:55:43 +00:00
Mark Linimon
db8225268f Add a new case, 'gcc4', to try to catch the majority of the exceptions that
we are seeing on the latest -current runs.  Saves 40 seconds of time on
those runs, at the cost of 4 on the others.
2007-02-09 04:50:10 +00:00
Mark Linimon
4a1e61ae9c Fix up the code for amd64-6-exp; add a column to label builds that
still have processes running, and add that into the highlight logic.
2007-02-07 08:00:20 +00:00
Kris Kennaway
2c759cf3d0 Retire PORTOBJFORMAT 2007-02-01 03:28:03 +00:00