Commit graph

759 commits

Author SHA1 Message Date
Pav Lucistnik
87b4c7c511 Introduce straslivy.py, a fancier showrunning type of thing, written in Python. 2007-07-29 19:36:55 +00:00
Kris Kennaway
3eb8ee9622 * Remove processing of 4 and 4-exp logs now that the builds are no longer
run
* Add a 6-exp2 build which is a second experimental tree
* Remove an archaic 4.x/5.x log comparison
2007-07-29 19:36:33 +00:00
Kris Kennaway
5d0b9b3789 * Remove vestiges of procfs mounting/umounting from here; we have to
do it in portbuild from outside the jail thesedays

* Ignore /var/db/fontconfig which does not get restored to pristine state

* Save copies of master.passwd and groups and check them after the build
  for changes, to look for user/group additions that may not be correctly
  registered in UIDs/GIDs.  Future work will hopefully automatically
  check against those files and make unregistered IDs a fatal condition

* Correct logic mistake that was keeping distfiles for collection when
  the checksum mismatched
2007-07-29 19:35:00 +00:00
Pav Lucistnik
98990a91c7 - Log the failure to copy dependant package from pointyhat 2007-07-29 19:27:48 +00:00
Kris Kennaway
fd3a8f51f6 Refactor common variables from case statement for various branches.
Update default X11BASE.  Retire PARALLEL_PACKAGE_BUILD and HAVE_MOTIF which
are now obsolete.
2007-07-29 19:27:22 +00:00
Kris Kennaway
0a0b1be73c Retire this obsolete instruction file dating back to last century. We
have a more modern and complete version in the doc collection.
2007-07-29 19:21:47 +00:00
Mark Linimon
2e85070f37 Drop 4.X statistics. They are half-a-year stale. 2007-07-22 04:36:49 +00:00
Mark Linimon
df852181a7 Add 9 more cases of gcc4; remove some false positives for autoconf; add
an 'f77' reason; flag 'warnings being treated as errors' as 'cc'.  This
reduces the unknown output on i386-7 by about half.
2007-07-02 10:22:47 +00:00
Erwin Lansing
afade85a8a ARCH may be set in the environment for cross building, so only
set it conditionally.

Submitted by:	kris
2007-06-12 15:14:29 +00:00
Erwin Lansing
87b364f5fd Set default architecture to i386. 2007-06-11 21:18:46 +00:00
Erwin Lansing
05ba2446ab Make it possible to override the templates by using an environement
variable TEMPLATES.

RT:		105108 (reworked from)
Submitted by:	Curtis Jewell<swordsman@csjewell.fastmail.us>
2007-05-24 22:23:37 +00:00
Florent Thoumie
d4f0d0048a - Welcome X.org 7.2 \o/.
- Set X11BASE to ${LOCALBASE} for recent ${OSVERSION}.
- Bump PORTREVISION for ports intalling files in ${X11BASE}.
2007-05-19 20:36:56 +00:00
Renato Botelho
75b87da473 Fix addport to work correctly with ports-mgmt category.
Reported by:	miwi
Tested by:	miwi
2007-04-20 12:45:01 +00:00
Vasil Dimov
101df1b468 Use echo instead of printf because external commands cannot be executed
with very long arguments (>400000 characters).

The problem reveals for example if
/usr/ports/Tools/scripts/rmport -d print/ghostscript-gnu
is executed - it does
printf "%s\n" "... 451109 chars ..."

Spotted by:	rafan
2007-03-28 20:39:18 +00:00
Mark Linimon
43ad8e0f95 Add a column for 'build logs'. This is larger than the number of the
packages due to packages being trimmed by RESTRICTED.

While here, note that the 'missing' column will be off by the number of
duplicates in the other columns.  This happens when partial builds are
restarted.
2007-03-23 05:49:49 +00:00
Mark Linimon
9d6f970760 Workaround for the fact that 4/ports is no longer populated on pointyhat. 2007-03-09 02:43:55 +00:00
Vasil Dimov
eed3828ab1 * Run cvs update just before cvs commit to sync a potentially changed files
in the repository. Suggested by: gabor@
* Fix typo in the output
2007-02-19 07:24:36 +00:00
Kris Kennaway
7379c7f034 Increase resource limits to 400MB filesize and 2 hours CPU limit per
process.  Thankyou openoffice!

Pass in the jail IP address in the JAIL_ADDR environment variable
2007-02-18 09:05:28 +00:00
Kris Kennaway
ae0d1702e9 Default to -nocopy when using -norsync 2007-02-18 09:03:52 +00:00
Kris Kennaway
f400f2bb03 Also filter out gconf leftover files from /usr/local as well as /usr/X11R6.
Set DISPLAY to the IP address instead of using the :<...> implied form.
This is required for jails.
2007-02-18 09:03:21 +00:00
Kris Kennaway
ac6a452879 Don't require arch to be passed in on the command line, query it using
uname
2007-02-18 08:59:51 +00:00
Kris Kennaway
b21c917ce8 Don't play silly games trying to guess the arch, we are passed it in on
the command line
2007-02-18 08:59:22 +00:00
Kris Kennaway
bc7eeeee96 Increase global build timeout to 100 hours 2007-02-18 08:58:46 +00:00
Kris Kennaway
487025d7c8 Improved sanitization of the build environment 2007-02-18 08:58:24 +00:00
Kris Kennaway
da57eeaeb0 Don't require an /etc/arch file, we can obtain the information from uname 2007-02-18 08:58:07 +00:00
Kris Kennaway
d48df213cd Also attempt to clean /root/.ccache in case it is in use. When cleaning
on a machine that has use_md_swap=1, allow for the possibility of reusing
a md between builds if md_persistent=1.  This requires a patch from pjd
to support BIO_DELETE in md devices, but it is a big optimization when
it can be used.
2007-02-18 08:57:20 +00:00
Kris Kennaway
21c449a7fa Index building needs to happen before prunefailures is run, otherwise it
will completely erase the list.

When using -continue, don't append to the duds.errors, we don't want to
preserve the old cruft from a previous build
2007-02-18 08:55:39 +00:00
Kris Kennaway
56b6a0b170 Increase cleanup timeout to 5 days from 48 hours, the openoffice builds
now regularly take longer than this :(
2007-02-18 08:53:21 +00:00
Mark Linimon
e472b6f28c Fix late typo in previous commit. 2007-02-14 08:24:23 +00:00
Mark Linimon
385bfc1533 Eliminate some false positives for missing_header, sometimes seen in the
configure step.
2007-02-14 04:16:30 +00:00
Mark Linimon
b0904b6955 Combine all like terms to get another 10% speedup in processing time.
There is no change in any of the individual terms; this is merely a
rearrangement.

This change undoes what I was trying to do back in 2004 of breaking up
each individual test into a grep, for readability.  The performance of
the script has continued to suffer as new greps were added over time,
to the point where this is now a bad tradeoff.
2007-02-13 21:58:21 +00:00
Mark Linimon
f95f03392e Move two more cases into the 'uncommon' section. 2007-02-10 09:19:24 +00:00
Mark Linimon
87143d71d6 Append one character to the gcc4 case (missed during cut-and-paste). 2007-02-10 06:59:13 +00:00
Mark Linimon
8bff98fdb9 Move the uncommon cases below the common ones. A 10% improvement on large
directories, but a 5% loss on smaller ones.

No code changes (yet) except for the deletion of one duplicate
("fetch: transfer timed out" -> "fetch_timeout".
2007-02-10 06:57:53 +00:00
Mark Linimon
31e52eb01c Remove the other 2 cases of apxs. 2007-02-09 21:49:11 +00:00
Mark Linimon
4dd06f27ed Speed up by removing the following obsolete tags:
alignment apxs assert CATEGORIES cgi-bin forbidden stl
2007-02-09 20:55:43 +00:00
Mark Linimon
db8225268f Add a new case, 'gcc4', to try to catch the majority of the exceptions that
we are seeing on the latest -current runs.  Saves 40 seconds of time on
those runs, at the cost of 4 on the others.
2007-02-09 04:50:10 +00:00
Mark Linimon
4a1e61ae9c Fix up the code for amd64-6-exp; add a column to label builds that
still have processes running, and add that into the highlight logic.
2007-02-07 08:00:20 +00:00
Kris Kennaway
2c759cf3d0 Retire PORTOBJFORMAT 2007-02-01 03:28:03 +00:00
Erwin Lansing
f3f3875210 Drop the build on RELENG_4 2007-01-30 14:49:21 +00:00
Vasil Dimov
92615ddfc4 A bunch of cosmetic changes - add one empty line to make the output more
readable and use double quotes everywhere.
2007-01-26 14:25:39 +00:00
Vasil Dimov
a62c886fca Make output messages more useful 2007-01-26 10:19:23 +00:00
Vasil Dimov
0d451e0598 * Make it possible to skip existing dependencies and ports references in LEGAL
and continue with removal anyway. Requested by miwi@
* Pipe dependencies information (if any) through a PAGER because INDEX lines
  are very long and hard to read when wrapped
2007-01-19 15:49:49 +00:00
Mark Linimon
c43f8047ea Eliminate more false positives for install_error. Very noticeable on
latest gcc test build.
2006-12-25 18:09:34 +00:00
Vasil Dimov
59a026b0f1 Query Gnats for PRs by using the query-pr command on freefall instead of the
web-interface becase the later is not reliable.
2006-12-20 13:18:25 +00:00
Vasil Dimov
e2f1d375a7 Add check for existing non-closed PRs related to the port being removed.
If any are found then ask if the port should be skipped from removal or
removed regardless of the PRs.
2006-12-13 07:32:58 +00:00
Dirk Meyer
3ac2f4749e - add check sysvipc
happens when security.jail.sysvipc_allowed=0
2006-12-03 17:59:09 +00:00
Mark Linimon
2830a3d3df Work around the fact that 5-exp, these days, is really another 6-exp,
except in disguise.
2006-11-25 15:51:02 +00:00
Erwin Lansing
596b15845d Replace a simple call to localtime() to strftime() to include the
timezone.
2006-11-23 14:04:52 +00:00
Erwin Lansing
bdbae3ea52 eik@ has been inactive for quite some time and as I am running
these scripts on the cluster, transfer maintainership of these
to myself.
2006-11-23 12:16:59 +00:00
Erwin Lansing
6e65d26f6e Include a time when the script was started in the mail templates.
This script can sometimes take several hours to run on builder,
and thus leading to confusing of why it still reports an error
that was fixed in cvs some time ago.  Including the time when
the ports tree was updated should reduce some of this confusion.
2006-11-23 11:59:04 +00:00
Vasil Dimov
9125407be5 Add deprecated reason to the output of -F 2006-11-02 09:20:10 +00:00
Vasil Dimov
e1bcb99398 * Create custom commit message template and populate it with the ports
to be removed, possibly with the expiration date and deprecated reason
* If port is not marked for expiration than put "Removed" in ports/MOVED
  entry instead of "Has expired"
* Implement -a option to remove all expired ports
* Ask if the cvs diff output should be recreated/reviewed again thus
  giving the committer a chance to edit files by hand and view diff
  results afterwards
* Cosmetic changes
2006-11-01 09:57:36 +00:00
Vasil Dimov
1854c2ed1d When checking for dependencies of foocategory/barport also find constructs
like .../barport/something e.g. expect either space, slash or eol after
port's name
2006-10-05 10:40:29 +00:00
Mark Linimon
3c741dc25b Since we already have the data from the grep, go ahead and make the 'latest
log' cell a link to it.  My weak awk/sh skills are evident here, but it
does the job.
2006-09-23 07:44:34 +00:00
Mark Linimon
c5e446540c Add links to cvsdone and INDEX files. Add some whitespace. 2006-09-23 05:39:53 +00:00
Vasil Dimov
800d19da74 When checking dependencies also look for portname references in Mk/*.mk
in addition to */*/Makefile* and ignore references in comments
2006-09-14 09:13:39 +00:00
Kris Kennaway
81aca5b157 * Back up duds to duds.old before, not after we create the new one
* Add more XXX comments for future work

* Only record a cvsdone timestamp if we updated cvs

* When building with -trybroken, it's safe (and desirable) to run the
  prunefailure script

* Reorganise a few things for better parallelism

* Instead of keeping a duplicate copy of the previous logs and errors
  under bak/, just store a symlink to the archival location

* When doing an incremental build, also cycle out the old logs to avoid
  broken links on the website (the logs from the previous build are
  removed until the packages are rebuilt).  Use cpio to create
  hardlinked copies of the previous logs.  XXX when these are bzipped
  by cron to save space the links will be broken and it might actually
  take more space.

* Don't bother bunzipping old logs, now that the processlogs scripts
  can handle it.  This was a waste of time anyway since they'd all be
  rebzipped by the next nightly cron job.

* When the build is complete, stash a copy of the restricted ports in
  bak/restricted/ before deleting them from packages/, and restore from
  here when doing an incremental build to avoid needlessly rebuilding
  them each time.
2006-09-14 05:43:23 +00:00
Kris Kennaway
b7cf9de7e5 * When calculating the IP address to use for the jail, don't use
127.0.0.0 or 127.0.0.1

* Don't use chroot to mount the linprocfs, or it won't be visible
  within the jail.
2006-09-14 05:33:12 +00:00
Kris Kennaway
8dcaefb8ab * Support bzipped logfiles
* More robust extraction of the header values (copes with the addition of
  the build uname to the header)
2006-09-14 05:30:04 +00:00
Kris Kennaway
db6e3408a5 * If .force exists, rebuild index unconditionally
* Support .log.bz2 files

* Clarify that BROKEN ports are not frequently rebuilt so probably
  won't appear
2006-09-14 05:26:08 +00:00
Kris Kennaway
d35698c80e Correct typo in previous commit 2006-09-14 05:24:31 +00:00
Kris Kennaway
57407fb769 * Drop alpha support
* Increase sparc64 build timeout to 24 hours (we have so few build
  machines that we cannot afford to tie them up for longer)

* Increase other arch build timeout to 100 hours (hello openoffice!)

* If we successfully build a formerly broken package, touch errors/.force
  which will kick off a rebuild of the html files
2006-09-14 05:24:09 +00:00
Kris Kennaway
2df0e38d60 * Retire 4-exp builds and add support for 6-exp.
* Use a generation number for the bindist tarballs, with compatibility
  symlink.  Eventually we'll use this to avoid building in a "stale"
  chroot (i.e. populated by old world).

* Don't bother running ldconfig on i386, it is evidently not needed since
  the other arches work fine without it
2006-09-14 05:21:49 +00:00
Kris Kennaway
3c9d082dd2 * Silence gconf spam from the mtree checks (for now)
* Don't try and mount/umount procfs, it won't work when we build inside a
  jail.

* Report the uname -mr of the build environment, to ease confusion of
  people reading the error logs by mail.
2006-09-14 05:19:09 +00:00
Kris Kennaway
cef3a38ffb * Use SRCBASE for source location
* The 5.x variant for populating /etc does not work on 6.x and above, and
  vice versa.  Conditionalize old and new methods.
2006-09-14 05:17:12 +00:00
Kris Kennaway
72ac1bdadb Go back to force unmounting because not doing so was too unreliable. 2006-09-14 05:14:19 +00:00
Kris Kennaway
69e1fc873f Silence the clean-chroot invocation 2006-09-14 05:13:50 +00:00
Vasil Dimov
f5a601c10a Remove grep's -w switch from the dependency check since it misses some
dependencies. This needs some more polishing but for now it is better to
emit false positives than skip dependencies.
2006-09-12 08:35:16 +00:00
Mark Linimon
cb217a1914 Get rid of multiple false positives for "arch" due to compiler warning;
fix true positive for "runaway_process".
2006-08-20 18:38:08 +00:00
Mark Linimon
8765aa08c8 Catch a few more new cases of linker error; reclassify most of the linker
errors seen on -CURRENT as "compat6x" e.g. "someone needs to create a
compat6x port and then make it a dependency here."
2006-08-18 19:23:19 +00:00
Erwin Lansing
50513d5d52 Add support for DESTDIR part I.
This commit should largele be a NOOP as it only adds support
for DESTDIR undefined. This does allow us to start testing
ports with DESTDIR set, but this is as of yet not supported.

Although this has been extensively tested on pointyhat, this
is a very intrusive change and some cases may have been
overlooked. Please contact Gabor and me if you find any.

PR:             100555
Submitted by:   gabor
Sponsored by:   Google Summer of Code 2006
2006-08-04 12:34:50 +00:00
Mark Linimon
4ab6687b05 Catch another case of 'disk full'. 2006-07-31 20:05:12 +00:00
Erwin Lansing
4f52cd8ba6 Change locking from per architecture to per architecture and branch.
Especialy i386 has enough horsepower to run two branches simultaniously.
This change has been on pointyhat for some time.
2006-07-30 14:40:43 +00:00
Shaun Amott
068a568fca Add an option to the addport script: -b requests that any files in
${FILESDIR} which look like patches be treated as binary files. This
prevents RCS tags in patch fragments causing a problem for CVS.

Approved by:	garga (maintainer),
		ahze (mentor, implicit)
2006-07-14 01:49:35 +00:00
Mark Linimon
8465bb92e6 Add the cvs checkout date so people can see how long the runs take; add
a comment about openoffice.org*; generalize a bit.
2006-07-13 03:52:28 +00:00
Mark Linimon
48eea8eed4 Make it a little more robust. 2006-07-09 19:02:14 +00:00
Mark Linimon
8ccf5fa29d Convenience script to tersely display all currently running package builds. 2006-07-09 18:16:32 +00:00
Mark Linimon
8ae57c1526 'portcomment' appears to be obsolete. 2006-07-08 21:38:55 +00:00
Mark Linimon
ed5aedd9d3 Fix the disk_full rule. 2006-07-08 20:29:41 +00:00
Mark Linimon
7d3e7fee38 Add the -exp builds down at the end. Probably only of interest to portmgr,
so do them separately in order not to break up the ease of browsing the
most important stats, above.
2006-07-08 04:09:42 +00:00
Mark Linimon
8b9d5c5231 White space adjustment after refactoring. 2006-07-08 03:33:51 +00:00
Mark Linimon
655da76c6d Refactor. No code change (yet). 2006-07-08 03:33:06 +00:00
Mark Linimon
349e8952a4 Add some color to the display. 2006-07-06 04:54:53 +00:00
Mark Linimon
7de73dca03 Fix CVS tag. 2006-07-05 23:30:42 +00:00
Mark Linimon
c2c0a5bbad Refactor. No code changes yet. 2006-07-05 23:29:45 +00:00
Mark Linimon
34d3330d94 Add a column for last-date stamp and clean up the formatting a bit. 2006-07-05 23:17:53 +00:00
Vasil Dimov
ed290037f5 Escape characters (like +) that may appear in ports' names and
break sed -E regular expressions
2006-07-05 08:31:49 +00:00
Renato Botelho
0bc560fc74 When automatic commit log is generated, change @ to " at " on submitter email
to prevent spam
2006-07-04 23:12:10 +00:00
Mark Linimon
c693c172bc Handle the case where the size of INDEX is zero (e.g. during new builds). 2006-06-28 22:46:18 +00:00
Mark Linimon
e4e1eb4a82 Fix edit-o. 2006-06-28 04:27:43 +00:00
Mark Linimon
a2947a3c30 Creates an HTML file with tables for each architecture which summarize the
number of packages built vs. packages that failed, along with some other
related information.
2006-06-27 05:31:32 +00:00
Mark Linimon
ef2b06d533 - Remove false positives for 'makefile' (and possibly a few others).
- Remove obsolete explanations which are no longer seen, for speed:
   ELF, MOTIF, MOTIFLIB, X_manpage, awk, bison, ffs_conflict, forbidden,
   getopt, getopt.h, imake, lc_r, malloc.h, pod2man, sed, stl, soundcard.h,
   texinfo, union_wait, values.h

 - Add more cases to: arch, bad_c++, compiler_error, depend_object,
   install_error, linker_error, mtree, perl5

These changes reduce many dozens of false positives; add a few dozen
true positives; and for certain directories, improve the speed about 10%
(a few drop by 15%).

It turns out that the performance issues are mainly due to the multiple
greps.  If performance is an issue we need to go back to the moderately-
unreadable, everything-on-one-line paradigm.  Before that happens, I would
like to experiment with some refactoring, so that the patterns are built up
in the shell line-by-line, so you could still be able to read it.

Tested on:	pointyhat
Hat:		portmgr
2006-06-24 19:51:53 +00:00
Erwin Lansing
7fbbcb4402 MFP4:
Actually use ${SCP_DEST_TMP} to fix the case where
        SCP_DEST_TMP != /tmp/
2006-06-22 18:52:57 +00:00
Erwin Lansing
c090c55e39 MFP4:
When copying INDEX to the server, copy it first to a staging area and
first then to the real location. The copying can take long enough for
users to get a truncated file when downloading during the upload.
2006-06-19 15:44:16 +00:00
Renato Botelho
3cf8cfc8cd - Change addport script to do the job locally instead connect to freefall every
time to add a new module. If you want to still use old way, just use
  "-M freefall.FreeBSD.org" option
- Take addport maintainership
- When modulesupdate fail, ask user to retry
- Change modulesupdate to work fine with addport

Approved by:	will (maintainer)
2006-05-16 15:44:29 +00:00
Vasil Dimov
12dda3240e Add support for specifying ports to be removed as directories on the
filesystem (either absolute or relative) in addition to `category/port'.

PR:		ports/96649
Submitted by:	Cheng-Lung Sung <clsung@freebsd.org>
Reworked by:	vd
2006-05-02 11:31:54 +00:00
Vasil Dimov
8529c41f37 Improve dependency check:
When removing category/port - look if other ports' Makefiles contain
`/port' rather than `category/port', since the later misses things
like `${.CURDIR}/../port'
2006-05-02 09:16:10 +00:00
Vasil Dimov
b2a9550574 Add support for removing multiple ports which depend on each other 2006-04-17 08:08:37 +00:00
Kris Kennaway
38bfaf4774 Explicitly mount the NFS images ro, instead of relying on the server to
export it ro.  This saves thrashing the server if we accidentally try to
rm -rf a NFS mount (e.g. because a chroot cleanup failed).
2006-04-16 02:29:36 +00:00