Changes:
- Clean up language of the zsh completion for bulk -c and -C.
- Manpage fixes
- jail: Fix handling of games distribution for recent CURRENT.
- testport: Show error when unable to extract pkg
- jail -uc: Support a SETNAME-src.conf
- Add 'builder' start/stop hook. Params: id mnt
- Add a check to see if id(1) works in the jail.
- HTML: Show proper percentage in progress bar.
- Fix creating cpdup ignore files in jail if cpdup is not used (ZFS).
- Cross-building updates
The next update will be 3.1-RC1.
Changes:
* The HTML interface is now fixed for file:// and direct access. This
currently only works for the build directory. The jail-level and top-level
do not yet work, but may in the future.
* Fix usage over NFS when root does not have access to current directory.
* Fix performance for qemu/native-xdev by not using /bin/sh for pkg(8)'s
ABI_FILE
Changes:
* Minor display fixes
* poudriered cleanups
* ports -l: Support -n like jail
* Add a 'jail stop' hook called right before jail is shutdown
* Auto fix shadowed files in repository that are owned by pkg(8)
* jail -cx: TARGET/TARGET_ARCH fixes
Changes:
* Fix another race in balance_ool() causing crashed builds
* More color fixes
* jail -u: Fix detection of invalid jails
* Better handling of crashed builds
* Fix incremental rebuild for pkg/pkg-devel
- Poudriere's home has moved to https://github.com/freebsd/poudriere/wiki
Changes:
* Fix some error handling with the recent races fixes, which led to them
still crashing.
* Display some nicer errors in some cases rather then just exiting.
* jail -m null: Fix /etc/login.conf.db being stale after deleting
the jail.
* Fix a possible case of dependency calculation errors being lost.
Changes:
* HTML: Fix NaN showing in a case
* Fix handling of SIGPIPE for cleaning up.
* Improvements to cleanup handling such that recursive errors will not
stop the cleanup process.
* HTML: Add an apache example
* testport: Allow tee(1) more time to flush buffer before killing it off.
* bulk: Remove the use of tee(1)
* bulk: Fix several races leading to "stuck_in_queue" and "Unknown stuck
queue bug".
* jail -m src=: Also copy in the src tree now. .git and .svn are ignored.
* bulk: Fix high CPU usage/spinning when the queue is waiting on the last
jobs to finish.
* Remove use of stripcolors pipe which used far too much CPU.
* bulk: Fix a case where computing deps was not delaying errors until
the end of deps calculation was intended.
Changes:
* testport: Detect invalid origin in -o
* Still symlink logs to errors/ for 'stuck_in_queue'
* Add a jail:start hook. Can be used to customize the jail for builds on
startup.
* jail -u: Stop trying to nullmount /usr/src on top of itself, and
fix /compat warning.
* bulk -t/testport: Only show NEEDED libraries using readelf(1) rather than
ldd(1) which was showing indirect library dependencies.
* bulk -R / NO_RESTRICTED: Show which files are being removed by
clean_restricted
* Fix showing missing MOVED as a warning
* jail -i: Fix to not require a ports tree
* bulk -i: Fix installation of ports-mgmt/pkg
Changes:
* ports -u: Fix with git
* bulk: Some dependency failures are now delayed rather than immediately
fatal. This allows identifying more build problems rather than
needing to iteratively run bulk to continue finding more.
* Add a note that DEVELOPER is ignored without bulk -t/testport.
* Also remove DEVELOPER_MODE for ports not being tested. (pkg uses it and
causes fatal pkg-lstat errors on bad plist)
* Only show boost priority if actually building the packages listed
* Add PKG_REPO_META_FILE passthrough for repo meta file for pkg-repo. Not
currently useful so undocumented. It will be more useful when pkg-repo -m
learn mores metadata options and there is a client-side way to view the
metadata.
* Fix "ln: ...Latest/: File exists" errors.
* Allow setting ALLOW_MAKE_JOBS_PACKAGES blank
* Move ALLOW_MAKE_JOBS_PACKAGES to -v
* Fix newly built pkg not being used to build the repository, rather than
using the one that existed at the start.
* Add TRIM_ORPANED_BUILD_DEPS to override behavior added to trim the queue
of build dependencies that are not needed for the build but were found
and possibly deleted from the repository.
* Fix testport of ports-mgmt/pkg. It has not worked in a very long time, if
ever.
* Allow securelevel>=1 if using USE_TMPFS=all.
Changes:
* HTML: Fix printing of "Remaining" on the jail page
* Build reason: Change "missing LDFLAGS" to "missing_LDFLAGS" to have a one
word reason
* Update example nginx config for 1.6: nginx 1.6 now includes
application/json in default mime.times
* Add USE_PROCFS and USE_FDESCFS to allow disabling requirement on them
Changes:
* bulk: Always clean restricted packages when done building if NO_RESTRICTED
is set
* bulk: Don't rebuild unneeded deps which were not specified on the cmdline.
NOTE: This means that partial builds will not create a consistent repo of
other existing packages. I.e., running bulk PORT1 PORT2 and then
running bulk PORT1 will make it only create a consistent repo
for PORT1. Always build with the ports you expect to be there.
* Fix 'msg_error' not defined errors.
* pkgclean/distclean: Consider hardlinks for size calculation
* distclean:
- Fix some false-positives
- Support multiple -p options
- Parallelization improvements
* HTML:
- Fix skipped column sorting
- Pkg/Hour: Don't show 'Infinity' for older builds
- More column width tweaks to fix word breaking
- Fix skipped filter
* jail:
- Better support for native builds on powerpc and other archs,
-a now accepts TARGET.TARGET_ARCH or TARGET_ARCH.
- Add several new methods (see poudriere.8). Note that some
of these may change a lot before release.
* Null import an existing jail: -m null -M path
* Install from existing src/obj dir: -m src=PATH
* Install from tarball: -m tar=PATH
- Always run -u after -c to get latest freebsd-updates
- Fix -r and give hints on next steps
- Try to modify the jail on creation less and do it during build
startup instead.
* Updates to log processing to offset common "cluster" reason
* Better QEMU support
* Better deadlock detection analysis
* testport: Add -w to save workdir just as bulk has
* Remove TRACK_BUILDTIMES
* Sign the pkg.txz for pkg(7) when using SIGNING_COMMAND
* portsnap interactive support
* Try to handle long mount points better with MNAMELEN limit of 88
by shortening many internal paths and hashing the mastername when
required.
* Support poudriere.d/[jail-][ptname-][setname-]poudriere.conf (this support
is a bit unreliable still)
* status: Add -r to show results of a build
* Remove DURATION_FORMAT and always show HH:MM:SS to avoid wrapping at
24 hours back to 00.
* Improvements to internal timestamp utility
* Fixes for update_stats lock crashes
* Fixes for 'make clean' crashes which typically crashes a builder
* Avoid restarting jail on build start to help with lock contention
- Fix build on 9.1 which does not have F_DUPFD_CLOEXEC
Changes:
* status/jail -l: Fix sort(1) error on <10
* Fix crash with SIGINFO done too early in build
* Prevent appending the same make.conf multiple times
* Fix (-s) showing in sanity when pkg is missing
* Fix jail -u
* jail -u: Name the jail with -default so it can be stopped
with jail -k.
* Add ALLOW_MAKE_JOBS_PACKAGES which lets the named package globs
to always run with MAKE_JOBS regardless of ALLOW_MAKE_JOBS. Default
is set to allow 'pkg' and 'ccache' to build with MAKE_JOBS as they
typically holdup the rest of the build.
* No longer run 'make clean' before building a port as it is not needed
due to always using a WRKDIRPREFIX
* Fix -a to work with bad-styled category Makefiles
* sem(4) is only needed on <9
Changes:
* Reduce jail restarting for networking access by now always starting
2 jails. One with networking support and one without. This allows
not constantly restarting jails to enable/disable networking, which
causes a lot of jail lock contention.
* Color support has been added. This is disabled by any of:
1. poudriere -N <sub-command>
2. stdout or stderr NOT in a TTY (piped)
3. USE_COLORS=no in poudriere.conf
* Add DURATION_FORMAT and BUILDTIME_FORMAT to allow changing time
display and default build name.
* madvise(MADV_PROTECT) the main poudriere process to allow it to cleanup
on failure from out-of-swap.
* Running commands as non-root will *auto queue* them to poudriered
if it is running. See rc.d/poudriered.
* HTML/Web major changes:
- Stop polling the server once the build is done.
- There is now a top-level and jail-level page. The top-level only
shows the *latest* build for each jail-ports-set combination.
The jail-level only shows for a specific jail-ports-set combination.
These pages will only be updated with builds done after upgrading.
This may change before 3.1 release using a statsd to populate with
all historical builds as well.
- The main build page has been majorly reworked.
* It no longer has 2 columns which overlap each other when zoomed in.
The site is now much more responsive to zooming and mobile viewing.
* More links to logs and elapsed times are shown
* More system and build stats are shown such as "Impulse" which is
the rate of package building over the last 2-10 minutes. This is
done in browser/js currently so will reset when reloading the page.
- The layout for httpd setup has changed. Now instead of duplicating
the html/assets into every build dir, the html dir from
/usr/local/share/poudriere/html/ should be served directly
and the /data dir should be symlinked or aliased to the equivalent
/usr/local/poudriere/data/logs/bulk/ dir on your system. See
/usr/local/share/examples/poudriere/nginx.conf.sample for an
example nginx setup.
** Note that this is still being improved for 3.1 and may undergo some
backwards-incompatible changes before release.
* status: Major changes.
- By default only show running builds
- Use -a to show all builds, not just latest
- Use -f to show finished builds, only latest
- Add -c for a more compact format
- Displaying the SIGINFO/builder output now requires -b.
- -B/-j/-p/-z can be used to filter the output.
* Show buildtimes in output now.
* TIMESTAMP_LOGS has been improved
* poudriered is no longer installed in sbin/
* Fix MAX_MEMORY so it actually works. Note that this may break openjdk
building.
* Fix incorrect timeout message shown during install phase
* jail -l, ports -l, status: Auto format width of columns to output
* jail: Add back -i support to show information about a jail
* Improvements to fork less often
* testport: Now default to -n. Add -P to test with custom prefix.
* testport -I: Fix longstanding bug of displaying wrong jail -k hint.
* qemu-mips support
* ports: Allow -B with git method
* All the fixes from the 3.0.16 release as well:
* Disable DEVELOPER from make.conf; Poudriere will automatically
set it when appropriate. This prevents testing of port dependencies
that were not requested to be tested.
* distclean: Prevent removing all files by accident.
* Having PKG_REPO_SIGNING_KEY set but the file missing is now fatal.
* kern.securelevel >=1 is now fatal.
* Fix 'kill: No such process' warnings with older /bin/sh (8.x)
* Fix crash on 8.3 due to bug with $(()) handling of _vars (fixed in
base r234001)
* Fix case of packages always rebuilding with "changed options" when
the port has an OPTION defined multiple times (such as fixed in
ports r352512)
* Fix 'Terminated' warning showing at shutdown with older /bin/sh (fixed
by r218105 but never made it to stable/8)
Changes:
- Queue is now priotitized by the dependency-chain
length and not by number of direct dependencies.
- Add PRIORITY_BOOST in poudriere.conf which takes
PKGNAME globs to boost up to the top of the queue.
Example:
PRIORITY_BOOST="apache-openoffice* pypy* mysql-workbench*"
- bulk/testport: Add -k to keep going on test failures.
The failures are stll considered failures but do not skip
further builds depending on that port.
- bulk -at: This now defaults to -k
- Leftovers and Orphan checking now uses the scripts in
ports so we can update them frequently without a poudriere
release and allow Tinderbox to use the same checks. [1]
- This fixes many false-positives with orphans
- Now @dirrm are not shown if they are already in the plist
- Hook-up stage-qa into the tests and remove our redundant
checks.
- testport -i: Fix common double-install error
- Fix leftover mtree not showing all results when a directory
is modified. I.e., changed user and group, only one of which
was showing.
- bulk -a: Don't consider missing origins as fatal.
- Rework distfile handling to be more atomic
- Fix many race conditions with cleanup after
port build, especially with skipping ports.
- Optimize port cleanup after builds
- Speedup compute_deps
- Fix crashed build detection (commonly shows as
"unknown stuck queue bug")
- status: Fix regression causing it to not work with multiple
builds at once
[1] It's possible we may change the format of the scripts in ports
once Tinderbox integrates with them. If this happens you will
need to update poudriere-devel.
- Cleanup Makefile to use pkg-plist [1]
Changes:
- make_index fixes (use ports/portsnap version now)
- Documentation updates
- Use autotools for build
- All incremental messages are in a consistent format now.
- New daemon poudriered using libucl with fine-grained permissions, but
not yet hooked up or documented. It will be hooked up to poudriere-queue
and documented soon.
- Add an nc(1) wrapper to work around src change r249499.
- status: Add -H for scripted mode
- status: Show URL if available
- Allow to specify revision when running poudriere jail -u with svn method
with -t
- bulk: Add -i and -I for interactive testing of all specified ports at once,
requires ports r348859 change to Mk/bsd.port.mk.
- Report modification of directories during leftover check (owner or
permission changed).
- Fix ipv6 only usage
- Ensure /sbin and /usr/sbin are in PATH when running
- Remove trailing slashes from listed_ports.
- Exit quicker after ^C during packaging phases by killing pkg-static
- Fix child process cleanup logic to only sleep if child still running.
- Always dismount filesystems and try cleaning up when exiting and in jail -k
- For stage_orphans, ignore modified files (such as nobody vs root)
- testport -I: Mark cleanup done, not as recursively failing
- testport -i: Use a local.conf instead of PACKAGESITE
- testport -I: Suggest same login method which is more sane/clean than exec
/bin/sh
- options: Require dialog4ports as it runs from host and
jail may be for a different ARCH
- Ignore parents of home directories too during leftovers check
- Restrict ZFS on 9.0/9.1 due to known deadlock
- bulk: Parse MOVED for given port build list
- Add support for svnlite and overriding SVN_CMD
- Show warning when jail is newer than host
- Show host/jail OSVERSION in build logs
Submitted by: Johannes Jost Meixner <xmj@chaot.net> [1] (based on)
PR: ports/187373 [1]
Changes:
* bulk: Support port origins ending in /
* ATOMIC_PACKAGE_REPOSITORY: Fix bulk/testport when running on 9.1,8.4,8.3
as they do not support mv(1) -h to rename a symlink. This is fixed by
adding a custom rename(1) that just calls rename(2) on the arguments
to ensure an atomic rename(2) is done.
- Update LICENSE format
Changes:
- Add a BUILDER_HOSTNAME that can be set to define the hostname
for package building
- Add PRESERVE_TIMESTAMP (default off) to allow preserving timestamps
in svn checkouts
- Auto clean packages when their version does not match the jail's version
- jail -l: Add -n to only show name
- Ensure /sbin:/usr/sbin is in PATH
- jail: Add -r to rename jails (jail -r newname -j oldname)
- Fix ipv6-only usage
- Require sem(4) kernel support as some ports require it
Changes:
- Fix hang in sanity check
- Fix race causing "unknown stuck queue" after ports finish building
- Fix PARALLEL_JOBS always set to prepare number
- Cache fixes
- cpdup -X usage fixes
- bulk: Avoid rollback on first startup (speeds up startup)
- Cleanup some error handling with parallel processes,
errors will now cause a 30 second wait for other processes
to cleanly exist instead of sending a SIGKILL right away.
- Fix KEEP_OLD_PACKAGES_COUNT when package root
is a symlink
- Ignore home directory of users created by ports during
leftover check
- Load MOVED on startup and use it to track changed origins
in existing packages, which fixes more invalid dependency errors
and "new dependency" deletions in sanity check.
- Checksum is now done as root in case it needs to refetch
- testport: Remove extra -J description
Changes:
* Fix sanity checking not properly recursively deleting dependencies.
* bulk/testport: Add PREPARE_PARALLEL_JOBS and -J n[:P] where P is
used for pre-build parallel jobs which may be more IO-bound
* Typo fixes in manpage
Changes:
* Add new MAX_MEMORY and TMPFS_LIMIT to allow limiting memory
* bulk: Show (-C) when used when deleting existing packages
* bulk: for install,deinstall,package,extract limit execution
time to 1 hour to avoid endless loops like devel/meta-cvs in
r337304
* Allow BUILDNAME to be overridden in poudriere.conf, such as:
BUILDNAME=$(TZ=UTC date +%Y%m%dT%H%M%SZ)
Changes:
* Package builds are now atomic. They will not update the
package repository directory until the build finishes.
This can be disabled with ATOMIC_PACKAGE_REPOSITORY (default yes).
This will convert the repository to a new format on first build.
If a build fails, all of the successful packages are retained
for the next attempt, but the old packages do not get modified.
* Disabling COMMIT_PACKAGES_ON_FAILURE (default yes) will have it
not touch the repository if the build fails; the package repository
will only be updated once a build succeeds.
* bulk now has a dry run mode (-n). Requires ATOMIC_PACKAGE_REPOSITORY
* Add KEEP_OLD_PACKAGES which will keep KEEP_OLD_PACKAGES_COUNT
number of old repositories. This can be useful for rollbacks
or bisecting. Requires ATOMIC_PACKAGE_REPOSITORY
* Add RESTRICT_NETWORKING (default yes) which can be used to disable
the network restrictions outside of 'fetch' phase.
* Add new PORTTESTING_FATAL (default yes) which can be disabled so that
failed ports (leftovers) do not cause ports depending on that port to be
skipped. This is useful for QAT.
* Fix stage orphans excluding /var/run/PORTNAME
* Stage orphan fixes for ports using kdeprefix
* /proc and /compat/linux/proc now excluded for leftovers
* nmtree fixes for head
* Misc performance improvements
* build_fs_violation was broken
* Poudriere now requires pkg-1.2 in its ports tree
Changes:
* pkgclean: Fix to properly remove correct duplicated package
* Only markfs prepkg for bulk/testport, not pkgclean
* jail -l: Fix crash if no jails defined
Changes:
* pkgclean: Detect and cleanup duplicated packages due to PKGNAME
renaming
* Misc typo fixes
* ports -m git
- Use --depth=1 to not download all history for the repository.
- Support -v to show output
Changes:
* poudriere.8: Document that -j is required for bulk
* bulk/testport: Delete packages that have changed PKGNAME during startup,
which will avoid pkg-repo duplicate package warnings/bugs. This will not
clean up existing duplicates, just fix new ones going forward.
* bulk: Fix start hook starting before validations
Changes:
* Fix random crashes with parallel process handling
* sanity check is now parallelized for a big speedup
* Pkg deleting during startup has been optimized further
to avoid a bottleneck during parallelization
* jail -cu: Disable CCACHE on 10
* Manpage typo fix
* testport/bulk -t: Fix staging absolute link checking with files
that have spaces
* testport: Fix so that MAKE_JOBS runs by default
* bulk -t: Fix so that MAKE_JOBS does not run by default
* testport: Fix so that the port being tested shows in the web ui.
Note that this is still in data/logs/bulk/
Changes:
* Loopback address fixes
* Fixes for when running poudriere in a jail (do not use fdescfs or
devfs to hide/unhide devices). PATCHED_FS_KERNEL overrides this
* Prevent SIGTSTP from reaching child builders
* jail -c: Fix -m ftp-archive for 9.0
* jail -cu: Add svnlite support and SVN_CMD override
Changes:
* testport: Fix /prefix error when using pkg_install
* Fix sample bulk.sh hook for stats
* Don't set PREFIX when using 'testport -n', which avoids
hiding cases where bsd.port.pre.mk are needed
Changes:
* Add BUILD_AS_NON_ROOT and set to 'no' by default. Set to 'yes' to
attempt to build ports as non-root. Ports may need USES=uidfix to
build, or NEED_ROOT=yes to mark as requiring root
* Check for absolute symlinked files when staging
