names. The character '@' is illegal in such names and must be converted to
a legal character same as [-+./].
remobve post processing of "pkg check -dn" output, which was a left-over
from pre-PKG_NG times.
Reported by: koobs (invalid @), tatsuki_makino@hotmail.com (pkg check)
Approved by: antoine (implicit)
The patch attached to PR 181450 did no longer apply cleanly, therefore I
edited portmaster to follow the intent and in part the letter of the patch.
This patch should not affect port building in any way. I have no use for
the feature (and I'm still considering to completely remove support for the
installation of packages in portmaster), and therefore did not test that
it performs as expected by the submitter.
PR: 181450
Submitted by: rozhuk.im@gmail.com
Approved by: antoine (implicit)
origin and package name (sans version), as was the case for e.g. the
lang/cython3 port, which was moved to lang/cython@py36 with a package
name change from cython3-$version to py36-cython-$version.
Approved by: antoine (implicit)
being identified and replaced when the new package is installed.
Add library dependencies to the list of build dependencies to consider.
Reported by: Robert Huff
Approved by: antoine (implicit)
Changelog:
pfind:
Add -N option, finds port that provides given package, including flavor
upgrade-ports:
Fix pkg -> port conversion for ports with one flavor
Fix backup package path generation
Fix sorting of flavored ports
Improve sanity checking procedure
Work directory can have different name depending on flavor,
parametrize its usage
Interactive selection fixes
previous version's problem and I forgot to
Fix shbang: Remove the -x that had been added for debugging.
Reported by: (Walter Schwarzenfeld) w.schwarzenfeld@utanet.at
Approved by: antoine (implicit)
place than before flavor support was added. Move its definition to an
appropriate place.
Silence one pkq query command that may be called for a non-existing
package name.
Approved by: antoine (implicit)
Reported by: Milan Obuch, Paul Beard
supports upgrades from non-flavored port versions based on the information
in the MOVED file.
For initial installations of flavored ports, the flavor must be specified
as part of the port origin, e.g. "devel/py-py@py36" for the Python-3.6
version of that port. Dependent ports will automatically be installed with
the correct flavor passed via the dependency mechanism.
It is planned to add a --flavor option to ease flavor selection for ports
that are initially installed with portmaster.
Approved by: antoine (implicit)
Changelog:
tbuilder:
- Add flavors support, add 'all' alias to build all flavors
upgrade-ports:
- Use deinstall instead of reinstall, make backup package so things
can be restored when install fails
- Improve usage help description
- Add flavors support
Changes: https://github.com/freebsd/poudriere/wiki/release_notes_323
- bulk: Fix MOVED origin not causing a package rebuild. This
could result in a reverse dependency continuing to rebuild
due to a "new dependency" now with FLAVORS support.
- jail -c -m null: Fix this to work rather than error about
having an existing directory already.
- jail -c -m src: Respect existing _/usr/src/.cpignore_
- jail -c -m null -S: Respect the `-S path` rather than assume the jail has
a _/usr/src_.
- jail -c -m null: Assert that the jail is populated already.
- HTML: Throw explicit error if unknown ports type is encountered. [1]
PR: 224065 [1]
- Accept DT_SONAME anywhere in .dynamic when parsing elf binaries
- Fix pkg version when it deals with the ports tree and INDEX after flavors
- Sort fts traversal in repo creation (for reproducible builds)
- Documentation improvements
- make pkg version -r imply -R
- Remove FLAVOR from py-docutils dependency in comms/uhd, in this case
it needs the rst2html command, not the docutils module
- Mark some ports as not compatible with python3
Ports using USE_PYTHON=distutils are now flavored. They will
automatically get flavors (py27, py34, py35, py36) depending on what
versions they support.
There is also a USE_PYTHON=flavors for ports that do not use distutils
but need FLAVORS to be set. A USE_PYTHON=noflavors can be set if
using distutils but flavors are not wanted.
A new USE_PYTHON=optsuffix that will add PYTHON_PKGNAMESUFFIX has been
added to cope with Python ports that did not have the Python
PKGNAMEPREFIX but are flavored.
USES=python now also exports a PY_FLAVOR variable that contains the
current python flavor. It can be used in dependency lines when the
port itself is not python flavored. For example, deskutils/calibre.
By default, all the flavors are generated. To only generate flavors
for the versions in PYTHON2_DEFAULT and PYTHON3_DEFAULT, define
BUILD_DEFAULT_PYTHON_FLAVORS in your make.conf.
In all the ports with Python dependencies, the *_DEPENDS entries MUST
end with the flavor so that the framework knows which to build/use.
This is done by appending '@${PY_FLAVOR}' after the origin (or
@${FLAVOR} if in a Python module with Python flavors, as the content
will be the same). For example:
RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}six>0:devel/py-six@${PY_FLAVOR}
PR: 223071
Reviewed by: portmgr, python
Sponsored by: Absolight
Differential Revision: https://reviews.freebsd.org/D12464
- HTML: Fix failed origin column
- testport -c: Don't read host /etc/make.conf.
- testport: Avoid reading ports files outside jail for FLAVOR/DEPENDS_ARGS
- Avoid using `injail` on system.
- ports/jail -c NOZFS: Consider non-empty-already-existing-directory a failure
- Fix usage printout before system is setup.
- jail -c: Replace colon in JAILMNT to avoid PATH issues in buildworld.
- image: Setup PKG ABI properly
- FORCE_MOUNT_HASH: Support yes/no.
- FORCE_MOUNT_HASH: Fix due to improper logic.
- Record lld failures as lld_linker_error.
- options -p: Warn/confirm if in a TTY about overwriting older options
directory.
- poudriere -vv: Show which poudriere.conf files are loaded
Changes from 1.10.1 to 1.10.2
- pkg clean is now happy if the cache dir non existing
- 410.pkg-audit now uses anticongestion on freebsd 12
- fix pkg install -r foo if foo is a disabled repository
- fix elf handling on non FreeBSD
- fix support for platforms without AT_SYMLINK_NOFOLLOW
- fix dropping privileges on Linux
- fix build with recent libbsd
- fix duplicate pkg registration overwriting the existing pkg
- fix again using pattern as package arch
- recognize armv7
- add %q to pkg_printf(3) and pkg-query(8) to print altabi
- add support for new periodic variables in multiple scripts
- add an extra line when printing pkg-messages to improve readability
- stop considering nfs as dangerous and if a corruption happens on nfs drive
by a message the user to the highly possible reason of the corruption so
that he can fix
- better ABI name on linux
- fix pollution of "locked" message
PR: 221129
exp-run by: antoine
Changelog:
upgrade-base:
- pass only needed variables to make, sudo -E passes too many
causing various build errors
upgrade-ports:
- improve distfile fetching code
- try to resolve plist conflicts by removing culprit package when it is already
marked for reinstallation - this resolves most of the problems
- implement multiprocess upgrade order queue sorting
- checking for missing dependencies is now multi process operation
making it a lot more efficient and faster
- don't offer to reinstall packages marked as locked
- make conflict detection multi process operation
- on reinstall detect when package is to be upgraded or downgraded,
include this information in summary for user consideration
- remove recursively conflicting packages on plist install error
- implement dialog interface for interactive package selection
- make pkg command validation more strict to avoid false positives
- Lots of fixes/improvements on periodic scripts
- Fix typo in the .pc file
- Only check for origin if the strings contains / in pkg version
- Support flags when exporting the metalog
- Ignore flags when installing as a user
- fix again using pattern as package arch
- recognize armv7
- add %q to pkg_printf(3) and pkg-query(8) to print altabi
- add an extra line when printing pkg-messages to improve readability
- stop considering nfs as dangerous and if a corruption happens on nfs drive
by a message the user to the highly possible reason of the corruption so
he can fix
- zsh completion imrpovements
Last minute fixes for 3.2. If there are no reports of issues this will
be released as 3.2 tomorrow.
- Fix maintainer in buildlog missing.
- bulk/testport: Undocument -s after its removal.
- DEPENDS_ARGS: Store a PKG_NOTE with the value used for the port
- delete_old_pkg: Fix some renamed PKGNAME issues.
This will be likely be 3.2 release very quickly.
- Remove bulk/testport -s. It is dangerous, bulk -S remains.
- jail -u: Update kernel based on what was passed into jail -cK.
- distclean: Require -a/-f/listed ports.
- testport: Support MOVED with destination FLAVOR.
- delete_old_pkg: Fix renamed PKGNAME detection after commit 612e3dc018c8d3.
- delete_old_pkg: Check for rename PKGNAME before version.
- jail -x: Tell where native-xtools is being built
- XDEV: Support the fixed native-xtools build target in FreeBSD head
that builds from the Jail rather than the host system's /usr/src. [1]
- Add XDEV_SRC_JAIL, if set will use the jail's source for xdev. This
should only be used in special cases and will effectively default to
on for head and soon for stable/11.
- Record flavor for port build results.
- jail -c: Fix EMULATOR being unset if ran via poudriered
- Assert EMULATOR is not empty on use
- .poudriere.ports.queued: Store flavor for dependencies
- jail -x: Respect MAKEOBJDIRPREFIX [2]
- jail -c: Disallow using a colon (:) in jail name.
PR: 216083 [1]
PR: 215541 [2]
- image: Speed up tar creation by using 'xz -T 0'
- FLAVORS: Default listed ports to main flavor only, @all to build all,
bulk -a does all.
- testport: Disallow testing @all
- Support loading hooks from POUDRIERED/hooks.d/plugins/<plugin-name>/<hook>.sh
- Support a plugin make.conf at POUDRIERED/hooks/plugins/<plugin-name>/make.conf
- bulk -n: Fix 'Ports to build' list to only show what needs to be built.
- Implement some new hooks for perf profiling
- run_hook: Pass along logdir as LOG
- gather_port_vars_ports: Show more information for debug SKIP message
- gather_port_vars_port: Add verbose output about which flavors will be expanded
- bulk -n: Show log information at the end
- Rename .poudriere.deps% to .poudriere.pkg_deps% based on its proper name
- Store all_pkgs as .poudriere.all_pkgs%
- QEMU: Allow overriding QEMU_{MAX_EXECUTION_TIME/NOHANG_TIME}
The flag has partial support across the tree, and the same effect
can be obtained by using ".undef WITH_DEBUG".
Reviewed by: mat, rene
Approved by: portmgr (rene)
Differential Revision: https://reviews.freebsd.org/D12548
- Fix FLAVORS assertion in compute_deps to properly show flavor
- Add FLAVOR to a few job status outputs
- Show origin for a FLAVOR assertion
- Check for and error if
'security.bsd.hardlink_check_uid' or 'security.bsd.hardlink_check_uid'
are not set to '0' with BUILD_AS_NON_ROOT=yes to avoid staging errors
* Do not complain about USE_GITHUB before USES [1]
* Make sure each variable in a dependency is fully expanded [2]
* Clean up the earlier checks for BROKEN, etc. to prevent bogus warnings [3]
* 'USE=execinfo' is removed with ports r450351. So stop suggesting to use it [4]
PR: 221971 [1]
221605 [2]
222409 [3]
222653 [4]
Submitted by: amdmi3 [1]
jrm [2]
Yasuhiro KIMURA <yasu@utahime.org> [4]
- testport + FLAVORS: Fix incorrectly calling the default flavor for
dependencies duplicates
- FLAVORS: Fix non-listed ports being deleted with -C and showing in
queued list
- Update to ptsort 1.20170904
- Fix more issues with internal list parsing
- testport: Fix time elapsed not showing in HTML listing.
- HTML: Be backwards compatible with data without elapsed for built/failed
- HTML: Fix queued ports list containing items not actually being built.
- testport: Add some specific tests for broken port or missing PREFIX.
- FLAVORS: Tweak exception to hint about unprovided FLAVORS.
- jail: Add -C (for clean) option in order to clean poudriere data when
deleting a jail
- image: Fix tar image type
is dead and so the port is not useable anymore.
If someone wants to continue the project please head up to
https://github.com/freebsd/redports/
Thanks and goodbye redports.
(via Mk/bsd.default-versions.mk and lang/gcc) which has moved from
GCC 5.4 to GCC 6.4 under most circumstances.
This includes ports
- with USE_GCC=yes or USE_GCC=any,
- with USES=fortran,
- using Mk/bsd.octave.mk which in turn features USES=fortran, and
- with USES=compiler specifying openmp, nestedfct, c++11-lib, c++11-lang,
c++14-lang, c++0x, c11, or gcc-c++11-lib.
PR: 219275
2017-08-30 www/speedtest-mini: Discontinued upstream (on June 30, 2017)
2017-08-31 devel/rubygem-sequel3: Use devel/rubygem-sequel instead
2017-08-31 audio/tomahawk: Project is dead upstream.
2017-08-31 mail/rubygem-larch: Unmaintained by upstream, no changes since 2013
2017-08-31 devel/rubygem-sidekiq4: Use devel/rubygem-sidekiq instead
2017-08-31 devel/rubygem-webpacker1-rails5: Use devel/rubygem-webpacker-rails5 instead
2017-08-31 devel/rubygem-sidekiq-cron04: Use devel/rubygem-sidekiq-cron instead
2017-09-01 www/py-django19: Unsupported upstream
2017-09-01 mail/movemail: Out of date
2017-09-01 german/phone: Out of date
2017-09-01 ports-mgmt/fbsdmon: no longer maintained, web site it sends data to has been taken over by cybersquatters
-image: Update the firmware description in manpage
- image: Clean up some grammar for image type
- image: firmware: -s is required
- image: For firmware, build an image that is actually the size we say it
should be.
- Detect invalid dependency lines
- Fix kern.randompid / parallel pid reuse resulting in 100% CPU.
- testport: Properly use BUILD_PARALLEL_JOBS (not PREPARE) for
dependency building.
- options: Honor the -p option
- Add <tree>-<jailname> CUSTOMISATION support
- options: if no jail is passed, the local host architecture is taken
- options: Add arch -a option
* Don't check for absolute paths inside make variables [1]
* Check pkg-plist for use of OPTIONS [1]
* Fix an issue with line number offsets [2]
* Don't alert on double-dollar (VARIABLE) [3]
* Allow BROKEN, IGNORE, DEPRECATED to appear before the *_DEPENDS block [4]
* Check that manpages are not installed into share/man [5]
* Check for a space anywhere between a variable and its value [6]
PR: 221507 [1]
221291 [2]
221397 [3]
221377 [4]
221455 [5]
221219 [6]
Submitted by: Dmitri Goutnik <dg@syrec.org> [4]
Chase the consolekit port move committed in r447399 (See Also: r447400)
causing INDEX failure:
make_index: /home/indexbuild/tindex/ports/ports-mgmt/packagekit: no entry for /home/indexbuild/tindex/ports/sysutils/consolekit
Reported by: cperciva (subsequent to indexbuild report)
image: Convert firmware to be like nanobsd with split partitions
image: Use the same image for the second partition as a sort of backup
image: Add /etc/save_cfg to the image
bulk logs: Reduce execs by using port_var_fetch for -V lookups
sh: Add mkfifo(1) builtin from FreeBSD head r314436
sh: Add unlink(2) builtin and use for simple file removals
Revert "Do not create a FS when '-m none'"
Partially revert "Do not delete a ports tree created with method=none."
ports -cF: Allow importing existing tree with -F
ports -c: Don't destroy filesystem on error with -f none.
ports: Add new method -m null for managing system ports trees.
ports -d: Fixup various deletion cases.
ports -cF: Allow specifying -m override with -F
Link a jail/latest-done when done building.
Don't show arrow when there's a timestamp.
* Trim comments after USE_GCC [1]
* Add a check for redundant SHEBANG_LANG values [2]
* Make sure both USES and USE_* are actually seen before checking
to see if one is included before the other [3]
PR: 221085 [1]
220500 [2]
Submitted by: gerald [1]
Reported by: dbaio [3]
- pkg clean is now happy if the cache dir non existing
- 410.pkg-audit now uses anticongestion on freebsd 12
- fix pkg install -r foo if foo is a disabled repository
- fix elf handling on non FreeBSD
- fix support for platforms without AT_SYMLINK_NOFOLLOW
- fix dropping privileges on Linux
- fix build with recent libbsd
- fix duplicate pkg registration overwriting the existing pkg
- better ABI name on linux
- lots of changes in internal API to use fd when possible
- removed option: pkg repo -L
* Remove the other check for conflicts that are too broad [1]
* Remove the check for portlint that is more than 30 days old [2]
* Check to see uf USE_* comes before USES [3]
PR: 220087 [2]
220340 [3]
Reminded by: dbaio [1]
- jail -c -m ftp: Install QEMU before running freebsd-update if needed.
- jail -m src: Support -x without -b
- jail -m tar: Support -x
- jail -m ftp/url/...: Support -x.
- bulk: Handle jail -x hardlinking during startup now.
- jail -x: Only build native-xtools if emulation will be required
- ports -u: Fix portsnap msg having timestamp for 'done'
- Fix inverted check which always failed for nested jails IPv4/IPv6
addresses
- jail -x: Add back a default for NXBDESTDIR for older base branches
- testport/bulk -i: Fix not installing main package in jail.
- bulk: Store build name in PACKAGES/.buildname
- jail -cu -m ftp: Fix freebsd-update not being properly ran.
- HTML: Don't show NaN:NaN:NaN for duration on older builds.
- HTML Remaining: Hide table if there's no [more] data.
- HTML Remaining: Hide find warnings from racing with clean.sh.
- pkgclean: Remove packages from .building directory if it exists.
- jail -u -m ftp: Don't run freebsd-update on unsupported branches.
* Changed comment headers to match my name and FreeBSD email
Reviewed by: matthew (mentor)
Approved by: matthew (mentor)
Differential Revision: https://reviews.freebsd.org/D11466
[Changes staged for 3.1.20]
- jail -c: Do not create a filesystem with *-m none*
- ports -c: Don't try removing a filesystem that was not created
- pwait: Fix *kern.randompid* hangs
- queue: Fix for sending commands to poudriered [not well supported]
- jail -u: Don't run freebsd-update update for PRERELEASE, STABLE or CURRENT
jails.
- jail -c: Don't allow creating a jail when its mountpoint already exists
- jail -c -m ftp: Avoid bad number comparison with '12-CURRENT'
- XDEV: Attempt to link /usr/bin/makewhatis
- sh: Update from FreeBSD head r319826
- sh: Enable nofork for: clock, dirempty, realpath, rename, rmdir, mkdir,
and mktemp
- jail -u: Only run freebsd-install if needed
- bulk CHECK_CHANGED_DEPS: Only run `ldconfig -r` once for the entire build
rather than for every package.
- poudriere: Allow _MAKEOBJDIRPREFIX_ from command environment.
- poudriere: Force umask 022 and allow override with UMASK in poudriere.conf
- Default RESOLV_CONF to host's /etc/resolv.conf
- jail -k: Don't force having a portstree for cleanup
- Fix MNAMELEN check consistency (2 different length paths were being used
for calculations)
- bulk -n: Fix .building directory sticking around on early exit.
- Jails: Don't use allow.socket_af, allow.raw_sockets in the no-networking
jail.
- Nested jails:
- Don't force allow.socket_af, allow.raw_sockets, allow.chflags,
and allow.sysvipc
- Check for and error if the follow flags are not enabled:
- enforce_statfs=1, allow.mount, allow.mount.devfs, allow.mount.nullfs,
and allow.mount.tmpfs
- children.max is not properly exposed in a nested jail and so is not
checked.
- Fix trying to add loopback address in nested jail when there is none.
[Changes staged for 3.2]
- image:
- Add to manpage
- Require -t
- Document new embedded type
- HTML
- Show remaining ports if HTML_TRACK_REMAINING is set to yes.
- Show original queued ports
- Show PKGNAME in builder/job table
- Store and link dependency graph from 'Logs' dropdown.
- Show time elapsed for built/failed ports.
- Support src-env.conf
- jail -m git: Support file://
- Limited DEPENDS_ARGS support for py3 slave ports.
- Export PACKAGE_BUILDING_FLAVORS to disable some seatbelt logic in ports.
- This allows leaving the default Python version at 2 and having
a port like net/tiny-network-utilities properly build with a default of
3 and have its dependencies properly queued and built without using
py3-* slave ports.
- This support is disabled unless the default is set to 2.
- Currently allows building "virtual" non-existent ports like
textproc/py3-hexdump or textproc/py34-hexdump and having it build
the expected version. Note this support is disabled once FLAVORS
is committed to Ports so expect it to change soon without any kind of
MOVED entries.
- FLAVORS support [experimental, needs documentation, syntax may change]
- Queued origin like 'devel/libtrue' will build all available FLAVORS.
- Queued origin like 'devel/libtrue@TRUTH' will only build the TRUTH flavor.
- Queued origin like 'devel/libtrue@-' will only build the first FLAVOR.
- FLAVORS support requires ports advertises PORTS_FEATURES+=FLAVORS.
- No support is currently committed in ports.
- logclean:
- Don't try building JSON when no files are found
- Add -B to specify buildname
- Document -z and -j
- Don't run anything if there is no log directory
- Recreate latest-per-pkg links at the end; avoid deleting all logdirs.
- Stop trying to use /usr/local/bin/gawk
- bulk:
- -a: Speedup listed_ports after the first iteration
- -n: No need to markfs prepkg
- -C: When using -a this will now behave like the more optimal -c
- startup: Fix some errors not being captured properly
- -C: Delete packages found using xargs
- -n: Optimize 'ports to build' output generation
- Add some basic automated tests for FLAVORS dependency calculations
- CHECK_CHANGED_DEPS: Some optimizations
- Detect packages stuck in queue at startup
- Validate ports have proper CATEGORIES set during build.
- Warn on bad PKGNAME dependencies and make fatal with
BAD_PKGNAME_DEPS_ARE_FATAL=yes
- Add pkgrepo:sign hook right before calling pkg-repo.
- Add a few more status / messages during startup
- Align SIGINFO job status when there is no origin.
- Show pkgname in a few more places for disambiguation.
- Fix FORCE_COLORS/-A to actually work
- Fix colors when using | timestamp.
- Remove some leftover pkg_install support code for delete_pkg
- poudriere: Allow FETCH_BIND_ADDRESS
- poudriere: Fix populating git version in a worktree
- Allow forcing mount hashing with FORCE_MOUNT_HASH
- ports -u: Reduce redundancy in updating messages.
- jail -cu: Show method rather than 'svn' or 'git'
- Allow overriding git with GIT_CMD
- zfs_getfs: Don't echo blank values from cache_call
- Add support for file:/// URLs with git
Changelog:
mkpatch:
- Generate patches similar to make makepatch ones
pinstall, preinstall:
- Implement option to build with debug information (-d)
- Implement interactive mode for pinstall and preinstall (-i)
allowing to choose which matched ports will be built
- Implement option (-c) to allow configure ports options before
install/reinstall
plist-sort:
- Correct match expression, fixes parsing lines with '@' character
that not contain pkg commands
upgrade-ports:
- Add conflicts detection in pkg database errors check
- Make skipping ports after error recursive
- Improve summaries wording, include number of items in lists
- Add error checking for MOVED entries handling, when everything fails,
remove old port and install new one
- Show conflicts message for packages removed in MOVED parsing
- Add force recompile option, force reinstallation of packages with
stale dependencies during sanity check phase
- Fix regression of not saving last UPDATING entry after succesful upgrade
* Remove a check for narrow conflicts now that we are in the post-pkg world [1]
* Genericize the USES_ sorting code and check to see if NOT_FOR_ARCHS
and ONLY_FOR_ARCHS are sorted. [2]
* Fix the check for JAVALIBDIR so that it falls before the variable expansion
code. [3]
PR: 219684 [1]
218823 [2]
219360 [3]
Submitted by: eugen [3]
Poudriere in particulr did not properly handle DEPENDS_ARGS which
made these ports not properly install dependencies. That bug
is being addressed along with adding FLAVORS support to it.
With hat: portmgr
MFH: 2017Q2
- pkgclean: -a now again keeps all known ports and -A removes all packages
- pkgclean: No need to boost priorities, and fix overwriting PACKAGES/.jailversion
- testport -v: Show dependencies like bulk does
- ATOMIC_PACKAGE_REPOSITORY: Fix .jailversion not being symlinked
- Show Poudriere's PKGENV/PORT_FLAGS in build log separately from environment.
- pkgclean: Fix zfs_getfs cache error on exit
- jail -c (ftp): Fix 'mount: /compat: No such file or directory' warning
- Fix order of jail/builder stop hooks on early exit.
- jail -c -m svn/git: Store VCS revision separately from __FreeBSD_version.
- jail -c -m svn/git/src: Use __FreeBSD_version rather than VCS for
forced package rebuilds.
- jail -c -m null/tar/src: Don't require a -v version since it can be
determined automatically
- jail -c -m src: Fix MAKE_JOBS not being used for installation
- Update sh from FreeBSD head @ r318502
- Store PACKAGE_BUILDING/FORCE_PACKAGE in make.conf to fix
bulk/testport -i usage.
- image: Use mdconfig for rawdisk
- Add LICENSE
* Add --automatic option. This option sets automatic flag for the package.
* Delete --noindex option, add --index option. When pkg_replace checks the dependencies before building the target packages, do "make package-depends-list" as default. This is more strictly checking the dependencies.
PR: 219397
Submitted by: kdeguchi@sz.tokoha-u.ac.jp (Maintainer)
Approved by: miwi (mentor)
Differential Revision: https://reviews.freebsd.org/D10849
- Raise 'package' phase timeout to 2 hours
- image: Use makefs(2) to generate ufs2 disks
- image: makefs use -s for size
- HTML json top: Remove disconnected code
- HTML json jail: Fix 'latest' not populating always.
- pkgclean: Exit 0 if no packages are left to cleanup.
- pkgclean: Fix -a to work properly
- err: Don't show 'Error' if exiting 0
- bulk/pkgclean: Don't allow specifying '' as a package
- pkgclean -a: Remove pkg repository as well
- pkgclean -a: Add a note about manually removing the PACKAGES directory
- Don't fail if 'ulimit -a' fails somehow
- LOCAL_MTREE_EXCLUDES: Add missing leading '.'
- Update sh from FreeBSD head @ r317912
- jail: Add -D option to keep git history
- jail: Document -D option for jail creation
Changelog:
- Add pinstall, preinstall frontends to upgrade-ports script
- Use ellipsis '...' to indicate time consuming operations
- Clarify usage descriptions
upgrade-ports:
- backup self conflicting packages and use them to restore
when upgrade fails
- allow to remove conflicting package in install phase
- show type of failure in upgrade summary when failed builds occur
- improve conflicts handling
- show removal reason in upgrade summary
- check rebuilt ports for BROKEN and IGNORE flags
- automatically change port name on move operation
- implement option for only checking packages database for errors (-c)
- improved MOVED entries tracking
- don't offer recompilation of packages linked to libraries not
found by system linker
upgrade-base:
- force sudo to preserve already set environment variables, this fixes
PORTS_MODULES not being passed when user is non-root
tbuilder:
- improve detection of required option for library dependency
- add profiles feature, it's files in $CONFIGDIR containing names of jails
for use in one build test batch.
pcommit:
- print correct object type in file_add() function
* Check for use of OPT_XXX_OFF syntax to prevent false incorrect option
usage assertions. [1]
* Remove now-obsolete warning about using :LU operators with make(1). [2]
* Fix the regex to prevent direct command use from hittin on lines
like FOO_USES=... [3]
PR: 218260 [1]
218325 [2]
218672 [3]
Submitted by: jrm [2]
Reported by: rezny [2]
lang/gcc which have moved from GCC 4.9.4 to GCC 5.4 (at least under some
circumstances such as versions of FreeBSD or platforms).
This includes ports
- with USE_GCC=yes or USE_GCC=any,
- with USES=fortran,
- using using Mk/bsd.octave.mk which in turn has USES=fortran, and
- with USES=compiler specifying openmp, nestedfct, c++11-lib, c++14-lang,
c++11-lang, c++0x, c11, or gcc-c++11-lib.
PR: 216707
* Fold in the same removal of the obsolete HTTP-only warning
* Add support for checking for direct dependency on bison, fmake, and libexecinfo. [1]
PR: 216268 [1]
HTTPS-only is normal novadays, also may@ says that the warning was
there because before 8.4 libfetch did not support going through a
proxy for HTTPS
PR: 217995
Approved by: maintainer timeout (marcus, 2 weeks)
Differential Revision: D10015
- Cache pw result to avoid over reloading nssswitch.conf
- Fix allow to set env per repo for everykind of repo
- Fix signing_command something saving a truncated signature in the file
as a result pkg will reject any repository signed
- Fix sqlite vfs overload with new sqlite
- pkg which: add -m to show what files matched using globs
- Revert the new UI which does not respect 80 columns
- Fix query '%q' printing the internal arch instead of the official one
- Fix signing_command something saving a truncated signature in the file
as a result pkg will reject any repository signed
- pkg which: add -m to show what files matched using globs
- fix sqlite overload with new sqlite
- Revert the new UI which does not respect 80 columns
- Fix query '%q' printing the internal arch instead of the official one
- Fix ZFS rollback handling to create needed file for checking for race
after a rollback.
- pkg/dist/logclean: Make size calculation account for compressed ZFS
datasets.
- Add a 'logclean' command that cleans build logs older than N days, or
builds over a certain count, by filter.
- Store temp and lock files in $(mktemp -dt poudriere), and remove on exit.
- Change SHASH_VAR_PATH out of umounting MASTERMNT on exit
- USE_TMPFS=yes: extend to data as well as wrkdirs
- JSON jail: Ignore broken builds rather than corrupting the data
- HTML JSON: Fix not writing on exit, and writing to /.data.json[.mini]
- blacklist: Fix loading of JAIL-SETNAME-blacklist
- Don't bother statting or creating pkg cache dirs when deleting them at startup.
- pkgclean: Clean cache files as well
- poudriere.8: Fix reference to native-xtools and document use of /usr/src
- jail -d: Fix not cleaning up jail's temporary mountpoint directory
- jail -uc: Don't leak host /etc/src-env.conf into jail builds
- timestamp: Fix not exiting when stdin closes.
- timestamp: Fix not properly handling newlines.
- Fix umounting when using MDMFS
- Make jail -k try harder to cleanup any messes left behind.
- Add an ALLOW_NETWORKING_PACKAGES to allow networking for select packages
- testport: Support -B
- Only load linux64 if the host is newer than ~10.2 Current that had it
- Determine UMOUNT_NONBUSY at startup, which allows jail -k to benefit
- jail cleanup: Fix only the first 2 builders cleaning sometimes on older
systems.
- build_pkg: Assert that rollbackfs worked
- Rework ZFS rollback race fix by creating the file at markfs time.
- Actually install cache.sh
- Fix very old bug with ZFS rollback not succeeding due to
race with zfs rollback + zfs snapshot.
https://www.illumos.org/issues/7600
- bulk/status summaries: Fix build time being wildly wrong
- MOVED: Convert to using shash
- jail -u: Don't show /compat mount warning.
- Automatically handle making some internal paths relative when
doing 'cd'
- Add a per-run caching mechanism for commands
- Cache ZFS list (for mountpath) lookups.
- jail -u, etc: Fix ZFS snapshots not being created after 33297b28f.
- destroyfs: Avoid zfs_getfs() if not needed
- testport: Support MOVED for main port being tested
- locks: Disable INT/TERM traps while in here
- HTML JSON is now generated with TERM delayed, meaning leftover tempfiles
in the logdir should be fixed and the JSON should be more consistent.
- poudriere: Allow -v to increase VERBOSE since not all sub-commands allow it.
- poudriere: Allow -A to force colors, even if not in a TTY
- HTML JSON: Use relative paths for reading/writing files in logdirs
- prefix_output: Append into proctitle to make debugging simpler
- testport -c: Fix after 88334520d1159.
- Introduce HTML_TYPE to select if HTML files are inline or hosted
- Rewrite parsing of 'bulk -f' and 'bulk PORT1 PORT2' (and testport)
handling of ports into the queue.
- Update sh to FreeBSD head r315005.
- compute_deps_pkg: Give a hint about failed pooldir creation being from
duplicates. This is still undergoing further changes. It's technically
an assertion still that is has proven to be possible.
- Support copying a static ccache into the jail with CCACHE_STATIC_PREFIX.
This is to support using something like devel/ccache-memcached-static
in the jail to support memccached without any bootstrap or cyclic
dependency problems.
- Fix spurious SIGALRM on poudriere processes.
- Add a LOCAL_MTREE_EXCLUDES to allow appending to the list of directories
to ignore in the mtree checks.
- Document RESTRICT_NETWORKING
- Fix regression from last build not causing ZFS+USE_TMPFS=all to
rollback to the clean snapshot before cloning.
- Fix mktemp using TMPDIR='' by default (thus /). This broke
some non-root operations.
- options: Fix harmless warning about missing 'arch' file.
- Fix very old crash with SIGINFO while cleaning the queue after
a port is done building.
- Revert a change for port statistics changes which may have resulted in
warnings such as:
mv: rename /usr/local/poudriere/data/logs/bulk/103i386-default/434652/.tmp..poudriere.stats_ignored to /usr/local/poudriere/data/logs/bulk/103i386-default/434652/.poudriere.stats_ignored: No such file or directory
Note that the change here does not restore ignoring errors like
the previous code did. So some odd crashes may occur, but all
callers of this code were audited and should be ignoring the
error anyhow.
- Fix build on <head
- Fix startup depqueue errors if the listed ports have duplicates.
- Fix NO_ZFS/USE_TMPFS=all jail cloning error:
[00:00:00] ====>> Creating the reference jail...realpath: Illegal option -r
- NO_ZFS now overrides existing ZFS datasets; ZFS won't be used at all if
NO_ZFS is set.
- ZFS datasets are now looked up with 'zfs list' rather than 'mount -t zfs',
which fixes some cases of ZFS datasets being left around after an error.
- Properly ignore /proc without invoking pseudofs kernel code.
- mtree: Remove exclusion of /usr/home
- mtree: Remove exclusion of .p4config
- mtree: Restore ignoring /var/tmp in prebuild/prestage.
- mtree: Don't descend into /dev, avoiding devfs_readdir, etc.
- mtree: Avoid descending into all directories.
- gather_distfiles: Use port_var_fetch to lessen needed make executions
- sh: Make touch a builtin
- Fix commit c0daac2f: /dev/null is a special character device, not directory.
- pwait builtin: Properly restore SIGALRM on exit
- pkgclean/distclean: Properly support PREPARE_PARALLEL_JOBS from
poudriere.conf
- Default PREPARE_PARALLEL_JOBS to 1.25*PARALLEL_JOBS
- Poudriere now sets process title for some operations via setproctitle(2).
- Dependency calculation majorly refactored to fetch more make variables
at once rather than fetching some and then more later. This rework also
speeds up some startups by up to 60% by keeping the compute_deps work
queue full and more parallelized. This is especially seen in meta
port lists. The storage for the make variables fetched is not very
efficient yet and will improve more in the future.
- Add more sh builtins: pwait, mkdir, rename, setproctitle, rmdir, mktemp
- nohang: Do read before logfile comparison.
- update_stats: Stop building from html_json coproc after done building
- More work to use relative paths, thus reducing namecache contention.
- rm is bundled if the base system version does not support rm -x.
- Determine at runtime if umount -n (MNT_NONBUSY) is supported
- buildlog: Properly show limits in jail
- The "balance_pool" operation after a port is done building is now
asynchronous.
- Fix some cases where stats were not updated on early exit.
- Detect race during jail startup; prevent multiple concurrent jail
startups for the same jail+set+portstree.
- Some unexpected older packages may be deleted during startup.
- bulk: Change dependency information to -v rather than -vv
- Limit parallel umounts if batched free vnode lists not supported.
- Only fetch LIB/RUN_DEPENDS and OPTIONS if the CHECK_CHANGED_DEPS/OPTIONS
is set
- Add some missing license headers in HTML code.
- run_hook: cd to / before running hook to avoid umount problems.
- Pass -DNOCLEANDEPENDS when cleaning to not clean dependency dirs.
- Slightly reword some of the port job statuses.
- api: Allow evalling from stdin
- Move /var/db/freebsd-update exemption into common_mtree.
- More tests added
- Fix PORTBUILD_UID lookup when user doesn't exist in jail.
- configure: Don't prefer GCC (poudriere build only)
* fix pkg which -g
* Do not use openssl for sha256
* Improve the default output when fetching data
* Update libfetch to the version of FreeBSD 11
* Fix NetBSD ABI
* Add a fallback for utimensat when reporting EOPNOTSUPP
* replace libsbuf with utstring for portability
* Overwrite some sqlite vfs functions to allow more capsicumisation of pkg
* move more pkg audit details into the library
* allow to run pkg query without any privileges
* extend the audit periodic script to report deprecated packages
* Fix fd leak to rcscripts
* stop dropping privieges during fetching it causes more issues than it solves
* speed walking though [reverse]dependencies
* Update blake2b to latest version
* Add blake2s to the list of supported checksums
* Internal digest now default on blake2b on 64 bits and blake2s on 32 bits.
* Fix segfault when ABI is defined but empty
* Do not use the shared library provides/requires on non FreeBSD/Linux
* Define _NETBSD_SOURCES on NetBSD to allow libfetch to use funopen
* Document vital flags
* Fix potential double free when dealing with @config keyword
* Add progress when fetching (number of files to fetch)
* Improved messages about locked packages
* Return 1 when a user rejects the proposed plan
* When dealing with configuration files '@config' never overwrites non tracked
files
* Warn everyone about deprecation of @dirrm and @exec
* Deduplicate metadata loading code
* pkg register now understands context aware messages
* Fix pkg register issues with some version of libarchive:
hardlinks not being made hardlinks sometime
issues when PREFIX is a symlink to another directory
* Use portable mechanism to find the number of CPUs which allows pkg repo to be
faster on linux
* New pkg upgrade --vulnerable
- Manpage updates
- Remove pkg_install support.
- Remove NO_STAGE support.
- Remove use of umount -f where possible.
- Use relative paths where possible to avoid namecache contention.
- USE_CACHED reworked to use POSIX mq (will soon change again)
- PRIORITY_BOOST is fixed to boost dependencies as well now with
USE_PTSORT enabled (default yes).
- Load linux64 when needed
- Improved ports_env support (less executions during ports interations)
- Show resource limits in build log
- Limit filedescriptors to 1024 by default with MAX_FILES
- Shutdown/unmount build jails in parallel
- Don't leak mount path in build log from ident(1)
- sh: Add some builtins to avoid fork/execs (mostly poudriere C helpers):
cachec, clock_monotonic, date +%s, dirempty, locked_mkdir, realpath,
sleep
- BUILD_AS_NON_ROOT: Remove redundant chown -R.
- Add HTML_JSON_UPDATE_INTERVAL (def: 2) to allow overriding JSON update
interval.
- Add support for USE_TMPFS=no
- jail: install the kernel requested with -K when using src method
- jail -c -m src: Add support for building the src tree with -b.
- jail -u -m src: Update the version for the jail as well.
- jail -c: Allow @rev for releng/X.Y branch
- jail: Switch to http protocol for ftp-archive.
- ports: Do not create a FS when '-m none'
- ports -d: Do not delete a ports tree created with method=none or -.
- ports -d: Confirm if in a TTY
- ports -d: Really delete the zfs dataset when TMPFS_ALL is used
- ports: Add git+http support
- ports/jail: Auto pass PROXY env vars from parent environment
- image: only isos have a restriction in the image name
- image: If the images dir does not exist, attempt to create it
- image: Handle a relative path with -o
- image: Various fixes.
- image: usb|*firmware|rawdisk now require a kernel and image size.
- testport: Fix fdescfs/procfs/linprocfs not being mounted.
- testport: Fix limits not being applied
- testport+SIGINFO: Fix very old deadlock with update_stats.
- pkgclean: Add hook: pkgclean done <DELETED_FILES: 0|1> <BUILT_REPO: 0|1>
- pkgclean: Cleanup corrupt packages
- pkgclean: Act on .latest directory for ATOMIC_PACKAGE_REPOSITORY
- poudriere.conf: Don't suggest changing svn.FreeBSD.org to a mirror anymore.
- QEMU+Arm64: Coppy the latest aarch64-binutils ld into the jail at startup.
- QEMU+Arm64: Link /usr/bin/ld to /nxb-bin/usr/bin/ld.
- QEMU+Arm64: Tell about copying ld
* QEMU+Arm64: Note that this is not working properly yet due to the
aarch64-binutils port not providing a static ld.
- options: Properly setup ARCH/MACHINE in make.conf for cross-builds
From the PR:
"The web site used by this port, http://fbsdmon.org, has been taken
over by cybersquatters. That means that users are sending their
system info to an unknown party. The port has had no maintainer
for almost two years and the original maintainer (and previous owner
of the website) is not responding to email."
Since there's no way to force this port to be deinstalled from users'
machines, to stop data being sent to an unknown party, also do the
following:
- Modify the port to never send data to fbsdmon.org, bump PORTREVISION
- Mark the port IGNOREd for ports users, but allow package to be built
and set extended 6 month EXPIRATION_DATE (or should it be extended
to a year?) to give a modified package chance to replace original one
on users machines
PR: 217099
Submitted by: asomers
MFH: 2017Q1
- Remove pkg2ng and pkg convert code
- Emit dep_formula within the manifests
- Use stack based read (32k buffer) instead of mmap
- Use 32k buffers for each reads
- Cache pw/group to reduce overreading nsswitch.conf
- Update libucl and remove over call of getcwd
The mistake was completely on my part, I somehow connected the dots the
wrong way in my head.
The only exceptions (for now) are archivers/zstd and ports-mgmt/synth
which were already picked up by new volunteers in the mean time.
- Fix install as user
- Allow to set env variable per repo to pass to libfetch
- Plug a memory leak
- Fix repo initialization when a repo is disabled
- Fix installing sparse binary files
- Add an option for a user to change locking over nfs
Changelog:
pkg-deleaf:
- reset items list after packages deletion
upgrade-base:
- use sudo when run as normal user
- offer to run rebbot for the user
upgrade-ports:
- include reason in message about port move
- fix updating commands parsing code
- improve sanity check summary, fix required by printing and
show missing required libraries for each reinstalled package
- fix obsolete reason extraction
- don't remove reverse dependencies of self conflicting packages so
that after upgrade no user packages is missing
- check for boundaries while calculating ports upgrade order
preventing stack memory exhaustion
mkpatch
-------
Makes patch from (all or pattern matched) modified files in WRKSRC directory.
pcheck
------
Check for various errors that portlint don't catch:
* misspellings in description files
* files with no final new lines
* leading articles in COMMENT, tabs in WWW lines
pclean
------
Keeps your ports svn repo tidy, removes work directories, leftover files
and reverts any changes made.
pcommit
-------
Wrapper for svn commit, removes and adds files, sets props, etc.
pfind
-----
Simple port's search wrapper. Search parameters include build dependency,
category, name, maintainer, regexp pattern and slave ports.
pkg-deleaf
----------
Leaf package remover with nice dialog interface.
plist-sort
----------
Smart plist sorter which understands PLIST_SUB substitutions and options
determined conditional files.
plist-update
------------
Script utilizing check-plist port's framework target output for updating
outdated pkg-plist, understands PLIST_SUB substitutions and sorts
final files list.
plog
----
Easy way to send poudriere build logs to your FreeBSD account by scp or
view them on your computer.
reinplace-check
---------------
Checks for files with stale REINPLACE_CMD actions or shows diff's of them.
tbuilder
--------
Poudriere testport wrapper that makes easier to queue multiple builds of:
- specific port
- ports linking with certain library
- ports with certain names
- ports with Makefile matching regular expression
- slave ports etc.
upgrade-base
------------
Helper script for upgrading FreeBSD base. Detects and sets number of make
jobs possible. After upgrade rebuilds ports with kernel modules and offers
to remove old files using dialog inerface.
upgrade-ports
-------------
Helper script for updating ports tree and upgrading your ports. Combining
information gathered from UPDATING and MOVED files tries to provide upgrade
process as problem free as possible. This includes showing new UPDATING
entries since last upgrade, ports origin move tracking, simple conflict
resolution, removal of obsolete ports and sanity checking afterwards.
Before any change is made user is presented with summary of changes
intended similiar to pkg upgrade.
1) Create a platform-specific (FreeBSD/DF, NetBSD, Linux, Solaris) version
of /etc/shells for the builders. This resolves installation errors on
packages that manipulate /etc/shells.
2) Give in and finally mount /usr/games if available. This is a
performance impact ultimately caused by removing the strfile
application from ports with the faulty logic that the 400-line C
program can't exist because it's in a non-standard location in base.
The alternative is that Synth can't build any applications that
require strfile from games, e.g. all fortune ports. Thanks! <sarcasm/>
This version blocks TTY signals to and from background processes, allowing
lang/sbcl to build as well as avoiding disruption of an active ncurses
display by "you've got mail" message.
It also includes a minor duration time formatting fix on build logs and
the many-thousands-open-file-descriptor problem fix on purge-distfiles
command provided by 1.66_1 between releases.
There is also a change in the build order algorithm. Previously the
categories were ordered alphabetically, but rather scanned in the order
that appears by walking the directory. Now when the "nothing depends on
these ports" ports block starts building, it's in A-Z order by category/
portname whereas before ony the portname was ordered A-Z (the categories
were blocks in random order). There is no real benefit to the reordering;
it's mainly to satisfy those bothered by disorder.
The purge-distfiles command reads the distinfo file of every port in the
tree, but unfortunately never closed the file descriptor when the scan
was complete. This resulted in 26,000+ open file descriptors, possibly
exceeding the host server's FD limit. (issue #70)
- Removed git file and folder generator before applying a patches
- Small optimizations
- Pulls poudriere.conf for pathing, small changes for readability
- Changed bulkfile default location, changed the way poudriere bulk is called
PR: 215483
Submitted by: Ultima1252@gmail.com (maintainer)
Reviewed by: matthew
Approved by: matthew (mentor)
Differential Revision: https://reviews.freebd.org/D8890
The primary reason for the creation of v1.66 is to provide support for
pkgsrc on FreeBSD, DragonFly, NetBSD and initial support for Linux and
Solaris. It represents over 100 commits and was successful. Along the
way, some improvements were made for the ports version. A (likely
incomplete) list includes:
- Improve watchdog kill message by including inactive time for trigger
- Add SYNTHPROFILE variable and value to builders' mk.conf which can be
used by custom local makefiles
- Force users to define TERM in environment if it is not present
- Ensure fetch(1) doesn't check for valid SSL certificates. Since all
downloads are hash-checked for authenticity, misconfigured SSL of a
webserver is not a good reason to break fetching.
- New 6th log which documents why existing packages are obsolete (same
as setting WHYFAIL in env.) "06_obsolete_packages.log"
- New SELFTEST environment command for test mode which forces new "test"
phase to execute (for ports with regression tests)
- Updated man page
- Stop dropping privileges for fetching, it causes more issues than it solves
- Fix segfault when ABI is defined by empty
- Fix reporting of deprecated packages
- Provide instructions on how to unset vital flages
- Fix libfetch for NetBSD
- Fix @config potential double free
- Always warn about the deprecated @dirrm/@dirrmtry
- Always warn about the deprecated @exec
- Rework pkg register to share more code with pkg install
* It makes it more robust to libarchive regressions [1]
PR: 214381 [1], 215029 [2]
Reported by: lampa@fit.vutbr.cz [1]
Exp-run: antoine [2]
Switch from perl threads to Parallel::ForkManager. With perl 5.24, threads
used in distilator produce a many random warnings hinting at the fact that the
modules used to check the links are not (and never were) thread safe.
Remove WWW. There is no public website available.
PR: 208506 (based on)
Notified by: mat
MFH: 2016Q4
- In case of errors during pkg update exit with a non 0 status
- Fix pkg register issues with some version of libarchive:
* hardlinks not being made hardlinks sometime
* issues when PREFIX is a symlink to another directory
- Use portable mechanism to find the number of CPUs which allows pkg repo to be
faster on linux
- Add progress when fetching (number of files to fetch)
- Improved messages about locked packages
- Return 1 when a user rejects the proposed plan
- When dealing with configuration files '@config' never overwrites non tracked files
- Warn everyone about deprecation of @dirrm and @exec
- Deduplicate metadata loading code
- pkg register now understands context aware messages
- Add a fallback for utimensat when reporting EOPNOTSUPP
- replace libsbuf with utstring for portability
- Overwrite some sqlite vfs functions to allow capsicumisation of pkg
- move more pkg audit details into the library
- allow to run pkg query without any privileges
- extend the audit periodic script to report deprecated packages
- Fix fd leak to rcscripts
- stop dropping privieges during fetching it causes more issues than it solves
- speed walking though [reverse]dependencies
- Update blake2b to latest version
- Add blake2s to the list of supported checksums
- Internal digest now default on blake2b on 64 bits and blake2s on 32 bits.
- Fix segfault when ABI is defined but empty
- Do not use the shared library provides/requires on non FreeBSD/Linux
- Define _NETBSD_SOURCES on NetBSD to allow libfetch to use funopen
- Document vital flags
- Fix potential double free when dealing with @config keyword
* Allow https URLs for the WWW line in pkg-descr. Why are we only NOW doing
this? [1]
* Check TIMESTAMP to make sure it's in the present/past, but not more than 30
days in the past. [2]
PR: 212091 [2]
Submitted by: amdmi3 [1]
Differential Revision: D8462 [1]
Skip new "base" category during ports scanning
The new base category was never meant to be part of the build so the
category makefile was intentionally left off so poudriere and friends
would skip it.
Unfortunately, this idea didn't account for Synth which has a different
behavior: If no category makefile exists, *ALL* subdirectories are
scanned instead of none of them. (DPorts doesn't use category Makefiles
because by definition all ports are valid; there are no "unhooked" ones)
The proper solution would be to include a practically blank
ports/head/base/Makefile to maintain consistency with the rest of
the ports categories, but synth now actively excludes the base category.
Change name of 05 log
The command output log name has changed from "05_abnormal_cmd.out" to
"05_abnormal_command_output.log". The former tended to be considered
a binary file by webservers and would download rather than display.
Final tally added to end of main 00_last_results.log file
At the end of a bulk run, the final tally (queue size, built, failed,
skipped, ignored) would be displayed in curses mode. This change adds
the same useful information to the end the main bulk run log.
- Fix file descriptor leak during rc script execution
- Fix setting attributes on directories
- Fix locking on NFS
- Allow pkg annodate -S to run as a non priviledge user
- Extend the audit periodic script to report about expiration/deprecation
Change in hook behavior:
Now the initial building of pkg(8) triggers a success or failure hook
after building. Before ports-mgmt/pkg was the only port that could
never trigger these hooks. The "bulk run start" hook is triggered
afterwords as pkg(8) is a prerequisite to preparing the bulk run.
Fix potential bad unmount of /usr/src
In the case where /usr/src exists, but the profiles uses a system root
that is not "/" and doesn't have an "usr/src" subdirectory, an error
would be emitted as the unmount of the non-existant mount failed. This
uncommon use case has been fixed.
Curses display builder "Elapsed" label changed to "Duration"
This matches the format of the web-based report
DragonFly only: Support File(1) v2.0 version parsing
The output of file(1) version on DragonFly will change with version 2.0.
For example, what is now version 4.0.702 will be displayed as 4.7.2 on
newer versions of file(1). Moreover, the parsing for DragonFly 4.10+
would not have worked. Now both versions are supports and the double-
digit minor versions are properly handled as well.
At the request of a user, further review reveal that the pkg_* tools
aren't actually used. It just awks the INDEX file. The maintainer
forgot the technical details, especially since the SEE ALSO section of
the man page references pkg_* tools.
A minor update is needed to re-support DragonFly and update the man
page. Until this, it's been marked IGNORE for DragonFly.
- Fix fd leak on systems without utimensat (merged in release branch)
- Do not use openssl for sha256
- Improve the default output when fetching data
- Update libfetch to the version of FreeBSD 11
- Update libsbuf to the version of FreeBSD 11
- Fix NetBSD ABI
- Add a fallback for utimensat when reporting EOPNOTSUPP
Fix web report regression caused by cleaning of history files.
The history files were being removed too late. Any ignored/skipped
files erased after writing (but would return with first built/failed
port). If more than 500 ports were ignored+skipped, the web report
would fail to show the history completely and would not recover. The
old history files are now removed before the ignore/skip cascades.
Cosmetic: curses display
Capitalize "Swap" and "Skipped" field labels
Cosmetic: web report
After the build completes, fade out the builders table in 2.5 seconds.
Nobody needs to see a table showing all the builders in "shutdown" mode.
Bandwidth: json files
A history.json file holds up to 500 log entries. 16 bytes of spaces were
removed which reduces the history file size by 8000 bytes. Spaces were
also removed from summary.json which can account for 20%+ of the size.
Only rewrite history files upon change:
The last history.json file was being written/rewritten every 3 seconds
regardless if there were changes to it or not. Now it's only written
if it's changed since the previous write. This might help reduce errors
seen by in nginx access log which are thought to occur when file changes
as the file is being served.
Address "[N] Fetch prebuilt packages" bug:
Once the /usr/local/etc/pkg/repos/00_synth.conf file was created, there
was a good change that fetching prebuilt packages would stop with a
fetch error that the checksum didn't match. This seems to be caused by
pkg(8) using both the FreeBSD and the Synth repositories and often
deferring to the latter. This is incorrect behavior as only the FreeBSD
repository should be used for prefetching. This issue is hopefully
resolved now.
Web report behavior change:
Any existing ??_history.json files are now deleted at the start of
each run. They were only litter as the javascript did not pull in old
files, so this change just removes them.
While here, rename the "Elapsed" column on the builders table to
"Duration" to match the history table lable and also adjust the widths
of the origin and information columns on the history table.
- Drop privileges in many commands
- Drop privileges when fetching a file
- Add resource limitation in sandboxes
- Add support for METALOG
- pkg delete new prints a warning for pattens that matches nothing?
Major new feature: Web-based front-end
A dynamic html report is now generated for each build. The report is
placed at <profile log directory>/Report/index.html.
The report is updated 10 times per minute. The entire build history
is retained and is searchable, but that history is reset at the start
of the next run.
In addition, several fields are clickable and trigger a quick-filter
on the history log including the "Built", "Failed", "Ignored",
"Skipped", "No." column and "ID" column (see tiptool for action hint)
and the "Total" field clears the search pattern quickly.
An earlier but representative version of the web report can be seen at:
https://github.com/jrmarino/synth#web-interface
The Synth web report pulls 2 orders of magnitude less data than the
equivalent poudriere report because the json data is split into
multiple files limited to 500 ports each (e.g. the poudriere data file
might be 1.8 Mb on a big bulk run polled every 8 seconds where as
Synth only pulls the latest segment file (50kb avg) every 6 seconds.
Synth also stops polling when the run is complete.
Minor enhancement: The log files were cleaned up
The data/phase headers and footers were modified so that the build
log appears a lot cleaner now (subjective of course).
from ${OSVERSION}. This should work around a bug in portsnap whereby the
"Fetching N metadata files" step will fail if the identical INDEX files
are generated for different major FreeBSD versions.
It turns out that the C source files were getting built with the headers
from the base ncurses. The recent update in Synth's display exposed a
resulting problem with the add chtype routines, resulting in garbled
output. Since adacurses is a static library, synth requires a revbump.
Handles remaining resizing exceptions and improves display handling.
Yesterday's work handled most of the common display exceptions, but others
were still possible. Now all possible exceptions are handled.
Several improvements were made to the display:
1) lines no longer wrap if the size width is resized too narrow; they
get truncated as always intended
2) Elements such as the elapse timer don't get displayed in the wrong
place when the screen is too narrow (they just don't show)
3) The dashes now get restored if the screen is sized small and then
big again (or started small and then expanded). In many cases those
lines just never came back before.
4) The "full" refresh frequency was increased a period of 30 seconds to
a period of 4 seconds. This has a side benefit to text-mode watchdog
as well since that's the same timer for the log inspection.
5) The history window height ranges from 10 to 50 rows. If the xterm
window starts small, the history will be 10 lines. If it starts
big, the number of lines will be dictated by the original size of
the xterm window. Making the screen small and then bigger again will
reveal the full number of log lines.
Major bug fix: ncurses display resize hang fixed
Until now, resizing the window why synth is running in ncurses mode
caused synth to hang (it would finish the builds it was working on
but the display wouldn't update and no new jobs would start). This
was due to an unhandled exception thown by ncurses binding as a result
of the resize event, and now these are handled.
Minor fix: Ports with @info in pkg-plist now pass in test mode
The mtree exclusion file was improved to allow these leftover info
directories to be ignored (as is done in poudriere. Before only
info/dir was ignored, but the presence of "dir" prevented "info" from
being removed by pkg(8) upon deinstallation.
enhancement: Augment text mode (requested)
Now when a builder starts on a new package, the port origin will be
shown in the running log (before only the completion was logged.)
Fix regression in text-mode caused by activation of watchdog.
The watchdog is checking the lengths of the build logs to figure out if
a builder has stalled. It turns out that the logs were only being
inspected in ncurses display mode, so any port that took longer than
20 minutes to build would be aborted by the watchdog.
While here, bump the *BASE* time limit for the build phase from 20 to
25 minutes based on extreme causes (normally involving gcc or tex ports)
and also bump the check-plist phase limit from 3 minutes to 10 minutes.
Some ports have tens of thousands of files in them which takes a long
time to check under test mode, especially if the server is loaded.
- If a port has another upstream, remove GOOGLE_CODE
- If a port only has GOOGLE_CODE mark it BROKEN
Some ports have a local mirror configured but for security reasons, it
is not considered upstream.
Sponsored by: Absolight
This release changes the watchdog time limits from a fixed value to
one that is affected by the average load. Using fixed values didn't
work well on extremely (and intentionally) loaded servers, e.g. an
overage load of 200 on an server with ncpu of 32.
The base times which were long to account for loading have been reduced,
and a multiplier of average-5 load / ncpu rounded to the nearest tenth
has been implemented (with 1.0 as the floor value). For example, a 4-core
machine with an average 5-minute load of 6.0 will multiply the base
timeout by 1.5. This approach worked well in an extreme test against the
entire ports tree.
This release improves robustness and activates the watchdog.
It leveraes the procctl functionality to ensure all processes spawned
from a builder are reaped, which in turn ensures that tmpfs mounts can
be dismounted. Previously stuck processes could prevent those dismounts,
trapping them as new mounts get placed on top.
This also finally enables the watchdog that will kill runaway builds.
The watchdog has a specific time limit per build phase where it will
kill the build if the log doesn't grow over the previous X minutes.
No activity timeout limits per phase are:
check_sanity : 1 minute
pkg_depends : 3 minutes
fetch : 480 minutes
checksum : 480 minutes (fetches if required)
extract_depends : 3 minutes
extract : 30 minutes
patch_depends : 3 minutes
patch : 3 minutes
build_depends : 5 minutes
build : 20 minutes
run_depends : 10 minutes
stage : 20 minutes
check_plist : 3 minutes
pkg_package : 120 minutes
install_mtree : 3 minutes
install : 10 minutes
deinstall : 10 minutes
A minor change regarding the swap display: If there is no swap installed,
it will now display "n/a" instead of "100%"
This is a minor bug fix version. Changes include:
* Set close-on-exec operation mode on popen. This is required to stop
leaking file descriptors in highly concurrent modes (e.g. 32 builders).
It's not supported on FreeBSD 9 or DragonFly 4.4 (and earlier) so this
modification is removed for those platforms.
* Remove procfs mount for lang/rust and lang/rust-nightly. This was
necessary for DragonFly, but rust has been fixed for DF 4.6 and later
* Add a stage-QA exception for entries left /var/spool/*. The presence
of these are not an identication of a port issue
* Sanitize synthexec to verify file descriptors 0 .. 2 are in use and
automatically close any file descriptor > 2 before execv fork.
Plasma5 ports
At the moment KDE ports use bsd.kde4.mk to handle their dependencies. When
working on the ports for KDE Frameworks and Plasma5 it seemed to be more
reasonable to create a new kde.mk instead of adding an bsd.kde5.mk.
The kde.mk in this review is a stripped down version of the one we are using in
the KDE Test repositories plasma5 branch [1] to only contain the parts relevant
to the current KDE4 ports in the portstree [2].
Changes to the KDE Ports needed by this:
Replace USE_KDE4 by USE_KDE [3]
Add USES=kde:4 [4]
[1] http://src.mouf.net/area51/view/branches/plasma5/KDE/Mk/Uses/kde.mk
[2] The version in the plasma5 branch also handles frameworks/plasma5 and
handles MASTER_SITES via a KDE_DIST variable similar to bsd.qt.mk for Qt
Ports -- I chose to leave this out for now, as the diff is already large
enough.
[3] I chose USE_KDE instead of USE_KDE4, USE_KDE5, USE_KDEX as the version we
want is already specified as argument to kde:<arg>
[4] For KDE Frameworks and Plasma5 ports this would be kde:5
PR: 210667
Approved by: portmgr, mat (mentor), rakuco (mentor)
Reviewed by: mat, rakuco
Differential Revision: https://reviews.freebsd.org/D6961
- sandboxes always drops privileges if run as root and chroot in /var/empty
- sandboxes limits resource usage to make them less useless on system without capsicum
- drops privileges and chroot to /var/empty in various commands if run as root:
* pkg audit
* pkg info
* pkg (r)query
- While here, add NO_ARCH and turn ECHO_MSG's into pkg-message to be visible to package users as well
PR: 211956
Submitted by: sakaue.mamoru@mwghennndo.com (maintainer)
