Changes:
- jail/ports -u: Default git pull to use --rebase.
- destroyfs: Run chflags -R 0 and on a smaller subset of files.
- ports: Ignore failures when removing the ports tree
- jail: Ignore errors trying to remove the jail
- Create port-specific directories in do_portbuild_mounts.
- Rework the cleanup to respect -x inside of the directory.
- Support -(j|p|z)<param> without space for poudriere.conf lookups
- hash_var_name: Quote the pattern to _gsub to avoid globbing local files.
- Update sh from head @ r296813
- Switch BUILD_AS_NON_ROOT as default to on.
Changes:
- ensure pre-install script failing are making the install process fail
- do not try to set attributes on directories if they already have the proper attributes
- Update to latest version 0.4.1
- Fix version snafu in dist tarball
- Add new py-docutils dependency
- Change download to GitHub (project no longer updates pypi)
Changes:
- Do not fail in case timestamps cannot be set on a directory
- Fix issue that can lead to some libraries not being provided
- Updates in pkg-repo(8)
Changes:
- rename install --from-root into install --recursive to avoid confusions
- cleanup old packages later which reduces the window when the filesystem is "unstable"
- fix issue with recursive deps when using pkg add
Changes:
- Fix a regression on systems wihtout utimensat(2)
- Revert a behaviour change with rootdir and repositories that
appeared to be more painful for users than helpful
- Fix pkg clean removing /var/cache/pkg if it is a symlink
- capsicumize pkg clean
- make pkg clean remove empty directories
Changes:
- Complete refactor of the package extraction:
* safer: ensure that only what is in the plist is actually extracted
* ensure the package can be properly on the filesystem before actually
replacing the files
* move atomicity to the package level meaning that in case of failure only
the non upgraded packages has to be proceed not the entire upgrade process
* in case of upgrade only extract the files that are unchanged
- Now pre install script failure can properly be catched and stoping the onging
process (at last!).
- New mechanism in the library events to allow frontend to know what they should
run in case of interruption
- handle ^C proprely and cleanup ongoing procedure
- lots of fixes of bugs reported by coverity
- finish implementation of the 'vital' feature
Now that the Ada Framework is based on GCC6 by default, it is no
longer required for Synth to bundle and build a local version of
devel/adacurses and misc/ini_file_manager.
Switch the port to normal BUILD_DEPENDS and remove the bundle
distfile information. No bump is required because these libraries
are statically linked into Synth, so there's no practical difference
between older and newer packages.
While there replace USE_SQLITE=x by USES=sqlite:x.
PR: 208971
Submitted by: mat
Exp-run by: antoine
With hat: portmgr
Sponsored by: Absolight
Differential Revision: https://reviews.freebsd.org/D5951
* Incorporate prefetch partial-revert patch (1.33_1) and remove the
patch and files directory.
* Add lang/rust and lang/rust-nightly to the list of ports that require
procfs to build. This is only true for DragonFly. The plan is to
eliminate this requirement, but it's a bit tricky right now. Once
done, the rust ports will be removed from the list again.
* Move /var/db/fontconfig directory exception from post-build to all
stages. The doxygen port has shown that /var/db/fontconfig changes
during the build phase and this is legitimate.
* Drop the need for PORTSDIR in dependencies [1]
* Allow a trailing ':' in URLs [2]
* Check to make sure the value(s) of SUB_FILES is healthy [3]
* Expand variables on all sides of the dependency tuple [4]
(this last change is rather big, and thus bumping minor version)
PR: 208272 [1]
207843 [2]
207477 [3]
206061 [4]
Submitted by: amdmi3 [1]
Changes:
- pkg asks again for small operations (regression)
- fix pkg auto upgrade mechanism (regression test added)
- Update libucl to latest version which fixes an issue with empty pkg.conf (regression test added)
- pkg install on an already installed package returns 0 again this regression was introduced when fixing pkg lock (regression test added)
Changes:
- Update libucl to latest version which fixes an issue with empty pkg.conf (regression test added)
- pkg install on an already installed package returns 0 again this regression was introduced when fixing pkg lock (regression test added)
The MANIFEST files were obtained by
(a) verifying the signature on the 10.3-RELEASE announcement against
the GPG key in the FreeBSD documentation repository;
(b) downloading all of the bootonly.iso.xz ISOs;
(c) verifying those files against the hashes listed in the signed release
announcement; and
(d) extracting the MANIFEST files.
This release fixes regressions:
- detection of new version of pkg could fail
- if the all process operation would have an inpact of less than 1MB on the FS
then the action is proceed with out asking the user to acknowledge it
Changes:
- Support new "vital" notion: prevent from implicit deletion
- Improved support for OpenBSD/bitrig (still lacks some functions in libc)
- Improved ZSH completion
- Updated libucl
- Lots of fixes in error messages
- Fixed "pkg lock" handling
- Allow to specify "inline" respositories in pkg.conf
- ipv6 fixes
- Allow to restrict the accepted url schemes
- More commands supports "-q"
- Multiple bug fixes
- As usual multiple new bugs
- Improved support for shlibs dependency generation when packaging base via ALLOW_BASE_SHLIBES
Changes:
- Add a basic automatic resolution plan for unsatisiable SAT problems
- Fix regression in pkg info -R
- Add a mechanism to finding libs when packaging base
- Fix some warnings on arm
- Fix hardlink extraction with --rootdir
- Improved error reporting
- Update libucl
- Fixes on locked package handling
- Add ip_version in pkg -vv output
- Add !~ in pkg query (negated glob match)
- Implement "vital" packages: prevent a package from being manually removed
- Add support for arm hardfloat
- Multiple bug fixes
Prefetching option fix:
Until now, if the "prefetch suitable packages" option was enabled, it
could happen that "synth force" and "synth test" commands wouldn't do
anything. Moreover, even though it wouldn't build, it would delete the
existing package and refetch it because force/test always pre-deletes
existing packages.
Now, ports listed with "force" and "test" commands are always built,
even if suitable remote packages are available.
Improvements with test mode:
1) Changes to /var/tmp are now ignored. These are legal modifications
2) Access time modification to the builder mount are now ignored. This
was possible if changes to /tmp or /var propagates upward (legally).
3) A new environment variable is added for test command, "LOCK". If it
defined to any value, the builder's localbase will be remounted as
read-only during the configure and build phases. This is a
diagnostic tool to help pinpoint where ports are committing file
system violations by writing to localbase during these phases.
New notice to user regard CONSERVATIVE_UPGRADE:
At the end of the "synth rebuild-repository" command, Synth will probe
the host pkg(8) to see if it's configured with CONSERVATIVE_UPGRADE.
If so, Synth will emit a notice because chances are good that the
user built repository for the local system and may try to upgrade the
system with pkg(8), and in this case, they may wish to turn off the
CONSERVATIVE_UPGRADE setting.
prepare-system, upgrade-system package deletion bug:
These two commands had a serious logic flaw that resulted in newly-built
packages to be immediately deleted prior the pkg(8) repo command. (And
the obvious result of the system not getting updated without re-running
the same command.)
The cause was that the first package validity scan marked the package
for deletion, and this setting was never reset. The obsolete package
was correctly deleted and rebuilt, but then it was quickly deleted
again during the second package scan because the previous deletion mark
was still present. The fix was simple: Reset the deletion marks before
rebuilding the repository so the second scan proceeds accurately.
Test case provided by Wapcaplet in FreeBSD forums
Pkg(8) bug workaround:
The pkg rquery man page is incorrect. It lists the invocation of
pkg rquery as "-U -r reponame" but if it's invoked in that order, it
will squawk errors if any of the repositories are misconfigured. The
command must be "pkg rquery -r reponame -U" in order to limit pkg(8) to
a single repository.
The order sensitivity came into play with "synth upgrade-system" command
on a brand new system. The 00_synth.conf pkg(8) configuration file was
created but there were no repository files created yet, cause pkg(8) to
emit errors on Synth repo when checking the official FreeBSD package
repo. This resulted in no packages prefetching even though "synth status"
indicated some would be fetched. Rearranging the internal rquery
commands works around the misleading pkg(8) documentation (or bug?).
FreeBSD-specific system root validity check.
The $SYSROOT/boot directory doesn't have to be present, but if it is,
there must also be present $SYSROOT/boot/modules directory. Synth now
checks for this on FreeBSD and stops with an informative message when
the modules directory needs to be created by the root user.
This is an enhancement release that contains a bug fix.
Description of bug:
If synth is launched from mountpoint of the ports directory
(e.g. /usr/ports), it may malfunction with strange messages such
as "invalid origins" and printing strange symbols to the screen
along with parts of a directory.
As a result, Synth now detects this launch location (as it already
did with /usr/local) and refuses to run until the current directory
is changed outside of the ports tree.
Enhancement 1:
It is now possible to remove alternate profiles. When more than
one profile exists, a menu option "<" appears that provides the
user with an opportunity to remove one to all inactive profiles
from the configuration. The man page has been updated as well.
Enhancement 2:
Synth will automatically convert any directory inputs on the
configure command to the true path. For example, if somebody types
in "/usr/xports" for the ports directory, but that's just a
symbolic link to /vcs/freebsd-ports", the entry will automatically
convert to its true resolved path (e.g. /vcs/freebsd-ports). This
enables the cwd detection of the bug fix above to always work.
Two minor bug fixes:
* A specific check during test mode would emit a failure to stdout when
testing devel/py-setuptools27. It turns out that there's a file there
with a space in the filename. The filename was an argument for
/usr/bin//file and it wasn't escaped. The file in question had
parentheses too which the shell was trying to process. The fix was
to escape the filename in the /usr/bin/file command.
* The builders were mounting the source directory from "/usr/src", not
$sysroot/usr/src as intended. This potentially causes breakage when
the $sysroot reflects a different versions/release than the host
machine has (e.g. making FreeBSD 10.2 packages on FreeBSD 11-current).
Now the source directory mount is relative to profile's $sysroot.
With portest, you can create, update, patch and revert a port tree with
svn, git and portsnap. Portest can also do portlint testing, test a port
with port test, or poudriere, and also generate a list of ports a
patchfile(s) will modify. Portest is written in shell with minimal
dependencies mostly using FreeBSD-base
WWW: https://github.com/Ultima1252/portest
PR: 207810
Submitted by: Ultima1252@gmail.com
Fix distinfo for the offending ports.
lang/yorick's tag was moved, and the added patch was no longer needed.
PR: 207644
Submitted by: mat
Exp-run by by: antoine
Sponsored by: Absolight
Differential Revision: https://reviews.freebsd.org/D4268
I knew the repository signing enhancements weren't fully tested, but it
was important to get the regressions fixed. I thought that if there were
any problems with the signing, people not using those features (e.g. 99%
of users) would not be affected. Unfortunately, that's not true.
Due to a simple boolean logic error, an inaccurate and harmless warning
about not finding the signature fingerprint will show every time the
repository is rebuild. This minor releases fixes that logic.
This was meant to be a pure enhancement release to support the signing
of the repository, but there were some regressions brought in with
version 1.1x and also some long standing bugs found.
Fixed:
* Regression: Interactive test mode
Changing the TERM setting to "dumb" semi-broke the interactive shell
when ENTERAFTER is used with the test command. The fix was to set
TERM back to cons25 in this particular case.
* Regression (serious): install command
The improvement in speed of package validation for the rebuilding of
the repository resulted in a new bug that caused the "synth install"
command to install every port in the repository. It was caused by
reusing a container that I thought was available, but it turns out
that the "install" command still needed it.
* Bug: handle exception that occurred when Synth asked a question
when no TTY was attached. Running a command like "force" would result
in a question, and if Synth was running in a cron job or having args
piped to it via xargs, no terminal would be available and Synth would
throw an exception. Now in this case, Synth just assumes answer to
question is "No" and ends gracefully.
* Bug: purge-distfiles command would delete everything if any error was
encountered. It should have just stopped, which it does now.
* Bug: purge-distfiles container had a bug which prevented all the
distfiles from being stored, which caused removal of good distfiles.
* Bug: purge-distfiles container didn't check for duplicates, resulting
in unhandled exceptions. Now fixed, purge-distfiles command works now.
enhancements:
* Show version information on status commands. On the display, a symbol
of "N" (New), "R" (result), or "U" (upgrade) shows next to each line.
Additionaly for "U", the old version and new version are shown. The
same information is shown in the log, but in addition the actual
version for "N" and "R" categories is also listed.
* Test mode: Add filters from poudriere to prevent false test failures
with leftover and modified files. Mostly it involves files that get
modified, but it's okay so the MD5 check failures need ignoring
* Support signing repository with local RSA keys. Adjust man page with
two new FILES (public key and private key) along with an example on
how to generate those files.
* Support signing repository with external signing command and
fingerprints. Adjust man page with two more new FILES and an example
of how configure this feature.
This fixes a regression in building ports that have dependences that
install kernel modules. When DTrace support was added by providing a
read-only mount of /boot to the builder, the kernel modules could no
longer be installed at /boot/modules by pkg(8).
Previously, although successful, module installs would have caused a file
system violation on test mode checks. Since /boot is now excluded from
checks (since DTrace support), leftovers in /boot/modules will not be
detected in test mode. The fix is too elaborate and FreeBSD-specific
to worry about (plus there's the philosophy question about why the ports
framework is even allowed to modify the base but that's out of scope).
This release addresses unacceptably long repository rebuild times for the
worst cases (FreeBSD [1], slow CPU, slow mechanical disk). Until now,
rebuilding the repository required a full tree scan (nearly 26k ports).
While I only saw around 4 minutes on a 4-year DragonFly machine equipped
with a SSD, others reported times exceeding 20 minutes.
This new method scans existing packages twice -- first to eliminate those
packages where the port was removed and also those with a mismatching
version (parallel). This sets up a second pass to serially and
recursively scan the ports of the remaining packages. That leads to the
final validation (same as previously done) and the actual repository
generation. Now the repository generation time is much shorter, but
corresponds to the number of build packages in the packages directory.
The long repository generation times were identified prior to the 1.0
release, but I targetted 1.1 for the enhancement.
bug fixes:
* cron operation fixed
The terminal is unset for cron jobs. The setting of TERM to "cons25"
in the environment caused a failure, but it's fixed by changing the
value to "dumb". Note that curses must be off in the profile invoked
by cron.
* dtrace building fixed [1]
On FreeBSD, /boot is now read-only null-mounted if it exists in the
profile's system root. This fixes building dtrace probes when the
option is set.
* fix muted pipe exit status
Pipes have to be read even when no output is expected in order to get
the correct exit status
* fix pipe child exit status
The previous implementation mixed child and parent exit statuses in
error; now they are isolated and returned correctly.
Enhancement:
If a muted pipe command fails, an appropriate error message is given
including which command failed. This helps identify the real error
instead of repercussions appearing later which mask the problem.
PR: 207361 [1]
Currently, the application is a MVP (Minimum Viable Product) which
provides only the following features: packages listing and searching,
installation and deinstallation of packages, and filtering based on
the package state.
WWW: https://github.com/culot/portal
Any attempt to alter echo and control flags when the standard input
stream is not connected to a tty will fail. Fix the resulting nuisance
messages by skipping these steps if a tty is not present.
Reported by kmoore@ on jenkins
The echo and control-flow flags weren't being restored correctly in
general, and they weren't being restored at program exit. The result
on PuTTY was that nothing would echo to the terminal after Synth
exited, but the new version fixes it.
I'm pleased to finally issue the first release of Synth. There were some
significant changes since the last release candidate:
* Rework piped command handling to fix command parsing
Synth was using ada-util to execute external commands and receive the
output. Unfortunately, this implementation currently can not handle
either extra whitespace nor can it support quoted values. The author
is going to fix this soon using popen and pclose for Unix, but I just
implemented the functionality by binding to those functions myself.
* This was the only use of ada-util (I originally expected to use more
of this library's functionality) so it's been dropped as a dependency.
This reduces the stripped executable size by about 30%.
* The environment was prepopulated with UNAME_[x] variables. This is
required for Synth to support building packages for different releases
or even architectures (e.g. building 10.2 packages on 11.0-CURRENT).
* Define OSREL in the builder's /etc/make.conf. This seemed to be
required for emulators/virtio-kmod which uses <pre> although it's not
clear defining OSREL is necessary.
* Fix graceful shutdown handling in text mode.
When the graceful shutdown key was changed from Escape to Control-Q, it
stopped working in text mode (i.e. when it wasn't in curses). I believe
that's because the curses display uses the "raw" tty mode and lets all
control characters through. In text mode, Control-Q has a flow control
functionality. I wrote a routine to disable that flow control and give
Synth access to control-Q keypress in all cases.
* The same routine disables TTY echoes when they are unwanted
* Fix "status-everything" dry-run issue where a graceful shutdown resulted
in an unwanted "sorry" message.
* In test mode file system violation check, mark changes to /var/run
during building to be excluded from checks.
* Add a guard that checks if Synth is launched when the current directory
is <sysroot>/usr/local (or a subdirectory of it). Synth will fail
to mount everything in this case, so now it stops and tells the user
to change directories and try again.
/usr/local/share/poudriere/MANIFESTS, where poudriere (as of version
3.1.11) checks for pre-distributed MANIFEST files before fetching them
from the ftp/http/https server.
This allows poudriere to ensure that the bits it is downloading and
installing really match the bits provided by the release engineering
team, and have not been subtly trojanned in transit. (Note that this
does not apply if poudriere is creating a jail from -STABLE or -CURRENT
since we cannot pre-distribute those MANIFESTs.)
The MANIFEST files were obtained by
(a) finding the GPG-signed announcements for 9.0 and later releases;
(b) verifying those signatures against the GPG keys in the FreeBSD
documentation repository;
(c) downloading all of the relevant bootonly ISOs;
(d) verifying the ISOs against the hashes listed in the signed release
announcements; and
(e) extracting the MANIFEST files from those ISOs.
Reviewed by: bdrewery
Security: If someone could trick you into building packages in a
world which they tampered with, they could do all sorts
of nasty things to those packages...
Changes:
- Optimize jail startup/cloning/rollback and mtree comparisons.
- api: Make more useful
- check_fs_violation: Use mktemp rather than hard-coded tmpfile
- Fix false-positive 'Leftover processes' warning after disabling jexecd
- Conditionalize the jexec logic around USE_JEXECD and add back missing jkills.
- QEMU: Remove _MAKE_JOBS not present in release-3.1 and noted to break
builds by sbruno
- Kill processes before remounting/rolling back the builder FS, rather than
after.
- jail -c: Support older FREEBSD_HOST=ftp.freebsd.org URL schemes still. [1]
- jail -c: Support pre-distributed MANIFESTs for checksum comparing.
PR: 206848 [1]
Unfortunately, there's been a bit too much change since 0.99_6 to
confidently release version 1.00, so another release candidate is
necessary. Both new features and bug fixes were added.
New features:
* Provide ability to define environment variables in a profile
(/usr/local/etc/synth/<profile>-environment)
* Support fetching by proxy using these environment variables
* Add zsh and bash completion scripts
* Accept port origins with trailing file separators (so people
using completion scripts don't have to backtrack to remove them)
* In text (non-curses) mode, output the current package build
tally every 200 seconds (approximately)
Bug fixes:
* Fix support for system roots that don't match host (e.g.
ARCH, OSRELEASE, OSVERSION, etc
* Fix ABI check for system roots that don't match host
* Remove effect of system /etc/make.conf (originally seen when
MAKE_JOBS_NUMBER was defined there and disabled synth)
Changes:
- Fix hardlinks extraction with --rootdir
- Relax url scheme acceptance to allow file:/ and file://
- Accept pkg+ for url scheme where it was never intended to be used, because users used it anyway
- Remove pkg-message
Torsten has accepted the challenge of whipping portmaster back into
shape. In addition to changing MAINTAINER, he's starting with fixing
both non-default options by providing their necessary RUN_DEPENDS.
PR: 207075
I had hoped that 0.99_5 would be bug-free and the basis for the first
release (1.00), but couple were found. If use of 0.99_6 reveals no
further issues after a week or so, I'll re-release it as v1.00.
bugs fixed:
* if the origin started with a directory separator, an exception would
occur. Now it properly labels it as an invalid origin.
* The "extract" stage was labelled as "checksum". Internally everything
was fine, but on the display, the order was checksum, extract-depends,
checksum instead of checksum, extract-depends, extract.
* During one phase (build), the DEVELOPER flag was set unconditionally.
This was a regression as it wasn't always the case. This code was
tweaked several times since 0.99_5 and now the DEVELOPER setting has
been moved the builder's make.conf to ensure it's consistently
present or absent (as needed).
* It turns out that the ports tree scan is affected by the DEVELOPER
flag. It turns out the setting can affect the dependencies list so
it needs to be set (or absent) appropriately to match how it will be
on the builders. The make.conf solution above solves this too.
* There was "NO_BACKUP" set in the builders make.conf. This line is
for the DragonFly src builder and it's presence caused no harm, but
it's been removed now.
* Make ports makefile respect CFLAGS
new feature:
* Provide ability to break into a build at a specific point and
interact with it.
- Only available on "test" command
- Only active when one (1) port origin is given to "test" command
- Only active when ENTERAFTER is defined in environment as:
> extract
> patch
> configure
> build
> stage
> install
> deinstall
- All dependencies are built first with typical display and
DEVELOPER=1 set. Afterwards, Synth converts to text mode and
builds the specified port up to and including the phase specified
by ENTERAFTER. Then it launches a tcsh shell and gives control
to the user at the builder's root directory
- The user ends the interactive session with the shell cmd "exit"
- Synth will clean up and exit (it will not try to continue the
build due to possible corruption from the users)
* This is a port developers tool. The average user does not need it.
* The average user might use "test" command to generate a log to submit
as a FreeBSD Bugzilla PR attachment.
A fairly recent change caused a regression after a build was complete.
Previously a "tally" or summary of the build would appear after the
ncurses screen was restored to the regular terminal mode. It would
list how many ports were built, failed, etc. After the regressin, it
just ended abruptly.
This commit restores the tally to show as it did previously.
While processing Issue 206746 [1] for a security update to
security/py-rsa (For versions < 3.3), it was noticed that Portscout
had not identified the the newer version, released on 2016-01-13.
Investigation revealed that the PyPI SiteHandler in Portscout only
processed the first url/filename returned by PyPI, which in many cases
is not a tar.gz, the default EXTRACT_SUFFIX for source distribution
(sdist) files:
[py-rsa] VersionCheck()
[py-rsa] Checking site: https://pypi.python.org/packages/source/r/rsa/
Does site handler exist ... Yes
(Portscout::SiteHandler::PyPI) GET https://pypi.python.org/pypi/rsa/json
(Portscout::SiteHandler::PyPI) GET success: 200 Filename: rsa-3.3-py2.py3-none-any.whl
FindNewest: Checking rsa-3.3-py2.py3-none-any.whl ... against port DISTFILES.
FindNewest: Checking DISTFILE ... rsa-3.1.4.tar.gz (ver: 3.1.4, sufx: .tar.gz)
[py-rsa] Done
This change backports a commit [1] made to Portroach which adds a loop to
enumerate all URLs/filenames in the PyPI JSON response, not just the
first.
[1] e93b8331f6
PR: 206746 [1]
Obtained from: https://github.com/jasperla/portroach
This is a minor update to synth, which includes:
* Support for the WHYFAIL environment variable. If this variable
is defined (to any value) in the environment, Synth will turn on
the "debug" mode for dependency and option sanity checks. This
mode will provide exact details on how the package failed the check.
* README.md: editorial corrections, 3 images replaced to reflect current
version of Synth
* Man page: editorial correction, WHYFAIL documented, and the "Impulse"
indicator was documented (in NOTES section)
* Significantly improve ports scan error messages. In particular,
eliminate the 'bad value ""' messages that are caused by ports that
are partially or completely missing. Also propagate exception
messages when helping.
* Log 03 (ignored ports) did not list the actual ports, only the reason
the port was ignored. Fix bug to show category/port too.
Erratum on previous commit message: The "Graceful Shutdown" is initiated
with Control-Q, not Control-C! The typo is doubly unfortunate because
Control-C will exit Synth without cleaning up the mounts.
WARNING: rebuild-repository command has changed action! see below!
The follow changes have been made since v0.99_2:
* Change the graceful shutdown key from "Escape" to Control-C.
The former was easy to hit inadvertently (reported) and could be
interfered with by terminal ANSI codes and/or mouse wheels. The
documentation has also been modified to reflect this change.
* Fixed bug where installed packages that no longer had a port
might cause the scan to fail rather than be ignored as advertised
* New feature: SYNTHPROFILE environment variable
When SYNTHPROFILE is set toTill be loaded rather than the default
profile. This is aimed for synth's use in scripts.
* The "rebuild-repository" command has been renamed to "prepare-system".
This is partly because the former command will be repurposed.
* A new command assumed the name "rebuild-repository"; it performs a
sanity check on all the built packages, removes the bad ones, and
rebuilds the local Synth repository on command. It is primarily for
scripting use, but it has other legitimate uses.
* Fix case where prefetching packages would try to update a non-existent
local Synth directory. As a consequence, prefetching is only done
from a single external repository (the normal use case thought)
1) Fixed false "fetched failed" messages that always appear after
prebuilt packages are fetched
2) Fix bug where "synth configure" command would not run if any directories
were invalid. For new systems, /usr/ports/distfiles is always invalid
3) Following 2), greatly improve error message by saying exactly which
directory is missing and which configuration letter it pertains to
4) If synth is configured to a non-existent /usr/ports/distfiles directory,
also add a recommendation to consider a better location outside of the
ports tree and remind them to set DISTDIR in /etc/make.conf too.
Changes:
- jail -c: Always copy in QEMU emulator, even if not building from src.
- jail -c: Ensure MANIFEST is not empty or give an error.
- Clear CMD_ENV to disallow sneaking in environment vars
- Update libnv to the latest head @ r293361
- Update sh from head @ r293359
- Null-mount read-only /rescue and /nxb-bin into the jails as they will not
change.
- Add a MUTABLE_BASE option (default yes) to experiment with null-mounting of
base.
- Prefer https://download.FreeBSD.org for snapshot downloads from FreeBSD.
- Remove duplicate NM from make.nxb.conf.
I've been ordered by portmgr to remove DEPRECATION designation because
others have indicated they believe people should not be so directly
informed of its poor state. Despite the fact that there was no expiration
date set and that functionality was not affected in any way (leaving now
informed people free to use this unmaintained port), it was considered a
disruptive change.
Let the record show that I strongly object to this decision and that I
firmly believe that portmaster is a port that *must* have a competent
maintainer that can *develop* it. It should *not* be allowed to be
unmaintained and still maintain a presence in FreeBSD documentation.
1) When using prefetch option, list the packages that failed to download
rather than just say, "at least one failed to download"
2) sysutils/htop requires linprocfs but doesn't set USE_LINUX. Set this
port to mount linprocfs based on its origin
3) Fix linprocfs implementation, it was mounting out of order, basically
resulting in that it was non-functional
4) Close all the logs in the case where no packages are built. In that
case, the logs were never modified. Changes discarded?
Changes:
- Allow to limit the valid url scheme in pkg.conf
- Add support for SOURCE_DATE_EPOCH: https://reproducible-builds.org/specs/source-date-epoch/
- Fix pkg check -d
- Fix testsuite with kyua 0.12
- Fix completion on pkg info -l
- Reduce te probability to get an infinite loop on pkg version check
- Fix pkg version -q
- Update libfetch to the latest version from head
- Debug mode is now verbose when fetching
- Fix segfault with empty HTTP_USER_AGENT
- Fix segfault when parsing invalid URL
- Fix build on OS X
- Better error reporint on issue during pkg create
Note that this release will be merged to quarterly branch in 10 days
Changes:
- turn libfetch into verbose when pkg is run is debug mode
- remove all mention of MTREE_DIRS in documentations
- update libfetch to the latest version from HEAD
- Fix segfault with HTTP_USER_AGENT is empty
- Fall back to the default user agent if the provided on is empty
- Fix segfault when bad url are provided in the repository configurations
- Fix error reporting when an error occurs during package creation
The portmaster script hasn't had an official maintainer in 9 months and 2
years before that it was only patched in reaction to changes in the ports
framework. There are many unclaimed PRs in the bugzilla database, many
known bugs, and several areas where portmaster no longer aligns with how
ports work today. The problem isn't simply getting a maintainer; that
person has to be a ports framework expert and it appears that the people
with these qualifications don't want anything to do with this port.
Moreover, there are better options available. All FreeBSD platforms
support ports-mgmt/poudriere (although some many struggle under the load)
and the most common amd64 and i386 platform users have the additional
option of ports-mgmt/synth which is user-friendly, lightweight, and aimed
at users of portmaster, portupgrade, and even poudriere.
Unless something drastic regarding portmaster occurs, it's nearing its
natural EOL, so it's users should evaluate alternatives and try to
migrate off of it.
Now Synth has all the features envisioned for the first release. This
edition add an option (off by default) to fetch prebuilt packages if
they are suitable (ABI, options, dependencies match). This feature is
aimed at people that only want to build ports with customized options,
but for ports with default configurations, they are happy to use the
official packages. The feature is not heavily tested yet.
Once this version is sufficently tested, Release 1.00 will follow.
Other changes since 0.98_5:
* curses will redrawn itself (correctly now) every 30 seconds to
fix any corruption that may have occured
* Synth everything will build the repository without deleting packages
now (this was disabled on the last update)
* The repos directory for pkg is read from pkg config instead of using
the hardcoded defaults
* The man page has been updated with new feature descriptions
* linprocfs is mounted for linux_base ports
* The regression that prevented linux ports from building has been
fixed. It was caused by caching LINUX_OSVERSION, so this variable is
no longer cached.
The following changes have been implemented:
* The "test" command checks for file system violations between
the configure and build targets (inclusive)
* The "test" command hecks for leftover (extra), missing, and
unexpectedly modified files and directories between the stage and
deinstall targets (inclusive)
* Fix bug where success system-upgrade was indicated as a failure
* Bring in procfs mounts for x11-toolkits/gnustep-gui (only!)
It appears to the be only port that needs it, but procfs appears to be
pretty unstable, so we don't mount/dismount it unconditionally
* Similarly, change linprocfs mounts/dismounts to only occur when when
linux ports are building. Linprocfs stability is unknown (and I can't
test it on DF) so be conservative and use it as little as possible.
* Fix bug on builders /etc/group file (some groups were missing)
* Install /etc/master.passwd in builders, it is required for at least
one port
* Install /etc/rc.d and /etc/defaults/rc.conf in builders. It is
required for at least one port
* Disable repository rebuild after synth-everything. Twice it has
removed all packages (over 23,000!) after a build, so there's a bug
or missing safeguard there.
* Watchdog status: Situation is better if scons ports are unwatched, but
python3* freezes along with a handful of other ports. It works 99% of
the time, but not reliably enough yet to re-enable.
testing FreeBSD Ports. redports-node is running on each building
machine which asks for new building jobs from the master and
builds them using poudriere.
WWW: https://freebsd.github.io/redports/
After setting the group process ID and then execv'ing into the
ports build target, any port using scons just locks up immediately
without any kind of message. I assume it's a bug in scons but it
will take a while to resolve (probably requires a testcase case and
submitting upstream). Since it caused a significant regression and
since normally users shouldn't hit hung processes often, I'm disabling
the feature for now.
The following changes have been implemented:
* The builder command executer had been upgraded with a watchdog. It
does not monitor the overall time of a phase (things like fetch /
checksum vary depending on the internet connection and the volume it
needs to download ranges from bytes to gigabytes), but it does monitor
log progress. Each phase has a maximum amount of time allowed for the
log to be static (measured in lines, not file size). If the log is
static for too long for that phase, the processes of the builder will
be killed, and the builder log updated accordingly.
* The load indicator was stuck at 0.00 for some named locales (only on
FreeBSD) so this was resolved.
* Ports tree scanning time was cut nearly in half by caching make
variables on each builders make.conf
* Support for /usr/lib/lib32 was added for FreeBSD
* purge-distfiles command was improved by handling potential exceptions
and fixing the case of 100-1023 Mb purged (range was too narrow)
* Typos on man page fixed
* The directory ${PORTSDIR}/packages are now ignored. This is the
default package location and any existing packages were getting
treated as port directories.
* Skip some additional questions/actions if a graceful shutdown was
previously detected
On DragonFly, $LOCALBASE/etc/pkg/repos always exists, so I forgot to
verify that. On FreeBSD, it doesn't necessarily. Make sure to create
this path if it's missing before trying to write 00_synth.conf there.
While here, add an error message if it still fails for any reason.
The -devel version of tinderbox hasn't seen an update in over three
years and it's significantly older than the stable version. Since there
doesn't appear to be a need for -devel version anymore, let's just
remove it on 14 Feb 2016. (Don't forget to set secondly field in MOVED
accordingly!)
PR: 206138
Approved by: crees (maintainer)
The big one: FreeBSD distfiles are located by default inside the ports
tree (which is kind of insane). I forgot this and people that did not
modify DISTDIR had this directory full of files, which synth interprets
as non-functioning ports. Make directory scans skip this directory if
found to correct.
The smaller one: Synth was supposed to squawk if it couldn't find a
valid portsdir at the environment (PORTSDIR), /usr/dports, or /usr/ports.
The squawk didn't work. Moreover, it wouldn't check PORTSDIR set at
/etc/make.conf. This change makes synth check there after the env.
check (thank you for the hint, bapt@). It also correctly squawks
without creating synth.ini when PORTSDIR isn't valid.
Synth is a custom packge repository builder for FreeBSD and DragonFly.
It is intended to replace Portmaster, portupgrade, and poudriere for
the average user. It is simple to learn (the powerful options are
limited in number) and user-friendly, but it is extremely fast due
to its parallel building capability. It will "drop-in" on any system
as it leverages the stock pkg(8) facilities. All ports are built
in a clean environment, so it is finally safe to build ports as
needed on a live system. The default profile is the system itself, not
a new jail, which can be a valuable feature for some environments.
(more text on pkg-desc).
It's ready for wider testing now.
Fix a logic bug introduced in r401037, which caused ports without a
SiteHandler (GitHub, SourceForge, PyPI) not to be processed through the
fallback FTP or HTTP(S) cases.
I didn't pick this up in testing because only my ports were tested, most
of which use PyPI/GitHub, using the `restrict maintainer` setting in
portscout.conf.
[1] https://svnweb.freebsd.org/changeset/ports/401037
Pointyhat: koobs
Reported by: kwm (for mate, libdrm)
MFH: 2016Q1
A number of people reported no updates being detected for ports using
USE_GITHUB=yes, even after originally adding the GitHub site handler in
r401037 [1]. Investigation revealed that the FindNewestFile subroutine
and the vercompare() method assumed (or are designed so) that responses
returned from site handlers will be in a normalised version format. For
site handlers that return 'versions', this works well. For the github
handler, in the fallback use of the API for fetching repository tags, it
does not.
Additionally it turns out, portscout currently only uses/stores a
normalized version ('ver') in its database, in its general design
attempt to be a generic version comparison tool
In particular, portscout does not reference or store PORTVERSION
or DISTVERSION{FULL}, so we have nothing 'canonical' to compare
the responses from Github (tags) against.
This change special-cases Github in the FindNewestFile subroutine, which
was obtained via Portroach [2] (OpenBSD's portscout fork).
Extending this, we also now only match version-esque looking strings
from the tag, in an attempt to normalise, because they come in many
forms, including {foo-}X.Y.Z{-bar}, foo_X_Y_Z, among others.
While I'm here,
* Fix copypasta of $github_client_id, when $github_client_secret was
intended
* Add code to use authenticated requests for Github project tags in the
fallback (to /releases) case.
* Add and update some more debug messages to help diagnosis of future
issues
Special thank you's to:
* matthew, allanjude, mandree, des, Brendan Molloy for your regex, perl
help and moral support.
[1] http://svnweb.freebsd.org/changeset/ports/401037
[2] https://github.com/jasperla/portroach
Differential Revision: D4780
by simply providing the name of the module to the command. It accesses the
PyPI JSON API and does its best to generate a fully functional Makefile and
distinfo where sources are found.
WWW: https://github.com/freebsd/pytoport
PR: 205636
Submitted by: Brendan Molloy <brendan+freebsd@bbqsrc.net>
Changes:
- list of valid url scheme can be defined in pkg.conf
- add support for SOURCE_DATE_EPOCH for reproducible builds
- Fix pkg check -d
- Fix testsuite with kyua 0.12
- Fix pkg info -R
- Fix setting modes on extracted files/directories
- Fix zsh completion for pkg info -l
- Fix some solver crashed
- Initial attempt at fixing infinite loop on multirepositorry when updating pkg
- pkg version now handled -q correctly
* Attempt to properly detect gettext [1]
* Add support for @<pre|post>[un]exec plist directives [2]
* Add support for the new @sample plist directive changes [2]
* Modify the text when checking for used OPTIONS [3]
PR: 204370 [1]
204900 [2]
204575 [3]
${PERL5} points to a specific version of perl, say, perl5.22.1, it is
fine to use it in a ports Makefile to do Perly things, but ports using
it must use ${PERL}, that points to /usr/local/bin/perl so that if the
minor version is updated, the shebang keep working.
While there, make some ports use shebangfix, regen a few patches, and
bump PORTREVISION where a shebang went from PERL5 to PERL.
PR: 205367
With hat: portmgr
Sponsored by: Absolight
Changes:
- jail -d: If stdin is a TTY, confirm before removing jail.
- Disable rexec/jexecd for now as it has issues on <head and seems racy on
head as well, possibly leading to "Unable to execute id(1) in jail."
- QEMU: Use host pkg-static in the jail for repo to speed it up. For
this PKG_REPO_FROM_HOST=yes may be needed on mips.
This brings in almost a year of updates that had been held back.
Changes: https://github.com/freebsd/poudriere/wiki/release_notes_32
- New sub-command 'image' which is still in development, for generating
install images.
- Some poudriered fixes, but it is still experimental.
- DISTFILES_CACHES may be 'no' to use distfiles/ from ports tree.
- A cache daemon has been added in, off by default, for experimentation.
- The ports 'environment' feature from Mk/Scripts/functions.sh:export_ports_env
is used for dependency gathering now, which avoids 8 or so fork/execs for
every make -V call.
jail:
- New -K [KERNCONF] flag to build and install a kernel into the jail.
- New -S <path> flag to specify what srcpath to use from the master jail,
rather than /usr/src.
- Git support.
- New -U flag to specify what url to fetch from for git and svn.
- EXTRA_DISTS added to add to the distribution list fetched/extracted, from
poudriere.conf.
- NO_LIB32 added, when set to 'yes' skips installing lib32 distribution.
ports:
- Support '-m none' to disable portsnap usage.
- New -U flag to specify what url to fetch from for git and svn.
Build process handling:
- A jexecd process is spawned in the jail and communicated with via rexec
using a pipe. This is intended to lessen the jail locking contention,
allow more easily cleaning up processes, and limiting of TTY. It is
still considered experimental.
Changes:
- Fix cross building
- Improved zsh completion
- Improved error messages
- Improved documentation
- Bug fixes in the solver
- Allow to specify repositories directly via pkg.conf
- Allow multiple version of the same package in a repository
- Fix ipv6 testing in configuration
Changes:
- Fix cross building
- Wording fixes
- Always restore credentials after extraction in preparation for bigger changes
- Improve zsh completion
- Update libucl to the latest version
- Added support for OpenBSD/Bitrig
- Add a latest fixes to kill origin usage
- Improved linux support
- More user friendly error messages
- Fixes in the solver: Treat dependencies missed as errors
- Improved regression framework
- Allow to specify repositories directly un pkg.conf under the "repositories"
keyword
- All multiple version of the same package in a repository
- Fix configuration regarding ipv6
Changes:
- Fix 'builder stop' hook, which never worked.
- Don't transcend mount directories when running rm -rf
- Stop exporting MY_JOBID
- Run 'builder stop' hook always in stop_builder() [this means the hook
runs at startup as well to clean out stale builders].
- Enable post-mount jail hooks.
Unless something major is wrong here, this will be 3.1.10 and the next update
to -devel *will be* master which has a year of pending updates.
Changes:
-Always call make by absolute path of /usr/bin/make.
-Update stale comment about MFS
-Fun things happen with /nonexistent exists. Ensure it doesn't.
-testport: Allow -o to be optional.
-Mark poudriere-queue experimental.
-Using ccache on 10+ is fine so long as we use a PATH.
-XDEV: Don't endlessly append to make.conf on updates.
-Fix leftover lock file in /tmp at cleanup
-Support in-tree ccache for buildworld on recent head
-Always use the FAST_DEPEND option for buildworld.
-Go crazy, use -j for installworld targets on recent head
-Allow comments on the same line with port entries in blacklist
Over the past several months portscout.freebsd.org appears to have been
more frequently NOT finding updates, particularly for ports that use
CHEESESHOP (PyPI) as their MASTER_SITES.
Portscout has also never worked for ports using GitHub for distribution
files due to the following:
a) Portscout, prior to 'guessing', requests a randomly named file
from the Site and expects a 4xx (404) in response. If it doesn't
receive a 4xx response, it increments a 'lie counter' and does not
check the site again in the next run.
b) The GitHUB handlers (SUBDIR/MASTER_SITES) in bsd.sites.mk
construct a URL that ends in a a dummy query paramater (for the
filename), so that fetch saves the correct filename to DISTDIR.
This means for any DISTFILE name provided, a 200 OK response is
returned
These two factors unfortunately leave us in a position where there is no
good way to workaround this in the ports framework, including overriding
DISTFILES, DISTNAME, FETCH_ARGS, or the SUBDIR URL itself for various
reasons (not matching distinfo, file conflicts in DISTDIR, etc)
Fortunately, the portroach project (OpenBSD's fork of portscout)
contains a site handler for GitHub and PyPI (among others) already [1].
These site handlers use API endpoints at GitHub and PyPI that respond
JSON respectively, providing a faster and more accurate way to determine
the latest version of a package, without having to go through the
'guessing' process.
This commit:
- Adds GitHub and PyPI site handlers, and modifies or extends them to
accept/match our MASTER_SITES URL's.
- Adds authenticated API request support and two settings for the
GitHub site handler
- Add p5-JSON to RUN_DEPENDS (needed by new site handlers)
- Add HTTPS option for supporting https:// MASTER_SITES. Currently
portscout does not check (fails) https:// MASTER_SITES [2]
- Take MAINTAINER'ship
- Adds badly needed logging/debugging messages to key parts of the
process retaining the conditional logic that ties the verbosity to
"quiet" or "debug" portscout.conf settings.
- Renables the SQLITE3 option (previously commented out) and renames it
to SQLITE (the standard, as per bsd.options.desk.mk)
- Creates a DATABASE option group allowing either/or SQLITE or
POSTGRESQL to be selected
- Switches option conditionals where possible to options helpers
- Backport a fix for maintainer matching/mapping [3]
- Adds LICENSE (BSD2CLAUSE)
- Updates and sorts pkg-plist
This change was tested again ports maintained by me, and resulted in
'new versions' being found and reported for 42 of my ports (of 123).
[1] https://github.com/jasperla/portroach/tree/master/Portroach/SiteHandler
[2] Reported by: truckman
[3] 2f6ee134dd
PR: 203996
Approved by: maintainer (timeout, 15 days)
Differential Revision: https://reviews.freebsd.org/D3996
* Add support for @(...) notation [1]
* Remove the check for USES being sorted. Order is important. [2]
* Add support for making sure @owner and @group are properly reset [3]
PR: 202570 [1]
203908 [2]
202711 [3]
Changes since pkg-1.5 series
- Lots of improvements in the solver (in particular fixes the case like the
recent jpeg upgrade)
- Lots of fixes in the 3 way merge code
- pkg add can now work without a version specified in the dependency line
- pkg check -d now also check the required libraries
- Improved support for partial upgrades
- Improved zsh completion support
- Improved linux support (now all regression tests passes on linux)
- Messages can now be context aware: (only print a given message during
installation, upgrade - version aware -, removal, or always)
- @keywords now accepts new entries to add context aware messages
- Add the ability to generate graphiz's dot format representation of the
solver's problem
- pkg search now default on showing the comments of of the matched packages
- Lots of bug fixes and code cleanup
- Document @{pre,post}[un]exec in plist added in pkg 1.5
Both portshaker and portshaker-config tries to manipulate the portshaker.conf
So converting this one to @sample make it conflict with the portshaker package
Reported by: mat
Pointyhat to: bapt
Changes:
- Important fixes in the solver (fixes the jpeg->jpeg-turbo issue)
- +DISPLAY can now be a ucl file (contextual messages)
- pkg info -D now understand ucl format
- keywords can now define messages to be printed (depending on the context)
- PKG_ROOTDIR is now always set in scripts (default to /)
- pkg version now accept -n <pkgname> to check a given package version against
remote repository or ports tree or INDEX file
- Improve Linux support: testuite now passes entirely on linux
- pkg register only tries to load +DESC if it exists
