On DragonFly, $LOCALBASE/etc/pkg/repos always exists, so I forgot to
verify that. On FreeBSD, it doesn't necessarily. Make sure to create
this path if it's missing before trying to write 00_synth.conf there.
While here, add an error message if it still fails for any reason.
The -devel version of tinderbox hasn't seen an update in over three
years and it's significantly older than the stable version. Since there
doesn't appear to be a need for -devel version anymore, let's just
remove it on 14 Feb 2016. (Don't forget to set secondly field in MOVED
accordingly!)
PR: 206138
Approved by: crees (maintainer)
The big one: FreeBSD distfiles are located by default inside the ports
tree (which is kind of insane). I forgot this and people that did not
modify DISTDIR had this directory full of files, which synth interprets
as non-functioning ports. Make directory scans skip this directory if
found to correct.
The smaller one: Synth was supposed to squawk if it couldn't find a
valid portsdir at the environment (PORTSDIR), /usr/dports, or /usr/ports.
The squawk didn't work. Moreover, it wouldn't check PORTSDIR set at
/etc/make.conf. This change makes synth check there after the env.
check (thank you for the hint, bapt@). It also correctly squawks
without creating synth.ini when PORTSDIR isn't valid.
Synth is a custom packge repository builder for FreeBSD and DragonFly.
It is intended to replace Portmaster, portupgrade, and poudriere for
the average user. It is simple to learn (the powerful options are
limited in number) and user-friendly, but it is extremely fast due
to its parallel building capability. It will "drop-in" on any system
as it leverages the stock pkg(8) facilities. All ports are built
in a clean environment, so it is finally safe to build ports as
needed on a live system. The default profile is the system itself, not
a new jail, which can be a valuable feature for some environments.
(more text on pkg-desc).
It's ready for wider testing now.
Fix a logic bug introduced in r401037, which caused ports without a
SiteHandler (GitHub, SourceForge, PyPI) not to be processed through the
fallback FTP or HTTP(S) cases.
I didn't pick this up in testing because only my ports were tested, most
of which use PyPI/GitHub, using the `restrict maintainer` setting in
portscout.conf.
[1] https://svnweb.freebsd.org/changeset/ports/401037
Pointyhat: koobs
Reported by: kwm (for mate, libdrm)
MFH: 2016Q1
A number of people reported no updates being detected for ports using
USE_GITHUB=yes, even after originally adding the GitHub site handler in
r401037 [1]. Investigation revealed that the FindNewestFile subroutine
and the vercompare() method assumed (or are designed so) that responses
returned from site handlers will be in a normalised version format. For
site handlers that return 'versions', this works well. For the github
handler, in the fallback use of the API for fetching repository tags, it
does not.
Additionally it turns out, portscout currently only uses/stores a
normalized version ('ver') in its database, in its general design
attempt to be a generic version comparison tool
In particular, portscout does not reference or store PORTVERSION
or DISTVERSION{FULL}, so we have nothing 'canonical' to compare
the responses from Github (tags) against.
This change special-cases Github in the FindNewestFile subroutine, which
was obtained via Portroach [2] (OpenBSD's portscout fork).
Extending this, we also now only match version-esque looking strings
from the tag, in an attempt to normalise, because they come in many
forms, including {foo-}X.Y.Z{-bar}, foo_X_Y_Z, among others.
While I'm here,
* Fix copypasta of $github_client_id, when $github_client_secret was
intended
* Add code to use authenticated requests for Github project tags in the
fallback (to /releases) case.
* Add and update some more debug messages to help diagnosis of future
issues
Special thank you's to:
* matthew, allanjude, mandree, des, Brendan Molloy for your regex, perl
help and moral support.
[1] http://svnweb.freebsd.org/changeset/ports/401037
[2] https://github.com/jasperla/portroach
Differential Revision: D4780
by simply providing the name of the module to the command. It accesses the
PyPI JSON API and does its best to generate a fully functional Makefile and
distinfo where sources are found.
WWW: https://github.com/freebsd/pytoport
PR: 205636
Submitted by: Brendan Molloy <brendan+freebsd@bbqsrc.net>
Changes:
- list of valid url scheme can be defined in pkg.conf
- add support for SOURCE_DATE_EPOCH for reproducible builds
- Fix pkg check -d
- Fix testsuite with kyua 0.12
- Fix pkg info -R
- Fix setting modes on extracted files/directories
- Fix zsh completion for pkg info -l
- Fix some solver crashed
- Initial attempt at fixing infinite loop on multirepositorry when updating pkg
- pkg version now handled -q correctly
* Attempt to properly detect gettext [1]
* Add support for @<pre|post>[un]exec plist directives [2]
* Add support for the new @sample plist directive changes [2]
* Modify the text when checking for used OPTIONS [3]
PR: 204370 [1]
204900 [2]
204575 [3]
${PERL5} points to a specific version of perl, say, perl5.22.1, it is
fine to use it in a ports Makefile to do Perly things, but ports using
it must use ${PERL}, that points to /usr/local/bin/perl so that if the
minor version is updated, the shebang keep working.
While there, make some ports use shebangfix, regen a few patches, and
bump PORTREVISION where a shebang went from PERL5 to PERL.
PR: 205367
With hat: portmgr
Sponsored by: Absolight
Changes:
- jail -d: If stdin is a TTY, confirm before removing jail.
- Disable rexec/jexecd for now as it has issues on <head and seems racy on
head as well, possibly leading to "Unable to execute id(1) in jail."
- QEMU: Use host pkg-static in the jail for repo to speed it up. For
this PKG_REPO_FROM_HOST=yes may be needed on mips.
This brings in almost a year of updates that had been held back.
Changes: https://github.com/freebsd/poudriere/wiki/release_notes_32
- New sub-command 'image' which is still in development, for generating
install images.
- Some poudriered fixes, but it is still experimental.
- DISTFILES_CACHES may be 'no' to use distfiles/ from ports tree.
- A cache daemon has been added in, off by default, for experimentation.
- The ports 'environment' feature from Mk/Scripts/functions.sh:export_ports_env
is used for dependency gathering now, which avoids 8 or so fork/execs for
every make -V call.
jail:
- New -K [KERNCONF] flag to build and install a kernel into the jail.
- New -S <path> flag to specify what srcpath to use from the master jail,
rather than /usr/src.
- Git support.
- New -U flag to specify what url to fetch from for git and svn.
- EXTRA_DISTS added to add to the distribution list fetched/extracted, from
poudriere.conf.
- NO_LIB32 added, when set to 'yes' skips installing lib32 distribution.
ports:
- Support '-m none' to disable portsnap usage.
- New -U flag to specify what url to fetch from for git and svn.
Build process handling:
- A jexecd process is spawned in the jail and communicated with via rexec
using a pipe. This is intended to lessen the jail locking contention,
allow more easily cleaning up processes, and limiting of TTY. It is
still considered experimental.
Changes:
- Fix cross building
- Improved zsh completion
- Improved error messages
- Improved documentation
- Bug fixes in the solver
- Allow to specify repositories directly via pkg.conf
- Allow multiple version of the same package in a repository
- Fix ipv6 testing in configuration
Changes:
- Fix cross building
- Wording fixes
- Always restore credentials after extraction in preparation for bigger changes
- Improve zsh completion
- Update libucl to the latest version
- Added support for OpenBSD/Bitrig
- Add a latest fixes to kill origin usage
- Improved linux support
- More user friendly error messages
- Fixes in the solver: Treat dependencies missed as errors
- Improved regression framework
- Allow to specify repositories directly un pkg.conf under the "repositories"
keyword
- All multiple version of the same package in a repository
- Fix configuration regarding ipv6
Changes:
- Fix 'builder stop' hook, which never worked.
- Don't transcend mount directories when running rm -rf
- Stop exporting MY_JOBID
- Run 'builder stop' hook always in stop_builder() [this means the hook
runs at startup as well to clean out stale builders].
- Enable post-mount jail hooks.
Unless something major is wrong here, this will be 3.1.10 and the next update
to -devel *will be* master which has a year of pending updates.
Changes:
-Always call make by absolute path of /usr/bin/make.
-Update stale comment about MFS
-Fun things happen with /nonexistent exists. Ensure it doesn't.
-testport: Allow -o to be optional.
-Mark poudriere-queue experimental.
-Using ccache on 10+ is fine so long as we use a PATH.
-XDEV: Don't endlessly append to make.conf on updates.
-Fix leftover lock file in /tmp at cleanup
-Support in-tree ccache for buildworld on recent head
-Always use the FAST_DEPEND option for buildworld.
-Go crazy, use -j for installworld targets on recent head
-Allow comments on the same line with port entries in blacklist
Over the past several months portscout.freebsd.org appears to have been
more frequently NOT finding updates, particularly for ports that use
CHEESESHOP (PyPI) as their MASTER_SITES.
Portscout has also never worked for ports using GitHub for distribution
files due to the following:
a) Portscout, prior to 'guessing', requests a randomly named file
from the Site and expects a 4xx (404) in response. If it doesn't
receive a 4xx response, it increments a 'lie counter' and does not
check the site again in the next run.
b) The GitHUB handlers (SUBDIR/MASTER_SITES) in bsd.sites.mk
construct a URL that ends in a a dummy query paramater (for the
filename), so that fetch saves the correct filename to DISTDIR.
This means for any DISTFILE name provided, a 200 OK response is
returned
These two factors unfortunately leave us in a position where there is no
good way to workaround this in the ports framework, including overriding
DISTFILES, DISTNAME, FETCH_ARGS, or the SUBDIR URL itself for various
reasons (not matching distinfo, file conflicts in DISTDIR, etc)
Fortunately, the portroach project (OpenBSD's fork of portscout)
contains a site handler for GitHub and PyPI (among others) already [1].
These site handlers use API endpoints at GitHub and PyPI that respond
JSON respectively, providing a faster and more accurate way to determine
the latest version of a package, without having to go through the
'guessing' process.
This commit:
- Adds GitHub and PyPI site handlers, and modifies or extends them to
accept/match our MASTER_SITES URL's.
- Adds authenticated API request support and two settings for the
GitHub site handler
- Add p5-JSON to RUN_DEPENDS (needed by new site handlers)
- Add HTTPS option for supporting https:// MASTER_SITES. Currently
portscout does not check (fails) https:// MASTER_SITES [2]
- Take MAINTAINER'ship
- Adds badly needed logging/debugging messages to key parts of the
process retaining the conditional logic that ties the verbosity to
"quiet" or "debug" portscout.conf settings.
- Renables the SQLITE3 option (previously commented out) and renames it
to SQLITE (the standard, as per bsd.options.desk.mk)
- Creates a DATABASE option group allowing either/or SQLITE or
POSTGRESQL to be selected
- Switches option conditionals where possible to options helpers
- Backport a fix for maintainer matching/mapping [3]
- Adds LICENSE (BSD2CLAUSE)
- Updates and sorts pkg-plist
This change was tested again ports maintained by me, and resulted in
'new versions' being found and reported for 42 of my ports (of 123).
[1] https://github.com/jasperla/portroach/tree/master/Portroach/SiteHandler
[2] Reported by: truckman
[3] 2f6ee134dd
PR: 203996
Approved by: maintainer (timeout, 15 days)
Differential Revision: https://reviews.freebsd.org/D3996
* Add support for @(...) notation [1]
* Remove the check for USES being sorted. Order is important. [2]
* Add support for making sure @owner and @group are properly reset [3]
PR: 202570 [1]
203908 [2]
202711 [3]
