For a long time, when cross-building, say from native=amd64 to
target=powerpc, it was necessary to:
1. cross-build a _powerpc_ package called cross-libtool-base-powerpc,
and then
2. install the powerpc package _natively_ with `pkg_add -m x86_64' to
override the architecture check that normally forbids this kind of
shenanigans,
in order to cross-build anything that uses libtool as a tool.
This is partly because libtool doesn't follow the normal GNU
convention of `./configure --build=<native platform> --host=<platform
package will run on> --target=<platform package is configured to
operate on>' -- in this example, build=amd64, host=amd64,
target=powerpc.
Instead, libtool expects to be cross-built itself, even if it's going
to run as a tool. It's not as bonkers as it sounds at first: libtool
is just a shell script, and it caches various information about the
(cross-building!) toolchain it is built with so it can use that
information later when it is run as a tool itself to cross-compile
other software.
To make this work, we need to create the toolchain wrappers for
libtool _as if_ we were cross-building even if we are building a
native package. So mk/tools uses a new flag TOOLS_USE_CROSS_COMPILE
instead of USE_CROSS_COMPILE, and libtool internally sets
MACHINE_ARCH=${TARGET_ARCH} (in the example above, powerpc) to make
it look like we're cross-building. The new TOOLS_CROSS_DESTDIR is an
alias for the (defaulted) CROSS_DESTDIR, which must now be set
unconditionally in mk.conf in order for libtool to know where the
cross-destdir will be; _CROSS_DESTDIR remains empty when building any
native packages (including the native cross-libtool package).
Finally, we need to make the resulting package be a native package,
with MACHINE_ARCH set to the one that it will be installed on (in the
example above, amd64), so I added an indirection _BUILD_DEFS.${var}
to replace var on its own in the build definitions that get baked
into the package, shown by `pkg_info -B'. Setting
_BUILD_DEFS.MACHINE_ARCH=${NATIVE_MACHINE_ARCH} ensures that this
mutant hybrid cross-built libtool still produces a native package.
All of this logic is gated on setting USE_CROSS_COMPILE in mk.conf or
LIBTOOL_CROSS_COMPILE in the package makefile, so it should be safe
for non-cross-builds -- when USE_CROSS_COMPILE=no and you're not
building cross-libtool, everything is as before.
which sets ${PACKAGES} to a subdir of ${PKGSRCDIR}/packages (on the same
NFS mount) so that the generated packages all land in one place.
From my Debian, Devuan, Ubuntu, and CentOS 8 (but not 7) VMs,
copying ${STAGE_PKGFILE} to ${PKGFILE} fails to preserve permissions,
which fails the copy and the "package" target with it. The error
looks like so:
===> Building binary package for foo-4.2
=> Creating binary package /home/schmonz/trees/pkgsrc-cvs/packages/Debian/All/foo-4.2.tgz
/bin/cp: preserving permissions for ‘/home/schmonz/trees/pkgsrc-cvs/packages/Debian/All/foo-4.2.tgz’: Operation not permitted
Assuming permissions on built packages are merely a nice-to-have, avoid
failing if we couldn't preserve them. Patch from sjmulder@, for whom
this fixes a similar problem with FreeBSD with ZFS.
As part of the pkgdb migration (NetBSD only), PGKTOOLS_REQD was set to
a recent value (20200828). However, that results in a cyclic
dependency of pkg_install on cwrappers on pkg_install. Once people
set PKG_DBDIR in pkg_install.conf and mk.conf to match their setup,
there is no need -- because of the migration -- to force newer tools.
Testing on TNF's pkgbuild machine indicates this revert works well,
and I received two positive comments and none against.
breakage for users who have other package managers that use /var/db/pkg
(Reported by Frederic Cambus on FreeBSD, OpenBSD)
Adjust warning: specifying PKGDB_DIR in mk.conf should be sufficient
for building packages for the compatibility pkgdb location.
This is still an error message, because:
- While we can handle the existing references of PKGDB_DIR, new ones
might be created.
- It is a minor inconvenience to people building packages from source.
The new release of macOS removes system libraries from the file system, only
providing access to them via a linker cache and dlopen(). This obviously
breaks many assumptions about how libraries work on Unix systems, and so we
unfortunately need to cripple various checks when running on those systems.
Introduce DARWIN_NO_SYSTEM_LIBS which, when defined, will trigger alternate
behaviour in the infrastructure. Currently this is in two places:
* In CHECK_SHLIBS, skip any path beginning with /usr/lib.
* In registered package metadata, any path beginning with /usr/lib is
removed from REQUIRES.
The former fixes all package builds, while the second will be necessary for
package managers such as pkgin, as they will no longer be able to verify that
those files are available on the target system.
This is obviously a gross hack, and removes our ability to ensure that the
target system is suitable for the packages we are attempting to install, but
Apple have left us with no alternative, and users will unfortunately be left
to find out at runtime instead.
It's likely this will need to be extended to /System/Library paths too, but
this is required first to actually get packages building before we can start
running bulk builds.
There is no evidence that any shell needs the empty string literal in
'for i in "" $var' to loop over a simple variable. This would only be
necessary if the shell code were generated by a preprocessor such as
bmake or autoconf, and the list of items might end up empty. In such a
case, the empty string literal prevents to generate 'for i in ; do',
which would produce a syntax error.
When RUN includes the -u shell option (see the pkgsrc guide, section
bulk.var.shvar), there is a stray "requires: parameter not set" error.
This error doesn't make the build fail immediately since it is on the
left-hand side of a pipe, where the exit status is ignored. It would
fail if the shell for bmake were set to bash and its option "pipefail"
were enabled.
Right now, users who install the pkg-vulnerabilities database find that
the vast majority of packages fail to build, penalizing them too severely.
Package auditing can still be done via "pkg_admin audit".
Alternatively, the previous behaviour can be restored with
ALLOW_VULNERABLE_PACKAGES=no in mk.conf.
Additionally, bmake-ify the check.mk logic. It was easier to do this,
as the package relied on a single long ${RUN} command.
Proposed on tech-pkg, with no objections to the idea of changing the
default, just the method of doing so.
Before, relative paths had been stored as-is. This affected those
packages that defined PATCHDIR or FILESDIR as relative directory instead
of prefixing it with ${.CURDIR}.
Since there are already several other paths that are interpreted relative
to the package directory (CONFLICTS, DEPENDS), allow PATCHDIR and
FILESDIR to be specified as relative paths, too. This makes the package
Makefiles a bit shorter.
Before, not only files containing an RCS Id were recorded in the
+BUILD_VERSION file but also files containing text that looked similar to
an RCS Id were recorded, even though these didn't contain any valuable
version information.
The effect was that before this change, pkgtools/pkglint was built over
and over again by the bulk builds since pbulk uses a different regular
expression for detecting modified files.
The regular expression for unexpanded RCS Ids is added to record files
that have never been checked in to CVS, just to have them recorded and to
distinguish them from the final committed version.
See https://mail-index.netbsd.org/tech-pkg/2020/01/11/msg022489.html.
Before, not only files containing an RCS Id were recorded in the
+BUILD_VERSION file but also files containing text that looked similar to
an RCS Id were recorded, even though these didn't contain any valuable
version information.
The effect was that before this change, pkgtools/pkglint was built over
and over again by the bulk builds since pbulk uses a different regular
expression for detecting modified files.
The regular expression for unexpanded RCS Ids is added to record files
that have never been checked in to CVS, just to have them recorded and to
distinguish them from the final committed version.
See https://mail-index.netbsd.org/tech-pkg/2020/01/11/msg022489.html.
digests to identify file versions. Defaults to no for now. For digest
mode, recursive into FILESDIR, so that e.g. the majority of pkg_install
itself is recorded as well SMF manifests.
There are a couple of places in pkgsrc where rcsid's are generated, and
these were leaking into "pkg_info -b" output. This had the side effect of
causing those packages and all their dependencies to be rebuilt for every
bulk build as the output and the respective files always mis-matched.
Previously packages configured these manually based on INIT_SYSTEM, this
centralises support for them so they only need to exist to be pulled in
accordingly.
Enhance the reduce-depends.awk script to reduce a larger set of
dependencies into a single dependency. The patterns representing
intervals of version numbers (can be open-ended) are of the form:
pkg>lower
pkg>=lower
pkg<upper
pkg<=upper
pkg>lower<upper
pkg>lower<=upper
pkg>=lower<upper
pkg>=lower<=upper
These patterns are now condensed into a single dependency of the
same form. For example, given the following patterns:
pkg>=1.0
pkg>2.0
pkg<3.0
pkg<=4.0
pkg>=2.5<3.5
the reduced pattern becomes:
pkg>=2.5<3.0
Add the test script used to help with refactoring and adding the
new feature to the script.
This is a mostly complete rewrite of the script; change the
license to the standard 2-clause BSD license used by TNF.
Only pass "-C ${CONFLICTS}" to pkg_create(1) if ${CONFLICTS} is
non-empty; otherwise, a fatal error occurs. This allows CONFLICTS
to be defined, but empty.
The pkginstall framework had an UNPACK action invoked early
in the PRE-INSTALL stage to unpack scriplets that were invoked
in later stages to perform extra work for installation, removal,
or both.
Add an UNPACK action that is a no-op for any scripts that don't
support the action.
The return values of the postinstall and postremove meta-tasks are
ignored unless the user-settable variable ${PKG_FATAL_ERRORS} is
"yes". ${PKG_FATAL_ERRORS} defaults to "no" to preserve the
existing behavior of install scripts by mk/pkginstall.
Add definitions for the gtk-update-icon-cache(1) command to the
script header template, and a makefile fragment that creates the
data file for the "icon_themes" package task if ${ICON_THEMES} is
"yes" in the package Makefile.
Also bump pkgtasks dependency to version 1.12 for the "icon_themes"
package task and compatibility with pkgsrc/mk/pkginstall.
