A particular challenge for pkgsrc on NixOS is that it usurps all
Unix conventions and stores its system binaries and libraries in
a crazy system of hashed sub-directories:
$ which ls
/run/current-system/sw/bin/ls
$ ls -l /run/current-system/sw/bin/ls
Lrwxrwxrwx 1 root root 65 Jan 1 1970 /run/current-system/sw/bin/ls -> /nix/store/xs02fpnpkq
frhqqfsxx3lpj48wrapd00-coreutils-8.32/bin/ls
We can make a "best effort" attempt to accomodate this by invoking
the compiler to figure out where libc is. In general, it's required to
adjust the Linux files to make fewer assumptions about the layout of the
filesystem.
However, since using a compiler and libc from NixOS results in /nix/store
paths being embedded in binaries, running the NixOS "garbage collector"
can result in binaries installed from pkgsrc becoming unusable. Use with
care:
$ readelf -a ~/pkg/bin/perl | grep nix
[Requesting program interpreter: /nix/store/p5sam91qwz995pi0160rfr7dkh6pibil-glibc-2.32
-39/lib/ld-linux-aarch64.so.1]
0x000000000000001d (RUNPATH) Library runpath: [/home/nia/pkg/lib:/home/nia/pkg/li
b/perl5/5.32.0/aarch64-linux/CORE:/nix/store/p5sam91qwz995pi0160rfr7dkh6pibil-glibc-2.32-39/l
ib:/nix/store/vv9nz0bwv1pfl70w14k7dgz6yx7hjwxk-gcc-9.3.0-lib/lib]
Apparently, the "stdenv.cc" package must be installed prior to
bootstrapping pkgsrc.
I worked on this patch last year for a friend who wanted to test
something on pkgsrc but had no other system available.
For a long time, when cross-building, say from native=amd64 to
target=powerpc, it was necessary to:
1. cross-build a _powerpc_ package called cross-libtool-base-powerpc,
and then
2. install the powerpc package _natively_ with `pkg_add -m x86_64' to
override the architecture check that normally forbids this kind of
shenanigans,
in order to cross-build anything that uses libtool as a tool.
This is partly because libtool doesn't follow the normal GNU
convention of `./configure --build=<native platform> --host=<platform
package will run on> --target=<platform package is configured to
operate on>' -- in this example, build=amd64, host=amd64,
target=powerpc.
Instead, libtool expects to be cross-built itself, even if it's going
to run as a tool. It's not as bonkers as it sounds at first: libtool
is just a shell script, and it caches various information about the
(cross-building!) toolchain it is built with so it can use that
information later when it is run as a tool itself to cross-compile
other software.
To make this work, we need to create the toolchain wrappers for
libtool _as if_ we were cross-building even if we are building a
native package. So mk/tools uses a new flag TOOLS_USE_CROSS_COMPILE
instead of USE_CROSS_COMPILE, and libtool internally sets
MACHINE_ARCH=${TARGET_ARCH} (in the example above, powerpc) to make
it look like we're cross-building. The new TOOLS_CROSS_DESTDIR is an
alias for the (defaulted) CROSS_DESTDIR, which must now be set
unconditionally in mk.conf in order for libtool to know where the
cross-destdir will be; _CROSS_DESTDIR remains empty when building any
native packages (including the native cross-libtool package).
Finally, we need to make the resulting package be a native package,
with MACHINE_ARCH set to the one that it will be installed on (in the
example above, amd64), so I added an indirection _BUILD_DEFS.${var}
to replace var on its own in the build definitions that get baked
into the package, shown by `pkg_info -B'. Setting
_BUILD_DEFS.MACHINE_ARCH=${NATIVE_MACHINE_ARCH} ensures that this
mutant hybrid cross-built libtool still produces a native package.
All of this logic is gated on setting USE_CROSS_COMPILE in mk.conf or
LIBTOOL_CROSS_COMPILE in the package makefile, so it should be safe
for non-cross-builds -- when USE_CROSS_COMPILE=no and you're not
building cross-libtool, everything is as before.
Relying on native variables like MKBSDTAR only works when using the native
make, and should be avoided as they are not set when using a bootstrap.
Should fix build of lang/go117 with bootstrapped NetBSD, as bsdtar from
pkgsrc is unable to handle the distfile due to locale errors.
Add a test to check that an xbase set is installed when a tool depends on
X11 and X11_TYPE=native.
Thanks to Greg and Edgar for their comments and suggestions!
Now it should be more obvious when a package needs it as a dependency,
as it will fail loudly if it isn't declared as a tool.
While here, some duplicate dependencies on itstool were removed from the
MATE packages
Previously this was only done on Big Sur to work around the issue where XCode
does not support running these programs via a symlink, breaking .tools/bin.
However, with the update to autoconf 2.70, the native GNU m4 from 2006 on all
Darwin systems is too old and breaks the build on Catalina and older, causing
massive dependency failures.
Avoiding them both completely at this time is the simplest way forward.
These no longer support being executed via a symlink, failing with errors
such as:
xcode-select: Failed to locate 'gmake', and no install could be requested
This breaks the entire .tools/bin directory, so we just have to avoid them
and use tools from pkgsrc instead.
It's likely a lot more will need to be added to this list, but this is
enough to get devel/cmake building at least.
This case can only happen in the following special case:
TOOLS_CREATE+= asdf
TOOLS_PATH.asdf= # empty
If there is a lonely TOOLS_CREATE without a corresponding TOOLS_PATH, it
defaults to ${FALSE} and thus doesn't trigger this code.
Packages that don't declare USE_TOOLS+=perl and whose configure script
invokes perl produce a warning.
Usually warnings are ignored, but they can also be configured as errors,
for example during a strict bulk build. In this situation it is
necessary to override the default behavior of the perl tool to fail
silently. Up to now, defining both TOOLS_BROKEN+=perl and
TOOLS_FAIL+=perl produced a duplicate make target.
To handle this situation, let TOOLS_BROKEN+=perl take precedence over
TOOLS_FAIL+=perl. This is much easier than finding out in each case how
to disable the perl check in the configure script, which is most often
done by adding any of the following to CONFIGURE_ENV: PERL=#none,
ac_cv_prog_PERL=#none, ac_cv_path_PERL=#none.
This information is useful for getting the variable name that corresponds
to a tool. In most cases this is just the uppercase name of the tool,
but there are exceptions like ${SETENV} for env, ${HOSTNAME_CMD} for
hostname.
Previously a "grep" tool was created, but GREP still pointed at the platform
grep, breaking any package that used the environment variables rather than
PATH when the native platform grep does not have GNU features.
When "bmake clean depends" was called for a package where the various
cookie files already existed, these would enable different rules than a
clean package directory.
Since "bmake clean" deletes all the cookie files before "bmake depends"
starts, in these combined command lines the cookie files must be treated
as absent.
If a package had said FLEX_REQD=1000.0, the actual dependency had still
been flex>=2.5.4. Now all version numbers from FLEX_REQD are taken into
account, too.
Before, the tool arguments were written to the log as plain strings. Now
the arguments are properly quoted, which makes it possible to replay the
commands by copying them from the .work.log file.
This only affects tools that are shell builtins (echo, true, false), get
additional arguments (mkdir -p) or define a custom TOOLS_SCRIPT
(pkg-config, to set an environment variable; or autotools). Tools that
are symlinked to the real tool are not affected.
The calls to the compiler are already properly logged since cwrappers
takes care of that. This commit therefore makes the log entries for the
compilers and the other tools more similar.
When a package or the infrastructure defined a tool with custom
TOOLS_ARGS or TOOLS_SCRIPT containing special characters, these could
lead to unintuitive interactions at the time when that tool invocation
was logged in the tool wrapper log. Some of the logging output ended up
on stdout, while some of the normal output ended up in the log, and parts
of the quoted arguments were even evaluated as shell commands.
The logging of the wrapped tool commands is not perfect yet, but at least
it's much more predictable now.
The check to avoid possible dependency loop was not updated to
check devel/nbpatch (instead of devel/patch).
Fix PR pkg/53920 reported by Dean Matzkov.
Up to now, there was a central list of variable name patterns that
defined whether a variable was printed as a sorted list, as a list or as
a single value.
Now each variable group decides on its own which of the variables are
printed in which way, using the usual glob patterns. This is more
flexible since different files sometimes differ in their naming
conventions.
Two variable groups are added: license (for everything related to
LICENSE) and go (for lang/go).
