A particular challenge for pkgsrc on NixOS is that it usurps all
Unix conventions and stores its system binaries and libraries in
a crazy system of hashed sub-directories:
$ which ls
/run/current-system/sw/bin/ls
$ ls -l /run/current-system/sw/bin/ls
Lrwxrwxrwx 1 root root 65 Jan 1 1970 /run/current-system/sw/bin/ls -> /nix/store/xs02fpnpkq
frhqqfsxx3lpj48wrapd00-coreutils-8.32/bin/ls
We can make a "best effort" attempt to accomodate this by invoking
the compiler to figure out where libc is. In general, it's required to
adjust the Linux files to make fewer assumptions about the layout of the
filesystem.
However, since using a compiler and libc from NixOS results in /nix/store
paths being embedded in binaries, running the NixOS "garbage collector"
can result in binaries installed from pkgsrc becoming unusable. Use with
care:
$ readelf -a ~/pkg/bin/perl | grep nix
[Requesting program interpreter: /nix/store/p5sam91qwz995pi0160rfr7dkh6pibil-glibc-2.32
-39/lib/ld-linux-aarch64.so.1]
0x000000000000001d (RUNPATH) Library runpath: [/home/nia/pkg/lib:/home/nia/pkg/li
b/perl5/5.32.0/aarch64-linux/CORE:/nix/store/p5sam91qwz995pi0160rfr7dkh6pibil-glibc-2.32-39/l
ib:/nix/store/vv9nz0bwv1pfl70w14k7dgz6yx7hjwxk-gcc-9.3.0-lib/lib]
Apparently, the "stdenv.cc" package must be installed prior to
bootstrapping pkgsrc.
I worked on this patch last year for a friend who wanted to test
something on pkgsrc but had no other system available.
This is a list of paths that will be excluded from SCAN_DEPENDS. Most users
will not need this, it's only really useful if you happen to use shared files
included by your mk.conf that may be touched even though there was no change
made to affect dependencies.
Looks like maybe the -Wl,-rpath-link business isn't necessary after
all -- will leave this as is until I find evidence otherwise. (joerg
says it was a workaround for NetBSD toolchain parts that weren't
properly adapted to use sysroot.)
With this, revert cwrappers version dependency to what it was before.
But keep it as TOOL_DEPENDS, not BUILD_DEPENDS.
Previously, there was no easy way to list all dependencies of a package.
The target show-depends only lists the direct missing dependencies, the
target show-depends-dirs lists all direct dependencies, but neither
lists the indirect dependencies as well. The newly added target
show-depends-all includes the indirect dependencies.
While here, document the existing targets since they differ subtly.
https://mail-index.netbsd.org/pkgsrc-users/2022/04/07/msg035444.html
There's no guarantee that POSSIBLE_GFORTRAN_VERSION is a numeric value,
so cannot be compared as such. For example on my macOS it is set to
"clang-12 (clang-1205.0.22.9)".
This really needs to be normalised correctly at some point.
For a long time, when cross-building, say from native=amd64 to
target=powerpc, it was necessary to:
1. cross-build a _powerpc_ package called cross-libtool-base-powerpc,
and then
2. install the powerpc package _natively_ with `pkg_add -m x86_64' to
override the architecture check that normally forbids this kind of
shenanigans,
in order to cross-build anything that uses libtool as a tool.
This is partly because libtool doesn't follow the normal GNU
convention of `./configure --build=<native platform> --host=<platform
package will run on> --target=<platform package is configured to
operate on>' -- in this example, build=amd64, host=amd64,
target=powerpc.
Instead, libtool expects to be cross-built itself, even if it's going
to run as a tool. It's not as bonkers as it sounds at first: libtool
is just a shell script, and it caches various information about the
(cross-building!) toolchain it is built with so it can use that
information later when it is run as a tool itself to cross-compile
other software.
To make this work, we need to create the toolchain wrappers for
libtool _as if_ we were cross-building even if we are building a
native package. So mk/tools uses a new flag TOOLS_USE_CROSS_COMPILE
instead of USE_CROSS_COMPILE, and libtool internally sets
MACHINE_ARCH=${TARGET_ARCH} (in the example above, powerpc) to make
it look like we're cross-building. The new TOOLS_CROSS_DESTDIR is an
alias for the (defaulted) CROSS_DESTDIR, which must now be set
unconditionally in mk.conf in order for libtool to know where the
cross-destdir will be; _CROSS_DESTDIR remains empty when building any
native packages (including the native cross-libtool package).
Finally, we need to make the resulting package be a native package,
with MACHINE_ARCH set to the one that it will be installed on (in the
example above, amd64), so I added an indirection _BUILD_DEFS.${var}
to replace var on its own in the build definitions that get baked
into the package, shown by `pkg_info -B'. Setting
_BUILD_DEFS.MACHINE_ARCH=${NATIVE_MACHINE_ARCH} ensures that this
mutant hybrid cross-built libtool still produces a native package.
All of this logic is gated on setting USE_CROSS_COMPILE in mk.conf or
LIBTOOL_CROSS_COMPILE in the package makefile, so it should be safe
for non-cross-builds -- when USE_CROSS_COMPILE=no and you're not
building cross-libtool, everything is as before.
- New option `sysroot=<dir>':
. Wrapper will add `--sysroot=<dir>' as first argument.
. For every rpath argument, e.g. -Wl,-R<path> to cc, wrapper will
pass `-rpath-link <dir><path>' to the linker.
This matches the old mk/wrapper/cmd-sink-cross-* logic.
- Create wrappers for the ${MACHINE_GNU_PLATFORM}-cc style of command,
as in ${CC}, ${CXX}, ${LD} for cross-builds.
- Bump version.
- Use TOOL_DEPENDS, not BUILD_DEPENDS, for cwrappers.
Ask GHC to put each compiled function in a separate section and ld(1) to
perform GC. This greatly reduces the size of executables linked with static
Haskell libraries. For example, lang/purescript shrinks down from 104 MiB
to 57 MiB! Reduced storage use means reduced I/O cost!
Prior to this change, packages that install both libraries and executables
were both linked dynamically when HASKELL_ENABLE_SHARED_LIBRARY is set to
yes. This turned out to be problematic when the executables end up
depending on hundreds of shared objects (which occurs rather frequently for
tools written in Haskell): the dynamic linker spends several seconds upon
loading the executables to resolve all the symbols.
Now we can selectively opt out from dynamic linkage by setting
HASKELL_ENABLE_DYNAMIC_EXECUTABLE to no. This should be done carefully,
because linking executables with static Haskell libraries means that those
executables will also use static RTS. This causes problems if they use GHC
API to interpret Haskell code at run time: static RTS would violate PaX
MPROTECT and suffer from ASLR while loading static objects.
* mk/haskell.mk no longer set USE_TOOLS+=pkg-config for all the packages
that include it. Instead it does TOOLS_FAIL+=pkg-config unless they
explicitly do USE_TOOLS.
* Fixed an issue regarding Template Haskell and our wrappers.
* Fixed an issue where packages that had things like PLIST.common but
lacked PLIST were incorrectly considered to have no PLISTs.
* mk/haskell.mk now supports packages that contain more than a single
library. The INSTALL and DEINSTALL scripts can now handle such packages
properly.
The original FUSE contains a patch version in fuse.pc or fuse3.pc. We must do
the same or otherwise pkg-config requirements such as 'fuse >= 2.6.0' will fail.
Add lua, nodejs, octave, qore, and tk-specific variables. (tk and tk85
define the same variable, so I only included tk.) This relates to
PR pkg/56301, where I noticed various REPLACE_* variables were not
listed in Appendix E of the pkgsrc guide.
There is a separate issue related to Appendix E here. If wip/mk is
found, the auto-generation of the help topic list for the pkgsrc guide
includes anything it finds under that path, but that's potentially
misleading, and causes the generated list to see-saw back and forth as
committers do or do not have wip visible (I do not).
Prune a bunch of variables that haven't been used for years. In some
cases, the packages they influenced don't even exist anymore. There may
be more of these, this is after a quick pass through.
Prior to this change PKGSRC_MKPIE was silently ignored when clang was chosen for
the compiler, i.e. executables were never built as PIE. This became an error
after introducing a post-build check for it.
Of course we should add a MKPIE support for clang, but for now we just emit a
warning. Otherwise we cannot build packages such as devel/gnustep-base which
requires clang to build.
This is needed by packages that require hand-holding in building PIE. Also a
post-build check for MKPIE is performed by default when PKG_DEVELOPER=YES.
resources when fetchers wouldn't be able to save what they download. On
my build farm with pkgsrc (and pkgsrc/distfiles) mounted over NFS,
however, the check gives false positives for NetBSD 9.2 and -current.
Downgrade it to a warning so that these fetches can succeed, while
leaving a breadcrumb in case someone encounters a true positive.
It would of course be interesting to sort out why, in my environment, a
wide variety of other OSes get 1 for "${TEST} ! -w $fetchdir" while
NetBSD gets 0. In the meantime, joerg@ suggested this workaround and
gdt@ agrees it's reasonable to try.
which sets ${PACKAGES} to a subdir of ${PKGSRCDIR}/packages (on the same
NFS mount) so that the generated packages all land in one place.
From my Debian, Devuan, Ubuntu, and CentOS 8 (but not 7) VMs,
copying ${STAGE_PKGFILE} to ${PKGFILE} fails to preserve permissions,
which fails the copy and the "package" target with it. The error
looks like so:
===> Building binary package for foo-4.2
=> Creating binary package /home/schmonz/trees/pkgsrc-cvs/packages/Debian/All/foo-4.2.tgz
/bin/cp: preserving permissions for ‘/home/schmonz/trees/pkgsrc-cvs/packages/Debian/All/foo-4.2.tgz’: Operation not permitted
Assuming permissions on built packages are merely a nice-to-have, avoid
failing if we couldn't preserve them. Patch from sjmulder@, for whom
this fixes a similar problem with FreeBSD with ZFS.
There are only a few releases where this appears to work, and the commit logs
show this to be a complete pain to get working. I've now confirmed it's also
broken on Big Sur where ksh simply segfaults on both x86_64 and arm64.
This may have been marginally useful in the past, but we now have cwrappers to
provide a much bigger performance improvement, and so this hack was only used
during bootstrap to build a small number of packages anyway.
OS_VERSION removes any non-numeric characters, so having a lowercase version
of it will always be identical. There's only a single package, a very old
version of gcc in wip, that references this variable so it should be safe to
just remove and avoid any potential confusion.
Something about NetBSD make(1), at least on 9_STABLE, doesn't like doing this,
resulting in literal "$$3" being passed to awk instead of being escaped down
to "$3". The same construct works fine with pkgsrc bmake(1).
There doesn't appear to be any need to evaluate it inline anyway.
This provides a single integer that can be used in arithmetic expressions
when comparing the current operating system version in make(1).
The two existing variables that are currently used for such tests are
MACHINE_PLATFORM and OS_VERSION. The former usually results in complicated
and hard to read empty() strings, and the latter is often used incorrectly in
version comparisons where developers are unaware that a string comparison is
being performed (for example ".if 5.9 >= 5.10" evaluates to true).
The default based on uname -r should be correct for most systems, and can be
overridden as it is on Darwin where the product version provides a more
useful number than the kernel version.
It's currently impossible to run system binaries inside an arm64 chroot, even
with SIP turned off, so the only way to build is to use an x86_64 chroot and
effectively cross-compile to arm64.
