The show-all code is mostly line noise, therefore it is all the more
important to provide at least a few hints to a potential reader, by
using descriptive variable names for the iteration variables:
g => grp
c => cat
v => var
w => width
x => word
I had been confused by the printf commands since some of them used '\n'
and some used '\\\n', which seemed as if there were some quoting issue
that would make it necessary to double the backslashes.
This assumption was wrong though. The printf commands for the
single-valued variables use the normal '\n', while the lines for the
multi-valued variables end with a real backslash in the output, to
mimick the continuation lines in makefiles.
As a hint that the '\\\n' means backslash + newline, add single quotes
between the two characters.
The previous code relied on the exact implementation of Var_Parse in
bmake, and that it does not issue any error messages in case of $$ in
variable modifiers.
In variable modifiers, a $ is escaped using \$, not $$, as documented in
the manual page.
At the time when I wrote the previous version with the _SHOW_ALL.d4 and
_SHOW_ALL.d8 hacks, I did not know about the backslash escaping rule,
and I just added dollar signs until everything seemed to work. I
couldn't explain why it worked though, which is not surprising since the
code was using an undocumented implementation flaw of bmake.
The previous shell script version's runtime was quadratic against the
number of distfiles to verify. Historically this has not been an issue,
with usually only a handful of files per package. However, with the
introduction of Go modules the number of distfiles used by a single
package can be very high.
For example, in an upcoming update of www/grafana to version 7.1.5, the
number of GO_MODULE_FILES is 821. Running 'bmake checksum' takes:
real 18m20.743s
user 17m27.975s
sys 0m49.239s
With the awk code, this is reduced to a far more sensible:
real 0m4.330s
user 0m3.241s
sys 0m0.875s
The script has been written to emulate the previous version precisely,
preserving the same output and error messages and supporting all of its
behaviour, with the one exception that previous exit values of 128 have
been changed to 3, in order to avoid any potential signed 8-bit issues.
The one change in the pkgsrc infrastructure is that the mk/fetch/fetch
script no longer sets a working default value for ${CHECKSUM}. This is
not a problem in a pkgsrc environment as all of the required variables
are set correctly, but if there happen to be any users who are using
this script in a standalone environment, they will need to set it
accordingly. This was probably required in many situations previously
anyway, as none of the script's environment variables were set, and
trying to support this would be fragile at best.
At some point CRAN added the https protocol to its repositories, but
this was never reflected in MASTER_SITE_R_CRAN. Add analogues for
all the http sites with responsive https servers.
There won't (or at least should never!) be any files under share/ or man/ that
require conversion for CTF or debug support, so set sensible defaults for both
CTF_FILES_SKIP and STRIP_FILES_SKIP. Further additions are welcome.
While here rearrange the ordering of the debug skips to match CTF and deliver
a small performance improvement by avoiding unnecessary file tests.
Combined, these reduce the runtime for "make install-ctf install-strip-debug"
in lang/rust down from wall/user/sys 10m33s/2m34s/9m30s to 1m13s/0m46s/1m4s.
There was only a single valid value for HASKELL_COMPILER, therefore the
variable was useless. It only made the implementation more complicated
than necessary.
The buildlink3 variable names are quite long. So long that using the
default column width of 24 characters, most of the variable values are
not aligned. In this case, it makes sense to shift them all to the right
a bit.
Now that haskell.mk distinguishes between plain and outdated PLIST files,
this is possible again. When haskell.mk knew only missing and outdated,
this was still ambiguous and therefore skipped.
The PLIST_SUBST and PLIST_PRINT_AWK definitions for Haskell library
packages are only useful if the package-description file exists. If
that file is absent though, these are skipped.
The test whether the file exists is made as late as possible since that
file does not yet exist at the point where the package Makefile is
parsed.
This also affects the show-all-haskell target, which only shows these
values after the install phase. This is not perfect but good enough for
practical cases.
Before, running "HS_UPDATE_PLIST=yes bmake update" in wm/xmonad did not
apply the proper substitutions to the generated PLIST file since the
PLIST file was created empty during the GENERATE_PLIST command, and that
empty PLIST file changed the status to "plain" instead of "missing".
Because of that, the HS_INTF and related placeholders were not defined.
The 2 conditions for the status "missing" had to be written in separate
.if clauses because of a bug in bmake that was introduced in 2015 and
will be fixed with the next bmake update. For further details, see
src/usr.bin/make/unit-tests/cond-short.mk.
It had been switched off to not affect packages in the stable branch
2020Q2. Now starts the last round where it is possible to disable this
check. After 2020Q3, all SUBST blocks must either find their patterns or
be explicitly marked as potential no-ops. This will help to find
outdated SUBST blocks.
With our current version of pdksh, a "false && something" construct under
"set -e" conditions will continue as it does with other shells, but if the
construct is within a for loop then it exits, causing failures in the
substitution code. An explicit "|| true" is necessary to avoid this.
Approved during the freeze by wiz.
Instead:
1. Package makefiles including their own options.mk
2. Packages say "SUBST_CLASSES+=djberrno" to get the hack, if needed
3. Packages adjust SUBST_FILES.djberrno, if needed
Should fix bulk build failures due to multiple inclusions of options.mk
and/or incorrect definitions of DJB_ERRNO_HACK.
Approved during the freeze by wiz@.
The package textproc/hs-cgrep does not install a Haskell library. This
was unexpected to mk/haskell.mk, which generated an obviously wrong PLIST
file for that package, and for 3 other packages.
Noticed by wiz.
This is intended to reduce the log output on ftp.NetBSD.org when
fetching all distfiles.
Also, we call the target in basically every step of package creation
(extract, patch, configure, build, install, package) - perhaps we should trim
this down some more.
