Some packages using mk/djbware.mk do not have error.h but need to fix the
errno declaration in other files. Up to now, there was no good way of
achieving exactly this.
Building packages like these in a SUBST_NOOP_OK=no build makes these
packages fail. To fix these packages, the configuration of the files
needed to be a bit more flexible.
The escaping inside the backticks had been wrong. Because of this,
parentheses and semicolons were interpreted as shell syntax.
Switching to $(...) command substitution removes the need for quoting
some of the characters and makes the whole command simpler to understand.
Doing the escaping for the backticks command properly would have involved
lots of special cases.
The $(...) command substitution was used sparingly in pkgsrc up to now
because some older or broken shells do not support it. Since these
shells do not end up as the shell that runs the commands from Makefiles,
that's not a problem.
To work properly, the $(...) should have been $$(...).
In pkgsrc the command substitution is usually done via `backticks`, for
compatibility with /bin/sh from Solaris. To fix the shell parse errors,
the special characters are properly escaped inside the command
substitution.
Since SUBST_FILTER_CMD is a shell command, it may contain arbitrary
characters. The condition in mk/subst.mk that tested whether
SUBST_FILTER_CMD was the default filter command was evaluated at run
time. In such an evaluation, the variables (lhs and rhs) are fully
expanded before parsing the condition. This means that these variables
must not contain quotes or unquoted condition operators.
Exactly this situation came up in one of the regression tests. The
quoted "0-9" was copied verbatimly into the condition, including the
quotes. The resulting condition was:
"tr -d "0-9"" == "LC_ALL=C /usr/bin/sed "
This produced a syntax error because of the left-hand side. Adding a :Q
modifier would have helped for the left-hand side, but this would have
been necessary for the right-hand side as well. Since an empty SUBST_SED
is defined not to "contain only identity substitutions", the first
condition can simply be removed.
The whole condition in the shell program had not worked anyway since it
expanded to either "[ true ]" or to "[ false ]", and both of these
commands exited successfully.
There are several cases where patterns like s|man|${PKGMANDIR}| appear in
SUBST_SED. Up to now, these had been categorized as no-ops and required
extra code to make the package build when SUBST_NOOP_OK was set to "no".
This was against the original intention of SUBST_NOOP_OK, which was to
find outdated substitution patterns that do not occur in SUBST_FILES
anymore, most often because the packages have been updated since.
The identity substitutions do appear in the files, they just don't change
them. Typical cases are for PKGMANDIR, DEVOSSAUDIO, PREFIX, and these
variables may well be different in another pkgsrc setup. These patterns
are therefore excluded from the SUBST_NOOP_OK check.
Before, adding "Binary file matches" (including the quotes) to
CHECK_WRKREF_EXTRA_DIRS led to a syntax error. Adding this string is so
obvious that it should have been added a long time ago already.
The indentation of the inner loop has been fixed.
The chmod is only run if the file has actually changed. In the other
case, the file would have been removed right after the chmod, which made
the chmod unnecessary.
For compatibility with ancient operating systems whose /bin/sh still does
not understand negated conditions (SunOS), these conditions have been
avoided and were written using && and || instead.
The inner loop has been flattened a bit, to compensate for the
indentation of the outer loop.
This avoids creating a temporary directory.
The "set -f" option is not used anywhere else in pkgsrc, even though it
has been available since 1985 in the 8th Edition of Research Unix. Even
AIX and IRIX have that option, so it seems a safe bet.
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html
says in section 9.3.2 BRE Ordinary Characters that only very few
characters may be preceded with a backslash.
As a side effect, this change allows parentheses in the variable names
listed in SUBST_VARS (even if that will never happen in practice).
The reason that the regression test had not replaced VAR.[] before was
simply that this variable had not been listed in SUBST_VARS.
if quoted. Seen on NetBSD 7.
#!/bin/sh
in="/path/to/dir with space/file"
: "${file=${in##*/}}"
: "${dir=${in%/*}}"
echo "dir:$dir"
echo "file:$file"
[ "$dir" = "$file" ] && echo "dir and file are same"
Leads to errors when adding packages such as:
./+FILES: cannot create /var/db/pkg.refcount/files/etc/rc.d/xenguest//var/db/pkg/xe-guest-utilities-7.0.0: directory nonexistent
This variable allows to make SUBST stricter than before. This will break
several packages that have redundant filename patterns. Most of these are
typos or outdated and should be updated or removed.
While here, add more verbose documentation on PLIST_SUBST since the way
from the package's PLIST file to the +PLIST file can easily be confused
with the other way round, which is handled by print-PLIST.
The .for loops are expanded by bmake as soon as they are parsed. Since
there are many variable groups, and since each of these groups has
several variables, this takes quite a bit of time.
The time for running "bmake clean" in pkgtools/pkglint went down from 1.0
second to 0.7 seconds. This may not seem much, but when multiplied with
23088 times 9, this little change may speed up a full bulk build by 62000
seconds, which is about 18 hours.
The side-effect is that the show-all-* targets can only be run from the
command line, not as a dependency of other targets. This restriction will
probably go unnoticed.
Used to form _HASKELL_PKG_DESCR_DIR. Defaults to DISTNAME, but for
some packages (hledger, incoming), the same DISTNAME is shared by
multiple packages. So let the package define it explicitly if need
be.
Before, searching for topic=socket did not find the documentation.
The detection of useful help topics is still not perfect since it now
finds sections that consist of a single word, such as the word
"undo-replace" in mk/install/replace.mk, but that will be fixed later,
after adding a few unit tests.
The code maps gcc-4.8.x to "4", and then won't find gcc4. This is a
bug, but it may be that it's just as well to never match 4.X of any
kind, and use 7 anyway. Explain this issue with a \todo to fix the
bug or document the consequences as intended.
(This is a comment-only change.)
Adjust regexp that removes .Y.Z from gcc-X.Y.Z.
Test for gcc being contained in PKGSRC_COMPILER, vs ==, so that a
value of "ccache gcc" is handled properly.
(ok for mk during freeze jperkin@)
Currently virtualsize is only defined for NetBSD, breaking builds that use it
on any other platform. Adding defaults for all other platforms should be done
at some point, this at least unbreaks package builds for now and provides a
safer default for any future additions.
There are several places in pkgsrc where the files to be patched are
listed individually instead of just saying util/*/*.sh. This is
unnecessarily detailed. Encourage package authors to use filename
patterns more often.
An example is REPLACE_PYTHON in lang/clang, which currently fails because
some of the listed files don't exist anymore.
These appear to have been cargo culted around for a while, don't even have
anything to do with DESTDIR mode, and are completely useless. Simplify the
logic a little while here. Noticed by rillig@.
When fixing the SUBST definitions in a package, it can hapen that the
substitution aborts in the middle. In such a case the cookie should not
be written and the substitution should be retried. Otherwise the build
may continue with half the substitutions done.
The severity now depends only on the setting of SUBST_NOOP_OK. Right now
this means that some former warnings will be reported as info only, but
that will change after switching the default of SUBST_NOOP_OK after
2020Q1. Then they will all be reported as warnings, followed by the final
error saying that the pattern has no effect.
This change makes it easier to detect inconsistencies and outdated
definitions, for example by setting the global SUBST_NOOP_OK=no and
redefining WARNING_MSG to actuall fail.
The diff program is only used to produce informative output in the build
logs, nevertheless its output might be translated if there are lines that
do not end with a newline.
This hack is mostly needed for older software like sysutils/daemontools
that was created when errno was still a global variable. Newer packages
like devel/bglibs don't need that hack anymore. Therefore make it
configurable, to avoid build failures when subst.mk does not find any
error.h to patch.
The variables are now sorted chronologically instead of alphabetically,
to express the order in which the actions happen. The sentences are a bit
shorter than before.
The visual layout has been changed to make the whole variable name stick
out more. Before, the part that was easily visible was the same "EXTRACT_"
for all of them.
Before, variables containing dollar characters displayed so wrong that it
was hard to explain.
To fix the problem, I typed almost random characters into the code until
the output was exactly as expected. I still do not understand:
* why the list variables need 8 dollars to survive the @x@ loop,
* why the code only works if the dollars come from an external variable
instead of being written inline,
* why the backslash in the :C modifier needs to be doubled.
Anyway, the output of "bmake show-all-extract" now contains the shell
variable $${extract_file}, just as it should. The dollars are now doubled
in the output and thereby match the source code from the Makefile
exactly.
This prevents unintended shell word expansion.
It also makes that argument visible in "bmake show-all-extract". Before,
it was left out entirely from the output. Now it is printed as an empty
string literal, which at least gives a hint that "there is something".
The default value of SUBST_MESSAGE is based on SUBST_FILES, and that
variable may use the :sh modifier to list files from WRKSRC, which may
not exist at load time.
In the case of pkglocaledir, the SUBST_FILES are generated by a shell
command. That command assumes that the WRKDIR already exists. Therefore
SUBST_FILES must be evaluated as late as possible.
See mk/configure/replace-localedir.mk; an example package that fails is
devel/gettext-tools.
In a bulk build with very strict settings (WARNING_MSG fails, as well as
no-op substitutions), it became clear that nearly all of the cases where
SUBST didn't replace anything were bugs in the package definition.
Most of them were just outdated, which is no surprise given that some
packages are already over 20 years old.
For backwards compatibility, SUBST_NOOP_OK defaults to "yes" right now.
After correcting the affected packages, the default will change to "no".
This file allowed one to in theory specify a wxGTK version, but only
two programs used it. Both now directly include a preferred version.
Generally, packages should use wxGTK30, unless there is a good reason
to use 28.
A major motivation earlier was not being able to install 28 and 30 at
once, but this has been solved.
As briefly discussed on pkgsrc-users.
In an attempt to resolve problems on SunOS, move EXTRACT_USING to the
early variables file, in the hopes that it will be defined before code
that checks it is executed.
