Before, relative paths had been stored as-is. This affected those
packages that defined PATCHDIR or FILESDIR as relative directory instead
of prefixing it with ${.CURDIR}.
Since there are already several other paths that are interpreted relative
to the package directory (CONFLICTS, DEPENDS), allow PATCHDIR and
FILESDIR to be specified as relative paths, too. This makes the package
Makefiles a bit shorter.
The previous implementation could not reliably detect outdated configure
options. This was apparent in devel/gettext-tools, where the option
--with-included-libcroco had become unknown between May 2019 and May
2020, but the check was not run.
The behavior is the same in the pkgsrc default configuration. Only if
GNU_CONFIGURE_STRICT=yes, the new check is activated and will make
packages fail that previously succeeded to build. Since that variable is
not widely known, there won't be much sudden breakage, if any.
Pointed out by wiz@.
This occured in math/libixion/Makefile.common until 2020-05-19, and still
occurs in math/xyconvert/Makefile. In all other packages, PKGDIR is
prefixed with ${.CURDIR} and is thus an absolute path.
It should not be necessary to always specify PATCHDIR as an absolute
path, and the code in mk/pkgformat/pkg/metadata.mk seems to be the only
place where relative paths are handled wrong.
This makes the SUBST blocks stricter than before, to detect outdated or
unnecessary definitions.
Filename patterns that are not affected by any of the SUBST_SED
expressions make the build fail. It is still ok if only some of the
files from a pattern are affected and some aren't.
The latest bulk build shows that most of the build failures are fixed.
The packages that fail in that build are mostly due to other failures,
like missing C headers, wrong PLIST files, unresolved references at link
time. There may be a few packages that still fail because of this, but
these are near the leaves of the dependency tree.
https://mail-index.netbsd.org/pkgsrc-bulk/2020/05/14/msg018919.html
The test can now be run with bash and ksh as well.
Bash exits if a function call returns failure, the NetBSD /bin/sh
doesn't. After reading POSIX, sections "2.9.1 Simple Commands" and "set
-e", I'm in favor of bash here.
Ksh exits in "set -eu" mode if "$@" is used but no arguments are given.
In a basic regular expression, a dollar-sign only means end-of-string if
it appears at the end of the pattern, or (at the choice of the
implementation) at the end of a \(...\) subexpression.
This affects the package converters/help2man that uses a regular
expression containing a dollar in a non-final position. This regular
expression had not been detected as an identity substitution even though
it is one.
This isolates the tests from the test infrastructure and allows the test
infrastructure to create arbitrary files for its own purpose without
affecting any of the tests.
Files like Makefile.am and configure.ac are usually not used during a
build, therefore there's no point in checking these for shell portability
issues.
A commonly occuring scenario is that a package patches the configure
script, but that the corresponding configure.in contains shell code that
is not portable. In cases like these, configure.in is typically not used
during the build, therefore there is no need to check it for portability.
This also applies to all other combinations where a file is patched and
the corresponding file.in contains unportable shell code.
Having quotes around the sed commands does not change their meaning.
These quotes must not lead to syntax errors when parsing the shell
command. This happened in mk/subst.mk r1.91 because the double quote was
accidentally escaped.
The escaping inside the backticks had been wrong. Because of this,
parentheses and semicolons were interpreted as shell syntax.
Switching to $(...) command substitution removes the need for quoting
some of the characters and makes the whole command simpler to understand.
Doing the escaping for the backticks command properly would have involved
lots of special cases.
The $(...) command substitution was used sparingly in pkgsrc up to now
because some older or broken shells do not support it. Since these
shells do not end up as the shell that runs the commands from Makefiles,
that's not a problem.
ShellCheck complained about a parse error in the empty functions, which
agrees with the shell grammar given in IEEE 1003.1, both the 2004 Edition
and the 2018 Edition.
To work properly, the $(...) should have been $$(...).
In pkgsrc the command substitution is usually done via `backticks`, for
compatibility with /bin/sh from Solaris. To fix the shell parse errors,
the special characters are properly escaped inside the command
substitution.
Since SUBST_FILTER_CMD is a shell command, it may contain arbitrary
characters. The condition in mk/subst.mk that tested whether
SUBST_FILTER_CMD was the default filter command was evaluated at run
time. In such an evaluation, the variables (lhs and rhs) are fully
expanded before parsing the condition. This means that these variables
must not contain quotes or unquoted condition operators.
Exactly this situation came up in one of the regression tests. The
quoted "0-9" was copied verbatimly into the condition, including the
quotes. The resulting condition was:
"tr -d "0-9"" == "LC_ALL=C /usr/bin/sed "
This produced a syntax error because of the left-hand side. Adding a :Q
modifier would have helped for the left-hand side, but this would have
been necessary for the right-hand side as well. Since an empty SUBST_SED
is defined not to "contain only identity substitutions", the first
condition can simply be removed.
The whole condition in the shell program had not worked anyway since it
expanded to either "[ true ]" or to "[ false ]", and both of these
commands exited successfully.
There are several cases where patterns like s|man|${PKGMANDIR}| appear in
SUBST_SED. Up to now, these had been categorized as no-ops and required
extra code to make the package build when SUBST_NOOP_OK was set to "no".
This was against the original intention of SUBST_NOOP_OK, which was to
find outdated substitution patterns that do not occur in SUBST_FILES
anymore, most often because the packages have been updated since.
The identity substitutions do appear in the files, they just don't change
them. Typical cases are for PKGMANDIR, DEVOSSAUDIO, PREFIX, and these
variables may well be different in another pkgsrc setup. These patterns
are therefore excluded from the SUBST_NOOP_OK check.
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html
says in section 9.3.2 BRE Ordinary Characters that only very few
characters may be preceded with a backslash.
As a side effect, this change allows parentheses in the variable names
listed in SUBST_VARS (even if that will never happen in practice).
The reason that the regression test had not replaced VAR.[] before was
simply that this variable had not been listed in SUBST_VARS.
The documentation now starts with a high-level introduction instead of
listing only the details.
The name of the function test_file had to be changed since the old name
was not expressive enough. Same for the variable real_pkgsrcdir.
