- Added a check for .include lines in all Makefiles and *.mk files to make
sure that only existant files are referenced and that pkgsrc packages do
not contain references to packages in pkgsrc-wip.
*.mk file is checked.
- Added warnings for unusual make targets. Everything except the usual
{pre,do,post}-* targets is considered unusual. Exceptions may be declared
in the Makefile using ".PHONY".
- The directives are checked to contain arguments if and only if needed.
- The .ifndef and .ifdef directives are marked as deprecated because
the parsing algorithm of NetBSD's make is so bad that it cannot
distinguish ".if" from ".ifdef".
- Added notes whenever ".undef" is used with a variable that had been used
in a ".for" loop before. Undefining the variable is simply unnecessary.
Changes since 4.46:
- Made the --explain command line option work.
- Added many explanations for existing diagnostics.
- Improved the diagnostics.
- In --autofix mode, no backup files are created anymore.
- Temporarily disabled the check for direct use of tool names.
- Fixed some false positive warnings.
- Added a check for the use of absolute pathnames in shell commands.
check had been removed some time ago due to the huge number of false
positives. Now that pkglint can parse shell commands quite well, it
has been reintroduced, as absolute pathnames often indicate unportable
features of a package. To implement this check (and a few others)
accurately, the whole code for checking shell commands has been
rewritten as a finite state machine.
warnings that LIST+=FOO=${FOO} should rather be LIST+=FOO=${FOO:Q}.
The cause was that I had added a capturing group in a regular
expression that also contained a back reference (\2). Adjusted the
back reference to \3. (This is the only place in pkglint where such
back references are used, so I don't have much experience with them.)
- Added the $line->replace() method for a convenient way to achieve simple
autofix tasks.
- When autofixing, no backup file is created (it had been created before).
1. Usually "cvs diff" is available.
2. All current autofixed changes are trivial.
3. The fixed file is first written to disk in a new file and then renamed
to the original file, greatly reducing the risk of data loss.
- Fixed a bug in checkdir_package(): In the call to load_package_Makefile(),
the last parameter had not been passed by reference, but by value.
Luckily this had not influenced any other part of pkglint.
- While there, I noticed that it is not necessary to pass some subroutines
the lines of all included Makefiles, so I removed that parameter.
- Removed the unused variable $opt_explain. It should really have been
PkgLint::Logging::set_explain().
- In accordance to my personal idea of beauty, the --explain messages are
indented as deep as the last diagnostics, and not by a single "\t".
files. New TODO items:
* extract the DISTFILES if they exist and check all files in them for:
- security holes,
- coding style violations,
- possible unportable constructs
Changes since 4.45:
- Added the type ShellWord.
- Added quoting checks for variables that are appended to a
List of ShellWord.
- Multiple -v increase the verbosity.
- The --autofix option is no longer undocumented, but still experimental,
as this is the only part of pkglint that can modify external files.
- Added an --explain option that provides additional help for the
diagnostics.
- Added checks for unportable CPP macro names in patches.
- Updated the documentation.
:M*:Q instead of a simple :Q. Currently only the GNU configure
scripts need the white-space stripped, so it's enough to quote
those (see regex_gnu_configure_volatile_vars in the source).
- Added detection of redirection operators and comments to the
regex_shellword constant.
- Changed the naming conventions for list data types. Now List simply
means a list. List+ means a list that should only be modified using
"+=", not "=". List! means an internal list. And List!+ is the
combination of both.
The distinction is necessary because of the introduction of
redirection operators in the regex_shellword. When checking the data
type of list elements, the lists are split up into shell words instead
of simply using split(). This leads to much better results.
- When splitting a variable value or shell command into words, anchor the
regular expression at the beginning of the string.
- Removed log_subinfo().
- Added log_debug().
- Multiple -v command line options increase the verbosity.
- Added the regex_shellword constant that will allow better parsing of
shell commands. Currently it is only producing debugging information.
- Long [info] messages have been changed to [debug] messages.
CFLAGS=${CFLAGS}, check for the correct modifiers. The above is
obviously not correct, as CFLAGS may contain white-space.
CFLAGS=${CFLAGS:Q} is also a little wrong in that it may contain leading
and/or trailing white-space, which must be discarded, too, because the
broken GNU configure scripts cannot handle them correctly. This can be
done using ${CFLAGS:M*:Q}, which first splits CFLAGS into a list of
shell words, then selects all of them and finally combines the words
forming a nicely formatted string without leading and trailing
white-space where all entries are separated from each other by a single
space.
- Added a new type ShellWord that is used for MAKE_ENV and the like to
check for invalid FOO="${VALUE}" additions. They really should be
FOO=${VALUE:Q}, as they may already contain embedded quotes. This is
especially important for CPPFLAGS and CFLAGS.
problems. Instead, use exec*() functions.
Replace PUSHOUT() macro and string buffers with a function that
operates on data structures.
If it is necessary to copy files into place from staging area, then use
pax to copy them.
Add functions in pexec.c to create a pipe for sending data to a child
process. Replace pipe code in create/perform.c with these functions.
Use these functions instead of command-line arguments when copying files
into place from staging area.
Three system() references remain: @exec, @unexec, and vsystem().
- Fixed perl -T warnings:
- Replaced $#{@{$lines}} with $#{$lines}.
- Avoided calls to external programs (sed and digest).
- Removed redundant warning if DISTNAME is set to an invalid package name
and PKGNAME is not defined.
- Changed dependency from pkgtools/digest to security/p5-Digest-SHA1.
Changes since 4.43:
- Removed the remaining code that had been imported from FreeBSD's
portlint.
- Removed the (undocumented) -Wvague option.
- Removed the -Wexec and -Wparen options. The latter had no effect at all,
and the former cannot lead to false positives, so it is always enabled.
- Variable names starting with an underscore are reserved for internal
pkgsrc use.
- Added some more type checks.
- Renamed the type Dependency to DependencyWithPath.
- Added new types Dependency, PlatformTuple and RelativePkgDir.
- Added some of the common variables to the list of typed variables.
Changes since 4.42:
- Reduced the number of duplicate diagnostics when checking multiple files.
Only diagnostics concerning the current package are printed.
- Added checks for EXTRACT_SUFX and PKG_INSTALLATION_TYPES.
- Added a work-around for the PHP patches warnings.
- PERL5_PACKLIST should not contain references to other variables.
- Added the -s|--source command line option to show the code along with the
diagnostics.
- Fixed a bug in get_logical_line(); logical lines have not had their
physical lines attached.
- Deprecated variables are not only checked when they are defined but also
when they are used.
- Added a check that PERL5_PACKLIST does not contain references to other
variables. Some packages have ${PERL5_SITEARCH} in it, which results in
a double slash, and the CHECK_FILES framework cannot handle this.
anonymous empty string. the macro uses the argument multiple times
and the logic relies on identical strings having the same address,
which is compiler dependent and not guaranteed to be the case.
problem observed with pax(1) built with sunpro 11 on Solaris.
from segv@netctl.net in PR pkg/32097.
When installing libkver, as required when NETBSD_RELEASE is set
in a pkg_comp configuration, do so in a special purpose prefix and use
it from there.
This is useful when running a bulk build in the chroot or simply
removing all installed packages in it: the libkver package won't
be removed hence the chroot will still be usable.
Approved by MAINTAINER.
in the $all_lines parameter, but the whole text of the included files in
the $whole parameter.
This change avoids duplicate diagnostics for *.mk and Makefile* in the
package directory. A side effect is that package authors only get the
diagnostics they can probably fix, as diagnostics from included files are
not given.
Changes since 4.41:
- In .mk files, line continuations are properly handled.
- An open bracket is allowed in variable and tool names.
- Added checks for ALTERNATIVES and INSTALL/DEINSTALL files.
- By default, don't check files in ${FILESDIR}.
- Warn about unknown file and directory names.
- Documented the --recursive option.
- Added an option -Cmk for checking .mk files besides buildlink3.mk.
- pkglint can handle individual files on the command line, not only
packages. (experimental)
- Replaced checkfile_buildlink3_mk() with checkfile_mk().
- Factored out the checkfile() subroutine from checkdir_package().
since the pkglint Makefile sets MANINSTALL but shouldn't, as this
variable is intended to be user-defined. This is only a work-around,
which needs to be addressed properly for the other packages setting
MANINSTALL, too.
- Removed all references to the build-time pkgsrc directory, ${PKGSRCDIR}.
This makes the binary package independent of the build location.
Fixes PR 32006.
Changes since 4.38:
- Disabled the check on Makefile variable order, as the discussion on
tech-pkg has not finished yet.
- Improved detection of valid tool names.
- Improved detection of direct use of tool names. (Less false positives.)
- Improved the diagnostics for direct use of tool names.
pkgsrc/mk/defaults/mk.conf. All those variable names are then checked as
being of type Userdefined. As that file is currently too unstructured
and contains too much garbage, this check cannot yet be enabled.
- pkglint(1) does not use the hard-coded PKGSRCDIR anymore.
This fix is related to PR 32006, but doesn't fix it, since the PR is
about lintpkgsrc(1), not pkglint(1).
- It is an error if packages define a variable whose name starts with an
underscore.
- Removed the (vague) "wip" check, as it has been replaced by special checks for
DEPENDS and .include directives.
- Disabled the check for variable ordering until the discussion on tech-pkg
has led to any results.
- PKGNAME is checked for being a valid package name.
- PLIST files should not contain filenames that match */CVS/*.
- Completely rewrote the check for variable ordering. The new code
operates on a data structure that's easily understandable and
extendable (see the source for an example). It also generates greatly
improved diagnostics. As the old code had been enabled only when
checking with -Wvague or -Wall, it has been seldom used anyway.
Also, simplify the package by making use of the PKGMANDIR-handling in
pkgsrc to remove the custom MANDIR code from the package Makefile and
the PLIST.
Technically, this type of a change would require a PKGREVISION bump,
but as this is the pkg_* tools package and there were no changes to the
binaries, we refrain from doing so.
- Removed the hard-coded values for valid tool names.
The detection is good enough.
- Make sure that the domain part NetBSD.org is written like this
in mail addresses.
- Added checks for TOOL_* variables.
- Added {pre,do,post}-extract to the list of valid stages.
- Fixed the regular expression for detecting tool names.
- Added a check for invalid syntax in tool names.
- Improved the diagnostic for enumerations.
- Added many of the variables found in pkgsrc/mk/* to makevars.map.
- Added the function log_fatal, whose output goes to stderr instead of
stdout.
- All files in pkgsrc/mk/ are excluded from checking, as they may use
private variables and do other questionable things.
- Removed the Language type, as it is a simple enumeration.
- Added the Userdefined type to distinguish user-definable and read-only
variables.
Implemented checking for enumeration types. Added new types Filemask,
Filename, Language, Option, Pathmask, Pathname, Stage, Varname and
WrksrcSubdirectory. Made the check for URLs stricter. Added some
variables to the makevars.map file.
Removed the -Wworkdir option. Added an --import option that replaces
-Wworkdir. Changed the warning about uncleaned work* directories into an
error. Updated the man page.
Fixed the --autofix handling of category Makefiles. Added an --import
option that helps importing packages from pkgsrc-wip. When checked with
--import, the package is checked as if it weren't part of pkgsrc-wip.
Fixed undefined behavior when reading a file that ends in a continuation
line. Any use of ${WRKSRC}/.. is considered an error, as ${WRKSRC} is
meant to point to the top of the build directories. A proper combination
of WRKSRC, CONFIGURE_DIRS and BUILD_DIRS should be used instead. Sorted
the makevars.map file and added SUBST_CLASSES.
checking routines don't use physical lines anymore, so there's no need
to distinguish them. Removed deprecated "@" line checks from the PLIST
checker. These lines are all reported as "Unknown PLIST directive" now.
The --autofix support has been rewritten to be more easily usable.
Automatic fixes are currently restricted to the sort order of SUBDIR
entries in category Makefiles. It had been the complete category
Makefile before. Added a new diagnostic, prefixed with "NOTE:", that is
used for important informational diagnostics, for example autofixed
files.
one backslash reach the argument to sed(1), the backticks are replaced
with a call to open("... |"). Now the first argument to sed(1) contains
\$ instead of a simple $.
possible and base all other checks on the logical lines. The physical
source lines are saved literally to make implementing the --autofix
option easier.
Changed the names of the datatypes (Yes_Or_Undefined => Yes, Boolean =>
YesNo, Integer => PkgRevision) in makevars.map. It is now an error if
PKGREVISION appears outside a package Makefile. Improved detection of
Makefile.common. Improved wording of the warning for relative
directories in the form ../package.
Rewrote the typechecking code for Makefile variables. The variable
definitions are extracted from logical lines instead of physical lines,
comments are separated from actual values, for Lists of something, each
something is checked. All URLs are subject to MASTER_SITES expansion.
MASTER_SITES is checked to be a List of URLs.
Added CONFLICT to the list of deprecated variables (actually it's a
typo). Added PLIST_SUBST to makevars.map as a List. Allow comments after
a YES/yes/NO/no value of variables.
Fixed the detection of list variables that are modified with operators
other than "+=". Added *_SKIP to the list of plural variable names.
Removed some unused variables from main(). (This change include
white-space changes.)
Added a data type Readonly for variables that must not be given any
value at all by the package Makefile. Marked PKGBASE and PKGVERSION
read-only, because leaving them read-write would make the way PKGNAME is
calculated too complex. Made the check for the "+=" operator independent
of the data type. Added more patterns for accepted variable names for
lists.
Added support to typecheck "List of Something" in Makefile variables.
DEPENDS and BUILD_DEPENDS are of type List of Dependency, CFLAGS are
simply a List.
Improved heuristics for packages that use some common Makefiles,
reducing the number of false diagnostics in -Wvague mode. Moved checks
for SVR4_PKGNAME out of -Wvague mode. Added check for misspelled
NO_{SRC,BIN}_ON_{FTP,CDROM}.
Unbreak url2pkg on sourceforge hosted packages -- the MASTER_SITES
did not contain the trailing slash, thus breaking the download.
XXX: prdownloads.sf.net is not recognized as a sourceforge "mirror"
since the previous version of url2pkg.
verifypc is a tool that sanity checks the dependencies of a package based on
the results of a successful build, assuming the package uses pkg-config to
detect dependencies.
verifypc will warn you if the configuration script requests an uninstalled
or unavailable package (not specified as a direct dependency) or if the
requested package does not match the version specification in the packages'
dependencies.
