- Added warnings for imperfect quoting style inside double quotes.
- All shell commands are checked for correct quoting of make(1) variables
in shell assignments like --prefix=${PREFIX:Q} and TMPDIR=${WRKSRC:Q}.
Changes since 4.49:
- Added checks for shell commands whose exitcodes are ignored.
- Added checks for CFLAGS, CPPFLAGS and CXXFLAGS.
- Improved diagnostics.
- Various bugfixes.
Changes to lintpkgsrc:
- Avoided double occurrence of pkg_install.
- Rewrote the message formatting.
- The indentation of explanations is changed to be always a tabulator
character instead of depending on the last message.
- In verbose mode, statistics about the frequency of the individual
messages are printed before exiting.
- The command line option -Wextra enables additional warnings that are
not enabled by default because I have been told that Alistair would kill
me for them. :)
- Improved the shell tokenizer by recognizing parentheses.
- Improved checking of pkgsrc-internal files (mostly in pkgsrc/mk).
- Added a (trivial) spellchecker.
- Added checks for shell code that ignores the exitcode of commands.
- Added checks for CFLAGS, CPPFLAGS and CXXFLAGS.
- Avoided false positive warnings for absolute filenames in AWK code.
- Added checks for .for variables.
- Pkglint can check single files in directories that are three levels deep.
This is mostly useful for checking patch files.
Changes since 4.48:
- Added a check for .include lines in all Makefiles and *.mk files to make
sure that only existant files are referenced and that pkgsrc packages do
not contain references to packages in pkgsrc-wip.
- Extended the --autofix option to all Makefile* and *.mk files. Before, only the
package Makefile could be fixed.
- Rewrote the detection of direct use of tool names.
- Append-only lists may be initialized using the "?=" operator.
- The typical form of ShellWords for CONFIGURE_ARGS, --foo=${FOODIR}, is checked for
quoting errors.
- All PKG_OPTIONs are checked whether they are documented in mk/defaults/\
options.description.
- Many small fixes and improvements.
a "$" in the modifier part, as long as they don't contain references to
other variables.
- Removed checkline_mk_direct_tool_use().
- Renamed checkline_mk_shellcmd() to checkline_mk_shelltext(). This sub-
routine is also used for checking the right hand side of variable assign-
ments.
- Renamed direct_tools_ok_vars to non_shellcode_vars, which is more
appropriate. Added some variables to it that produced false positive
warnings.
checklines_package_Makefile(). Then, renamed
checklines_package_Makefile() to checklines_mk(), as it not only
applies to the main package Makefile.
- Fixed the regular expression for detecting ambiguous variables like
$f, which could mean either ${f} or $$f.
- When checking for deprecated variables, also mention the ones that
don't have modifiers.
- Moved the whole complicated code from checklines_package_Makefile() to
checkline_mk_shellcmd() and checkline_mk_varassign().
- Reenabled the -Wdirectcmd options by default.
- Added specialized subroutines for checking a single line.
- Removed checklines_Makefile_varuse().
- Removed checklines_direct_tools(), which has been replaced by the
specialized routines mentioned above.
disabled by default, as I am currently working on it, and neither the
current nor the previous version is good enough for the end user.
- Pkglint can now distinguish the two kinds of tools: The ones that
should be used like ${TOOL} and the ones that are provided in the
.tools/bin directory. The list of the first kind of tools is extracted
from the mk/tools/ directory instead of being hard-coded. This adds
a whole lot of new warnings.
- Rewrote the code for loading the deprecated.map file. It is only
loaded once now. The code that checks for use of deprecated variables
can handle multiple variables in a single line now.
for directories starting with %D.
- Converted load_make_vars_typemap() to the common form of caching
subroutine. This makes pkglint about 10 percent faster.
- Added explanations for the :Q and the :M*:Q operators in Makefiles.
a single character. Seen in some package versions like "pkg-200211??".
- Allow the "," and the "@" in filenames.
- Allow the "gopher" scheme in URLs.
- Don't warn for unexpected PERL5_PACKLIST, bases on PKGNAME, if the latter
contains references to other variables.
- When PKGNAME is equal to DISTNAME, this is not a warning, only a note.
- Added a check for .include lines in all Makefiles and *.mk files to make
sure that only existant files are referenced and that pkgsrc packages do
not contain references to packages in pkgsrc-wip.
*.mk file is checked.
- Added warnings for unusual make targets. Everything except the usual
{pre,do,post}-* targets is considered unusual. Exceptions may be declared
in the Makefile using ".PHONY".
- The directives are checked to contain arguments if and only if needed.
- The .ifndef and .ifdef directives are marked as deprecated because
the parsing algorithm of NetBSD's make is so bad that it cannot
distinguish ".if" from ".ifdef".
- Added notes whenever ".undef" is used with a variable that had been used
in a ".for" loop before. Undefining the variable is simply unnecessary.
Changes since 4.46:
- Made the --explain command line option work.
- Added many explanations for existing diagnostics.
- Improved the diagnostics.
- In --autofix mode, no backup files are created anymore.
- Temporarily disabled the check for direct use of tool names.
- Fixed some false positive warnings.
- Added a check for the use of absolute pathnames in shell commands.
check had been removed some time ago due to the huge number of false
positives. Now that pkglint can parse shell commands quite well, it
has been reintroduced, as absolute pathnames often indicate unportable
features of a package. To implement this check (and a few others)
accurately, the whole code for checking shell commands has been
rewritten as a finite state machine.
warnings that LIST+=FOO=${FOO} should rather be LIST+=FOO=${FOO:Q}.
The cause was that I had added a capturing group in a regular
expression that also contained a back reference (\2). Adjusted the
back reference to \3. (This is the only place in pkglint where such
back references are used, so I don't have much experience with them.)
- Added the $line->replace() method for a convenient way to achieve simple
autofix tasks.
- When autofixing, no backup file is created (it had been created before).
1. Usually "cvs diff" is available.
2. All current autofixed changes are trivial.
3. The fixed file is first written to disk in a new file and then renamed
to the original file, greatly reducing the risk of data loss.
- Fixed a bug in checkdir_package(): In the call to load_package_Makefile(),
the last parameter had not been passed by reference, but by value.
Luckily this had not influenced any other part of pkglint.
- While there, I noticed that it is not necessary to pass some subroutines
the lines of all included Makefiles, so I removed that parameter.
- Removed the unused variable $opt_explain. It should really have been
PkgLint::Logging::set_explain().
- In accordance to my personal idea of beauty, the --explain messages are
indented as deep as the last diagnostics, and not by a single "\t".
files. New TODO items:
* extract the DISTFILES if they exist and check all files in them for:
- security holes,
- coding style violations,
- possible unportable constructs
Changes since 4.45:
- Added the type ShellWord.
- Added quoting checks for variables that are appended to a
List of ShellWord.
- Multiple -v increase the verbosity.
- The --autofix option is no longer undocumented, but still experimental,
as this is the only part of pkglint that can modify external files.
- Added an --explain option that provides additional help for the
diagnostics.
- Added checks for unportable CPP macro names in patches.
- Updated the documentation.
:M*:Q instead of a simple :Q. Currently only the GNU configure
scripts need the white-space stripped, so it's enough to quote
those (see regex_gnu_configure_volatile_vars in the source).
- Added detection of redirection operators and comments to the
regex_shellword constant.
- Changed the naming conventions for list data types. Now List simply
means a list. List+ means a list that should only be modified using
"+=", not "=". List! means an internal list. And List!+ is the
combination of both.
The distinction is necessary because of the introduction of
redirection operators in the regex_shellword. When checking the data
type of list elements, the lists are split up into shell words instead
of simply using split(). This leads to much better results.
- When splitting a variable value or shell command into words, anchor the
regular expression at the beginning of the string.
- Removed log_subinfo().
- Added log_debug().
- Multiple -v command line options increase the verbosity.
- Added the regex_shellword constant that will allow better parsing of
shell commands. Currently it is only producing debugging information.
- Long [info] messages have been changed to [debug] messages.
CFLAGS=${CFLAGS}, check for the correct modifiers. The above is
obviously not correct, as CFLAGS may contain white-space.
CFLAGS=${CFLAGS:Q} is also a little wrong in that it may contain leading
and/or trailing white-space, which must be discarded, too, because the
broken GNU configure scripts cannot handle them correctly. This can be
done using ${CFLAGS:M*:Q}, which first splits CFLAGS into a list of
shell words, then selects all of them and finally combines the words
forming a nicely formatted string without leading and trailing
white-space where all entries are separated from each other by a single
space.
- Added a new type ShellWord that is used for MAKE_ENV and the like to
check for invalid FOO="${VALUE}" additions. They really should be
FOO=${VALUE:Q}, as they may already contain embedded quotes. This is
especially important for CPPFLAGS and CFLAGS.
- Fixed perl -T warnings:
- Replaced $#{@{$lines}} with $#{$lines}.
- Avoided calls to external programs (sed and digest).
- Removed redundant warning if DISTNAME is set to an invalid package name
and PKGNAME is not defined.
- Changed dependency from pkgtools/digest to security/p5-Digest-SHA1.
Changes since 4.43:
- Removed the remaining code that had been imported from FreeBSD's
portlint.
- Removed the (undocumented) -Wvague option.
- Removed the -Wexec and -Wparen options. The latter had no effect at all,
and the former cannot lead to false positives, so it is always enabled.
- Variable names starting with an underscore are reserved for internal
pkgsrc use.
- Added some more type checks.
- Renamed the type Dependency to DependencyWithPath.
- Added new types Dependency, PlatformTuple and RelativePkgDir.
- Added some of the common variables to the list of typed variables.
Changes since 4.42:
- Reduced the number of duplicate diagnostics when checking multiple files.
Only diagnostics concerning the current package are printed.
- Added checks for EXTRACT_SUFX and PKG_INSTALLATION_TYPES.
- Added a work-around for the PHP patches warnings.
- PERL5_PACKLIST should not contain references to other variables.
- Added the -s|--source command line option to show the code along with the
diagnostics.
- Fixed a bug in get_logical_line(); logical lines have not had their
physical lines attached.
- Deprecated variables are not only checked when they are defined but also
when they are used.
- Added a check that PERL5_PACKLIST does not contain references to other
variables. Some packages have ${PERL5_SITEARCH} in it, which results in
a double slash, and the CHECK_FILES framework cannot handle this.
in the $all_lines parameter, but the whole text of the included files in
the $whole parameter.
This change avoids duplicate diagnostics for *.mk and Makefile* in the
package directory. A side effect is that package authors only get the
diagnostics they can probably fix, as diagnostics from included files are
not given.
Changes since 4.41:
- In .mk files, line continuations are properly handled.
- An open bracket is allowed in variable and tool names.
- Added checks for ALTERNATIVES and INSTALL/DEINSTALL files.
- By default, don't check files in ${FILESDIR}.
- Warn about unknown file and directory names.
- Documented the --recursive option.
- Added an option -Cmk for checking .mk files besides buildlink3.mk.
- pkglint can handle individual files on the command line, not only
packages. (experimental)
- Replaced checkfile_buildlink3_mk() with checkfile_mk().
- Factored out the checkfile() subroutine from checkdir_package().
since the pkglint Makefile sets MANINSTALL but shouldn't, as this
variable is intended to be user-defined. This is only a work-around,
which needs to be addressed properly for the other packages setting
MANINSTALL, too.
- Removed all references to the build-time pkgsrc directory, ${PKGSRCDIR}.
This makes the binary package independent of the build location.
Fixes PR 32006.
Changes since 4.38:
- Disabled the check on Makefile variable order, as the discussion on
tech-pkg has not finished yet.
- Improved detection of valid tool names.
- Improved detection of direct use of tool names. (Less false positives.)
- Improved the diagnostics for direct use of tool names.
pkgsrc/mk/defaults/mk.conf. All those variable names are then checked as
being of type Userdefined. As that file is currently too unstructured
and contains too much garbage, this check cannot yet be enabled.
- pkglint(1) does not use the hard-coded PKGSRCDIR anymore.
This fix is related to PR 32006, but doesn't fix it, since the PR is
about lintpkgsrc(1), not pkglint(1).
- It is an error if packages define a variable whose name starts with an
underscore.
- Removed the (vague) "wip" check, as it has been replaced by special checks for
DEPENDS and .include directives.
- Disabled the check for variable ordering until the discussion on tech-pkg
has led to any results.
- PKGNAME is checked for being a valid package name.
- PLIST files should not contain filenames that match */CVS/*.
- Completely rewrote the check for variable ordering. The new code
operates on a data structure that's easily understandable and
extendable (see the source for an example). It also generates greatly
improved diagnostics. As the old code had been enabled only when
checking with -Wvague or -Wall, it has been seldom used anyway.
- Removed the hard-coded values for valid tool names.
The detection is good enough.
- Make sure that the domain part NetBSD.org is written like this
in mail addresses.
- Added checks for TOOL_* variables.
- Added {pre,do,post}-extract to the list of valid stages.
- Fixed the regular expression for detecting tool names.
- Added a check for invalid syntax in tool names.
- Improved the diagnostic for enumerations.
- Added many of the variables found in pkgsrc/mk/* to makevars.map.
- Added the function log_fatal, whose output goes to stderr instead of
stdout.
- All files in pkgsrc/mk/ are excluded from checking, as they may use
private variables and do other questionable things.
- Removed the Language type, as it is a simple enumeration.
- Added the Userdefined type to distinguish user-definable and read-only
variables.
Implemented checking for enumeration types. Added new types Filemask,
Filename, Language, Option, Pathmask, Pathname, Stage, Varname and
WrksrcSubdirectory. Made the check for URLs stricter. Added some
variables to the makevars.map file.
Removed the -Wworkdir option. Added an --import option that replaces
-Wworkdir. Changed the warning about uncleaned work* directories into an
error. Updated the man page.
Fixed the --autofix handling of category Makefiles. Added an --import
option that helps importing packages from pkgsrc-wip. When checked with
--import, the package is checked as if it weren't part of pkgsrc-wip.
Fixed undefined behavior when reading a file that ends in a continuation
line. Any use of ${WRKSRC}/.. is considered an error, as ${WRKSRC} is
meant to point to the top of the build directories. A proper combination
of WRKSRC, CONFIGURE_DIRS and BUILD_DIRS should be used instead. Sorted
the makevars.map file and added SUBST_CLASSES.
checking routines don't use physical lines anymore, so there's no need
to distinguish them. Removed deprecated "@" line checks from the PLIST
checker. These lines are all reported as "Unknown PLIST directive" now.
The --autofix support has been rewritten to be more easily usable.
Automatic fixes are currently restricted to the sort order of SUBDIR
entries in category Makefiles. It had been the complete category
Makefile before. Added a new diagnostic, prefixed with "NOTE:", that is
used for important informational diagnostics, for example autofixed
files.
one backslash reach the argument to sed(1), the backticks are replaced
with a call to open("... |"). Now the first argument to sed(1) contains
\$ instead of a simple $.
possible and base all other checks on the logical lines. The physical
source lines are saved literally to make implementing the --autofix
option easier.
Changed the names of the datatypes (Yes_Or_Undefined => Yes, Boolean =>
YesNo, Integer => PkgRevision) in makevars.map. It is now an error if
PKGREVISION appears outside a package Makefile. Improved detection of
Makefile.common. Improved wording of the warning for relative
directories in the form ../package.
Rewrote the typechecking code for Makefile variables. The variable
definitions are extracted from logical lines instead of physical lines,
comments are separated from actual values, for Lists of something, each
something is checked. All URLs are subject to MASTER_SITES expansion.
MASTER_SITES is checked to be a List of URLs.
Added CONFLICT to the list of deprecated variables (actually it's a
typo). Added PLIST_SUBST to makevars.map as a List. Allow comments after
a YES/yes/NO/no value of variables.
Fixed the detection of list variables that are modified with operators
other than "+=". Added *_SKIP to the list of plural variable names.
Removed some unused variables from main(). (This change include
white-space changes.)
Added a data type Readonly for variables that must not be given any
value at all by the package Makefile. Marked PKGBASE and PKGVERSION
read-only, because leaving them read-write would make the way PKGNAME is
calculated too complex. Made the check for the "+=" operator independent
of the data type. Added more patterns for accepted variable names for
lists.
Added support to typecheck "List of Something" in Makefile variables.
DEPENDS and BUILD_DEPENDS are of type List of Dependency, CFLAGS are
simply a List.
Improved heuristics for packages that use some common Makefiles,
reducing the number of false diagnostics in -Wvague mode. Moved checks
for SVR4_PKGNAME out of -Wvague mode. Added check for misspelled
NO_{SRC,BIN}_ON_{FTP,CDROM}.
The last change introduced some false positive diagnostics. If a
distinfo file contains the checksum for a patch that does not exist,
this is not considered to be an error, but only a warning, as the
additional checksums do not influence the package.
Changed the file name pattern for patch files to avoid false positives
for DISTFILES that start with "patch-" (requested by salo). Added a
warning for invalid patch file names.
Changes since 4.20 include:
- In the diagnostics, FATAL is replaced with ERROR.
- The command line option -g changes the format of the diagnostics to be
similar to the one of gcc.
- Removed false warnings when checking if PLISTS are ordered.
- Removed the check for contiguous blank lines.
- Added a check to PLISTs that man pages are not installed in share/man.
- pkglint -r allows to recursively check directories.
- The pkgsrc root directory can be checked.
- Removed false warnings for patches that contain "---" lines.
- Switched to checking logical lines instead of physical lines.
Diagnostics for logical lines contain the range of physical lines
instead of a single line number.
- Some ../.. path components are stripped from the diagnostics.
- When checking for the direct use of tool names, only the context of the
use, not the whole shell command is output.
- Patches that contain RCS Ids should not have the -ko CVS mode anymore.
- Variables that are modified using the "+=" operator are considered to be
lists, so they should have a name indicating a plural.
