USE_TOOLS and any of "autoconf", "autoconf213", "automake" or
"automake14". Also, we don't need to call the auto* tools via
${ACLOCAL}, ${AUTOCONF}, etc., since the tools framework takes care
to symlink the correct tool to the correct name, so we can just use
aclocal, autoconf, etc.
This means that the MPLAYER_ENABLE_RUNTIME_CPU_DETECTION,
MPLAYER_DISABLE_DRIVERS and MPLAYER_USE_MEDIALIB become deprecated
(although still recognized).
Visible changes in the resulting binary packages should be minimum by
default (everything that was enabled before still is, and the same
dependencies are kept). A notable addition, though, is the support for
user-defined menus, closing PR pkg/29784.
Also note that (almost) all dependencies have now a corresponding option
to disable them in case you want to get a minimalist mplayer package.
'make show-options' is your friend ;)
With thanks to wiz@ and dillo@ for their comments and help.
Several changes are involved since they are all interrelated. These
changes affect about 1000 files.
The first major change is rewriting bsd.builtin.mk as well as all of
the builtin.mk files to follow the new example in bsd.builtin.mk.
The loop to include all of the builtin.mk files needed by the package
is moved from bsd.builtin.mk and into bsd.buildlink3.mk. bsd.builtin.mk
is now included by each of the individual builtin.mk files and provides
some common logic for all of the builtin.mk files. Currently, this
includes the computation for whether the native or pkgsrc version of
the package is preferred. This causes USE_BUILTIN.* to be correctly
set when one builtin.mk file includes another.
The second major change is teach the builtin.mk files to consider
files under ${LOCALBASE} to be from pkgsrc-controlled packages. Most
of the builtin.mk files test for the presence of built-in software by
checking for the existence of certain files, e.g. <pthread.h>, and we
now assume that if that file is under ${LOCALBASE}, then it must be
from pkgsrc. This modification is a nod toward LOCALBASE=/usr. The
exceptions to this new check are the X11 distribution packages, which
are handled specially as noted below.
The third major change is providing builtin.mk and version.mk files
for each of the X11 distribution packages in pkgsrc. The builtin.mk
file can detect whether the native X11 distribution is the same as
the one provided by pkgsrc, and the version.mk file computes the
version of the X11 distribution package, whether it's built-in or not.
The fourth major change is that the buildlink3.mk files for X11 packages
that install parts which are part of X11 distribution packages, e.g.
Xpm, Xcursor, etc., now use imake to query the X11 distribution for
whether the software is already provided by the X11 distribution.
This is more accurate than grepping for a symbol name in the imake
config files. Using imake required sprinkling various builtin-imake.mk
helper files into pkgsrc directories. These files are used as input
to imake since imake can't use stdin for that purpose.
The fifth major change is in how packages note that they use X11.
Instead of setting USE_X11, package Makefiles should now include
x11.buildlink3.mk instead. This causes the X11 package buildlink3
and builtin logic to be executed at the correct place for buildlink3.mk
and builtin.mk files that previously set USE_X11, and fixes packages
that relied on buildlink3.mk files to implicitly note that X11 is
needed. Package buildlink3.mk should also include x11.buildlink3.mk
when linking against the package libraries requires also linking
against the X11 libraries. Where it was obvious, redundant inclusions
of x11.buildlink3.mk have been removed.
doesn't happen until after the package is extracted. Prior to
extraction, cache the variables named in MAKEVARS using MAKEFLAGS
instead. This avoids invoking each expensive computation up to four
times before it gets cached. For packages that include lots of
buildlink3.mk files, this saves a lot of CPU time.
something other than [yY][eE][sS].
If a deprecated warning is defined, interpret it as YES (because some
variables in defaults/mk.conf have defined/not defined as yes/no).
Reviewed by dillo.
-z, --suffix
Causes the next argument to be interpreted as the backup extension,
to be used in place of ".orig".
From Joerg Sonnenberger via private mail.
option names:
PKG_OPTIONS_LEGACY_OPTS+= old:new
If PKG_DEFAULT_OPTIONS or PKG_OPTIONS.foo contains option old (or
-old) it is rewritten to new (or -new) and a warning is issued by
the supported-options-message target.
binary tool. We can now use the tools framework for this. Remove the
various nroff/groff buildlink3.mk/builtin.mk files as editors/jove was
their only user in pkgsrc.
created in a temp directory, issue a warning if we fail to install
it in the package directory. If the error happens on the top
level (master) cache file, then error out. For a subdirectory,
issue a warning and drop that directory from the master cache.
in packages, e.g on my NetBSD system:
$ cd pkgsrc/graphics/gimp && make show-var VARNAME=BUILTIN_PACKAGES
bzip2 heimdal openssl db1 gettext iconv zlib pthread
$
Ok'ed by jlam.
to detect the presence of libraries in the base system.
The input variable is BUILDLINK_FIND_LIBS, which is a list of library
names, e.g. ncurses, iconv, etc., that will be sought in the base
system. BUILDLINK_LIB_FOUND.<lib> is set to "yes" or "no" depending
on the result of the search.
An example use is:
BUILDLINK_FIND_LIBS:= intl iconv
.include "../../mk/buildlink3/find-libs.mk"
# ${BUILDLINK_LIB_FOUND.intl} and ${BUILDLINK_LIB_FOUND.iconv} are now
# either "yes" or "no".
-- the checks for grep and egrep are broken because when passed GREP
and EGREP in the environment, the script causes GREP and EGREP to be
set to empty strings, which causes GNU configure scripts to hang or
break. Pass the real paths to grep and egrep through using ac_cv_path_GREP
and ac_cv_path_EGREP as well to avoid the brokenness. This fixes the
build of textproc/gsed.
paths for the tools that the package uses through the shell environment.
We do this since these paths may be hardcoded into package scripts,
and if they're not pre-specified, then they'll be searched for in the
PATH, which would find the ones in ${TOOLS_DIR}.
The variable names that GNU configure scripts expect are named in
_TOOLS_VARNAME_GNU.* for the various tools.
operator does not result in an empty string as make(1) doesn't like
them. Also, rewrite the variable substitution for the find(1) command
to be more readable, and get rid of the extra grep(1) in favor of
using :N.
variables that use the :sh modifier. This still causes expansion to only
happen when referenced, and has the advantage of being :Q-safe.
Bring back the changes from revision 1.19 of mk/subst.mk now that the
problem noted above has been fixed. This passes the buildlink-unwrap
regression test.
assumptions being made by the USE_PKGLOCALEDIR code and the wrapper
framework since it added backtick expressions to the SUBST_FILES.*
variables, which were being mangled by the :Q modifier. This is
evident when running "make regress" in regress/buildlink-unwrap.
Mea culpa.
As data for a given binary package is loaded, increment a count
stored with PKGPATH as the index. Then use a psuedo-multidimensional
array with PKGPATH and the # as the index. This allows for much
faster lookups and scales linearly with the number of packages.
The old way scaled as the product of the number of pkgsrc entries
and the total number of binary packages. Not a pretty value.
This makes the actual README.html generation part run about 3x faster.
Approach suggested by soda.
${WRKSRC}. Just directly create the msgfmt wrapper in the proper
target. Also, note that the msgfmt handling should eventually migrate
to the tools framework so that build dependencies and binary paths are
correct.
some of the older packages on ftp.netbsd.org. For example,
pkg_info -B ftp://ftp.netbsd.org/pub/NetBSD/packages/1.5.2/vax/All/cascade-1.4.tgz
will give
OPSYS= NetBSD
instead of
OPSYS=NetBSD
due to a directory permissions error, then error out.
- when the master cache has just been newly created, don't also report
that the master cache is up to date.
Both were suggested by Christian Hattemer.
Plan:
(1) Change USE_PERL5=build into USE_TOOLS+=perl.
(2) Change all other USE_PERL5 into including perl5/buildlink3.mk.
Possibly, for packages that don't actually build anything with perl,
but merely require it for the perl interpreter, we can instead do:
USE_TOOLS+= perl
TOOLS_DEPMETHOD.perl= DEPENDS
but this is more verbose than simply including the perl5/buildlink3.mk
file.
Move the PERL5_REQD computation into a lang/perl5/version.mk file,
and only do the USE_PERL5 logic in bsd.pkg.use.mk if we're not using
the new tools framework. This consolidates all of the perl-handling
into two places -- lang/perl5 and mk/tools/perl.mk.
- completely redo the code which decides on the machine architecture,
operating system, and operating system version for the binary packages.
The old way just used to directory names to take a guess. The new
way creates a cache file containing meta-data for all the binary packages
in each "All" directory. This cache file is consulted when generating
the lists of available binary packages. The meta-data is obtained with
pkg_info so it should always be correct even if you do something silly
like mix OS_VERSION or MACHINE_ARCH packages up in the same directory.
Among the benefits are: works when PACKAGES is not $PKGSRC/packages,
works with a more or less arbitrary subdirectory structure, works
when there are subdirectories for multiple operating systems.
This portion of the fix should address PR25390.
The cache files are only updated when the contents of an "All" directory
changes or if the cache file format changes. There is some room for
improving the updating of the cache files, but its not too bad the way
it is.
- fix up some of the awk code so that generadme.awk works with Solaris
nawk as well as NetBSD's nawk and gawk (for pre-2.0 systems).
- remove some "if ! foo" shell constructs to increase portability.
- be more consistent with what variables get passed to mkreadme from
make and which ones are determined automatically. Mostly this meant
moving stuff into mkreadme to make it easier to run it standalone.
directory or the local patches directory exist. Also, emit the message
informing the user that pkgsrc patches are being applied that was lost
in the previous commit.
apply-distribution-patches and apply-pkgsrc-patches. Rewrite the
latter code to be more "whitespace-friendly" and to not need to invoke
${AWK} as many times. Drop support for PKGSRC_SHOW_PATCH_ERRORMSG
and simply always show the error message.
GNU tar archives. Create a new EXTRACT_USING value "nbtar" that causes
tar/ustar archives to be extracted using pax-as-tar, which understands
most GNU tar extensions. Default to EXTRACT_USING=nbtar, as quite a
few packages are distributed in GNU tar archives.
right in each variable, starting with the most generic one. Later
definitions override earlier ones. This makes the ``order'' testcase in
regress/pkg-options succeed.
which can take multiple values -- "pax" or "gtar". The default value
of EXTRACT_USING is "pax", which more closely matches reality since
before, we were using bootstrap "tar" for ${GTAR} and it was actually
pax-as-tar. Also, stop pretending pax-as-tar from the bootstrap kit
or on NetBSD is GNU tar. Lastly, in bsd.pkg.extract.mk, note whether
we need "pax" or "gtar" depending on what we need to extract the
distfiles.
"bison" and "bison-yacc". bison is just a symlink to the real bison,
but "bison-yacc" provides a "yacc" in ${TOOLS_DIR} that does "bison
-y". This allows the tools framework to provide everything that a
package might look for when asking for "bison" and avoids relying on
the bison being available in the PATH.
Ditto for mailx. Lastly, /usr/bsd/zcat isn't gzcat (doesn't understand
gzipped files), so don't use it as a gzcat replacement. Changes from
Georg Schwarz in private email.
the final man pages to be compressed. If MANZ is not defined, then
we want the final man pages to be uncompressed.
We need to figure out if during installation, we need either gunzip
or gzip to decompress or compress the installed man pages. If a
package sets MANCOMPRESSED to "yes" or "no", then it's an indication
to the install code that the package itself installed the man pages
either compressed or uncompressed. If a package sets
MANCOMPRESSED_IF_MANZ, then the package uses BSD-style makefiles, so
we need to determine if the BSD-style makefile causes the man pages
to be compressed or not. In this case, we need to check in PKGMAKECONF
whether MANZ is set or not.
XXX Perhaps we should also check for MKMANZ if a package uses BSD-style
XXX makefiles?
duplicates in the output, output warnings if necessary. The $$dlist at
the end is unquoted intentionally to omit spaces in the output.
Approved by jlam.
as needing both (no impact since they're both satisfied by the tar
binary installed by the bootstrap kit). There's some funniness in
the extraction code where we check for pax or GNU tar that needs to
be resolved. Remove the TAR=${GTAR} hack since it's no longer needed
after these changes.
XXX Later, a sweep needs to be made to see where we actually need GTAR
XXX and where we only need TAR, probably triggered by whether we call it
XXX with the "z" option or not. Packages that need GTAR should explicitly
XXX add USE_TOOLS+=gtar to the package Makefile.
separate file pkgsrc/mk/bsd.pkg.patch.mk. Also, include this file
ahead of the inclusion of bsd.tools.mk so that tools required to
patch are created by the tools framework.
other files without making assumptions about the location of the top of
the pkgsrc tree. This fixes problems with running "make bulk-cache"
in /usr/pkgsrc (PR pkg/30288).
at the time that they're defined: ${AWK}, ${GREP}, ${SED}, ${TRUE}.
Move the inclusion of compiler.mk below the inclusion of bsd.tools.mk
so that these variables are properly defined by the time they are
used. This should fix problems where pkgsrc gcc was not being properly
detected when using the new tools framework.
This change has the side effect of changing the relative order of the
compiler directories and the tool directory in the PATH, but this has
no impact on existing packages, and actually makes more sense anyway.
check for whether ${PATCHDIR} exists or not, or whether PATCHFILES is
defined or not. This avoids the use of != in a variable definition
just to find out if we need patch or not.
is undefined. This avoids many unnecessary make(1) warnings. As this file
is only intended to be used from packages, don't look for obsolete.mk in
other directories than ../../mk/defaults/.
modify bsd.pkg.extract.mk to use them. As a side-effect, we stop
hardcoding ${LOCALBASE} as the install prefix for these tools so this
becomes more pkgviews-friendly.
bsd.pkg.mk under the new tools framework. Instead, just note that if
we set PKGSRC_LOCKTYPE, that pkgsrc will require shlock. Remove
"shlock" from the large PKGSRC_USE_TOOLS list since we don't need it
all of the time. This fixes PKGSRC_LOCKTYPE != none when using the
new tools framework.
in mk/tools/perl.mk since many packages expect to be able to use the
variables defined in vars.mk, but those variables can only be defined
if PERL5 is correctly defined, and that is only true after it is set
here.
loop are only used afterwards, split out the EVAL_PREFIX code into a
separate file find-prefix.mk that can be included at any point to
generate a prefix-computation for the variables specified by FIND_PREFIX
at the inclusion point.
them to PLIST_SUBST. They may simply be defined later on in the
Makefile processing. Instead check whether USE_PERL5 was specified
to determine whether to pass along PERL5_{SITEARCH,SITELIB,ARCHLIB}
to PLIST_SUBST.
VAR= a
PTR= VAR
${PTR}?= b
Given these definitions, ${VAR} == "b" (unintuitively). We can work
around this by doing:
VAR= a
PTR= VAR
.for _v_ in ${PTR}
${_v_}?= b
.endfor
In this case, ${VAR} == "a" (as expected).
Use the second form of assignment in this changeset so that the expected
behavior happens for the "TOOL" name variables. This fixes "recursively
defined" errors for some of the "TOOL" variables when the pkgsrc tool
replaces the system-supplied one.
These files would normally go away after packages have been taught to
replace including autoconf.mk with "USE_TOOLS+= autoconf", and similarly
for automake.mk.
within make. This allows us to use EVAL_PREFIX to locate the installation
prefix for autoconf and automake, which is more pkgviews-friendly. In any
case, this implementation is a lot simpler anyway.
${TOOLS_DIR} so that we avoid inspecting various variables during the
top-level make invocation -- rather, we defer inspection until the
target is actually made. This allows TOOLS_REAL_CMDLINE.*,
TOOLS_REAL_CMD.*, and TOOLS_REAL_ARGS.* to be defined after bsd.tools.mk
is included and still affect the creation of the tools under ${TOOLS_DIR}.
value of TOOLS_REAL_CMDLINE.makeinfo. In this case, we split MAKEINFO
into the command and the arguments, and allow the default command line
to be constructed.
separate file pkgsrc/mk/bsd.pkg.extract.mk. Also, include this file
ahead of the inclusion of bsd.tools.mk so that tools required to
extract are created by the tools framework.
This makes more sense since there is never going to be a pkgsrc
replacement for ldconfig. We now always create an ldconfig tool in
${TOOLS_DIR} that either calls the system-supplied one if it exists,
or is a no-op.
that a package needs. Tools that pkgsrc needs are listed in
PKGSRC_USE_TOOLS, and tools that a package needs on top of that are
listed in USE_TOOLS.
Define "TOOL" variables, e.g. SED, AWK, MKDIR, etc. for each of the
tools that pkgsrc needs, and "TOOLS_TOOL" variables, e.g. TOOLS_SED,
TOOLS_AWK, TOOLS_MKDIR, etc. for each of the tools that a package
needs. These variables contain the full command line to the real
command and arguments needed to invoke the tool.
created under ${TOOLS_DIR} that invoke <tool>, e.g.
TOOLS_ALIASES.gawk= awk
The example above causes the "gawk" tool to be added to ${TOOLS_DIR} as
both "gawk" and "awk".
apps. Used at the moment for X11_TYPE=[xorg|XFree86] to fix
the build/installation of some packages, see
http://mail-index.netbsd.org/tech-pkg/2005/05/05/0024.html
for more information.
To switch between 'build' or 'full' dependency method use:
X11CLIENTS_DEPMETHOD=[full|build]
caches variable definitions that were computed by make. These variables
are specified by listing them in MAKE_VARS, e.g.,
.if !defined(FOO)
FOO!= very_time_consuming_command
.endif
MAKE_VARS+= FOO
bsd.pkg.mk will include only the one generated during the most recent
phase. A particular phase's makevars.mk file consists of variable
definitions that are a superset of all of the ones produced in previous
phases of the build.
The caching is useful because bsd.pkg.mk invokes make recursively,
which in the example above has the potential to run the very time-consuming
command each time unless we cause FOO to be defined for the sub-make
processes. We don't cache via MAKE_FLAGS because MAKE_FLAGS isn't
consistently applied to every invocation of make, and also because
MAKE_FLAGS can overflow the maximum length of a make variable very
quickly if we add many values to it.
One important and desirable property of variables cached via MAKE_VARS
is that they only apply to the current package, and not to any
dependencies whose builds may have been triggered by the current
package.
The makevars.mk files are generated by new targets fetch-vars,
extract-vars, patch-vars, etc., and these targets are built during
the corresponding real-* target to ensure that they are being invoked
with PKG_PHASE set to the proper value.
Also, remove the variables cache file that bsd.wrapper.mk was generating
since the new makevars.mk files provide the same functionality at a
higher level. Change all WRAPPER_VARS definitions that were used by
the old wrapper-phase cache file into MAKE_VARS definitions.
framework figure out by itself whether a wrapper or a symlink should be
created based on the real command and any arguments that may need to be
invoked.
- PKG_SUGGESTED_OPTIONS (set by pkg maintainer in pkg Makefile)
lists options enabled by default.
- PKG_DEFAULT_OPTIONS (set by user in /etc/mk.conf)
enables or disables (when prefixed with `-') options for
all packages, takes precedence over PKG_SUGGESTED_OPTIONS
- PKG_OPTIONS.foo (set by user in /etc/mk.conf)
enables or disables (when prefixed with `-') options for
a specific package, takes precedence over PKG_DEFAULT_OPTIONS
and PKG_SUGGESTED_OPTIONS
Also simplify conversion of legacy USE_XXX variables to options
via the new PKG_OPTIONS_LEGACY_VARS variable and emit warnings if
legacy variables are in use.
Add a new target describe-options that lists all supported options
with a one-line description, the options enabled by default and
the currently enabled optoions.
Reviewed by jlam and wiz.
XXX: defaults/options.description has to be filled in
XXX: the pkgsrc guide has to be updated
vulnerable packages are uploaded directly into the vulnerable subdir.
While here:
quote variables
better handling of the temporary directory
remove some backwards compatibility code that's been here long enough
opsys-specific package handling was doing the same as non-opsys
specific, so fold them together.
Written together with dillo.
