tools listed in USE_TOOLS -- some of them are required by the pkgsrc
infrastructure in variable assignment statements that look like:
VARIABLE!= ${AWK} ...
These tools are actually *required* by pkgsrc to be installed on the
system before it can even work (bootstrap situation). For these tools,
only override the "TOOL" name representing the tool if we're really
using the pkgsrc version of the tool.
We accomplish this by adding a new :pkgsrc modifier that is appended
to these tools listed in USE_TOOLS. We also list these tools in
bsd.prefs.mk so that all packages pick them up fairly early on.
that a package needs. Tools that pkgsrc needs are listed in
PKGSRC_USE_TOOLS, and tools that a package needs on top of that are
listed in USE_TOOLS.
Define "TOOL" variables, e.g. SED, AWK, MKDIR, etc. for each of the
tools that pkgsrc needs, and "TOOLS_TOOL" variables, e.g. TOOLS_SED,
TOOLS_AWK, TOOLS_MKDIR, etc. for each of the tools that a package
needs. These variables contain the full command line to the real
command and arguments needed to invoke the tool.
caches variable definitions that were computed by make. These variables
are specified by listing them in MAKE_VARS, e.g.,
.if !defined(FOO)
FOO!= very_time_consuming_command
.endif
MAKE_VARS+= FOO
bsd.pkg.mk will include only the one generated during the most recent
phase. A particular phase's makevars.mk file consists of variable
definitions that are a superset of all of the ones produced in previous
phases of the build.
The caching is useful because bsd.pkg.mk invokes make recursively,
which in the example above has the potential to run the very time-consuming
command each time unless we cause FOO to be defined for the sub-make
processes. We don't cache via MAKE_FLAGS because MAKE_FLAGS isn't
consistently applied to every invocation of make, and also because
MAKE_FLAGS can overflow the maximum length of a make variable very
quickly if we add many values to it.
One important and desirable property of variables cached via MAKE_VARS
is that they only apply to the current package, and not to any
dependencies whose builds may have been triggered by the current
package.
The makevars.mk files are generated by new targets fetch-vars,
extract-vars, patch-vars, etc., and these targets are built during
the corresponding real-* target to ensure that they are being invoked
with PKG_PHASE set to the proper value.
Also, remove the variables cache file that bsd.wrapper.mk was generating
since the new makevars.mk files provide the same functionality at a
higher level. Change all WRAPPER_VARS definitions that were used by
the old wrapper-phase cache file into MAKE_VARS definitions.
* Get rid of an explicit check for ${_IMAKE_MAKE} == ${GMAKE} in
bsd.pkg.mk to check for whether we need to depend on gmake or not.
Instead, we now note in Linux.mk that packages that need imake will
also need to use gmake by setting _IMAKE_TOOLS+=gmake.
* Push the definition of MAKE_PROGRAM from bsd.pkg.mk into make.mk where
it's closer to related code.
to provide "TOOL" definitions for tools used by a top-level make process
(usually because it uses them in a != variable definition). This allows
USE_TOOLS to be defined before bsd.prefs.mk is included by a package
Makefile, where USE_TOOLS lists the additional (non-default) tools that
are required to build the package.
Also, drop the fallback to existing "TOOL" definitions because we now
have TOOLS_PLATFORM.* for each platform in pkgsr/mk/tools/tools.*.mk.
to tech-pkg:
=====
* USE_BUILDLINK3=YES will be unconditional. (In fact, USE_BUILDLINK3 will
be ignored altogether by mk/; but see below.)
* NO_BUILDLINK and NO_WRAPPER will be ignored by mk/. If a build happens,
these phases will happen.
* The existing NO_BUILD will imply the previous NO_BUILDLINK and NO_WRAPPER.
If no build happens, those phases are not needed.
* NO_TOOLS will be ignored by mk/. The tools phase, which provides much
more than just the C compiler, will always happen regardless of package.
This will make metapackage builds only slightly slower, in trade for far
less user error.
(...and if it were optional, it should have been an .sinclude anyway.)
Sanity: If mk/platform/${OPSYS}.mk is missing, don't assume NetBSD is it.
pkgsrc now depends on a valid platform file for an OS, so require it.
(Still includes NetBSD.mk, but sets PKG_FAIL_REASON.)
spot that will come before compiler.mk (in bsd.prefs.mk). Previously,
LOCALBASE/bin was appearing earlier in the path than work/.<compiler>/bin,
which could cause the Wrong Thing to happen.
decides to set PKGSRCDIR to a relative path as seen in several old PRs
and which prompted the original switch to make PKGSRCDIR private in
revision 1.881 of bsd.pkg.mk.
as it's only used internally by bsd.prefs.mk.
* Make _PKGSRCDIR a public variable by renaming it to PKGSRCDIR.
Also, generate its value from ${_PKGSRC_TOPDIR} so it's less fragile
than the old method of stripping off the last two components of
${.CURDIR}. PKGSRCDIR may now be used after bsd.prefs.mk is defined.
* Change all references to _PKGSRCDIR to PKGSRCDIR.
(1) defs.${OPSYS}.mk --> platform/${OPSYS}.mk.
The "platform" subdirectory is where all of the ${OPSYS}-specific
infrastructure logic should reside.
(2) bsd.pkg.defaults.mk --> defaults/mk.conf
bsd.pkg.obsolete.mk --> defaults/obsolete.mk
Renaming bsd.pkg.defaults.mk to defaults/mk.conf is to mimic the way
that NetBSD has /etc/rc.conf as well as /etc/defaults/rc.conf, where
the latter is a full list of user-settable variables, and the two
files share the same name to reinforce the fact /etc/defaults/rc.conf
can be directly copied in place as /etc/rc.conf. This is the same
relationship shared by defaults/mk.conf and /etc/mk.conf.
included by bsd.prefs.mk. This allows the following variables to be used
before bsd.wrapper.mk is included:
WRAPPER_DIR WRAPPER_SRCDIR
WRAPPER_BINDIR WRAPPER_SHELL
WRAPPER_TMPDIR
the non-buildlink-related code and moves it out of mk/buildlink3 into
mk/wrapper. The buildlink3 code is modified to simply hook its
transformations into the wrapper script framework.
The wrapper script framework has some new features:
* Support automatically passing "ABI" flags to the compiler and linker
depending on the value of ${ABI}. Currently supports the SunPro
compiler with ${ABI} == 64 and the MIPSPro compiler with ${ABI} as
any of 32, n32, o32, and 64.
* making UnixWare GCC accept -rpath options and silently converting
them into an appropriate LD_RUN_PATH
* Add cmd-sink-interix-gcc and cmd-sink-interix-ld that errors out
when it sees -fpic/-fPIC and -shared/-Bshareable, respectively
(requested by <tv>).
* Much improved debugging output. It's possible to output the wrapper
work log in-line with normal output by setting WRAPPER_LOG to
"stderr".
Important differences in behaviour from the old buildlink3 code include:
* Only move the -l options to the end of the command line, leaving the
-L options in-place.
* Extend the autodetection of the libtool mode to detect "compile" and
"uninstall".
* Fix problem noted in both PR pkg/24760 and PR pkg/25500, where
-L/usr/lib/* was being mangled improperly.
* Remove the top-level "buildlink" target; instead, make buildlinking
occur as part of the "wrapper" target.
* mangle and sub-mangle are only meant to transform directories in
-I, -L, and rpath options, so remove the lines in
buildlink3/gen-transform.sh that transformed bare directories.
* Add the ability for the libtool wrapper to be called just to unwrap
an existing libtool archive by running:
libtool --mode=unwrap -o libfoo.la
The old --fix-la syntax no longer works.
20040818
========
* Initial release of a new wrapper script framework that encapsulates
the non-buildlink-related code and moves it out of mk/buildlink3.
These features include:
* making MIPSpro accept GCC options
* making MIPSpro "ucode" accept GCC options
* making SunPro accept GCC options
* making "ld" accept -Wl,* options and silently removing the "-Wl,"
* (NEW) making UnixWare GCC accept -rpath options and silently
converting them into an appropriate LD_RUN_PATH
One major benefit of this is that the buildlink3 code is now much
tighter and easier to understand since it concerns itself solely
with buildlink-related details. I haven't yet optimized the wrapper
cache, so the new wrapper scripts may take slightly longer to execute
than the old buildlink3 wrapper scripts, but I'll be improving this
over time.
20040821
========
* Move the inclusion of $cmd_sink outside of the main loop in wrapper.sh
so that the $cmd_sink script can be used to globally scan and process
the arguments. Move the LD_RUN_PATH code to a cmd-sink-unixware-gcc
script. Garbage-collect the now unused export_vars-related code.
* Add cmd-sink-aix-xlc for AIX xlc that munges -Wl,-R* into an
appropriate -blibpath option.
* Add cmd-sink-interix-gcc and cmd-sink-interix-ld that errors out
when it sees -fpic/-fPIC and -shared/-Bshareable, respectively
(requested by <tv>).
* Move the code that converts full paths to shared libraries into the
"-Ldir -llib" equivalents from the buildlink3 code into wrapper/logic.
Remove the same from bsd.buildlink3.mk and gen-transform.sh.
* Move the code that checks for absolute rpaths from the buildlink3
code into wrapper/arg-source. Remove the same from bsd.buildlink3.mk
and gen-transform.sh.
* Only move the -l options to the end of the command line, leaving the
-L options in-place.
* Add more debugging code.
20040824
========
* Fix quoting problems after arguments are transformed. Remove the
hack that was inserted that magically made almost everything work
because we do it the right way now.
* Move the inclusion of $logic outside of the main loop in wrapper.sh
so that the $logic script doesn't have to worry about underflowing
the argument buffer.
* Encapsulate the loop in wrapper.sh that fills the argument buffer
entirely within the arg-source script.
* Move from the logic script into the arg-source script the
transformations that merge or split arguments.
* Fix bug where skipargs was effectively being ignored if it was more
than 1.
* Handle the whitespace in transformations in the logic script that
turn one library option into multiple library options, e.g.
"-lreadline" -> "-ledit -ltermcap".
* Allow you to specify an environment variable WRAPPER_SKIP_TRANSFORM
for whether you wish to skip the transformation step in the logic
script. This is intended for testing purposes.
* Added check_prog() and init_lib() functions to the shell code library
to make it more reusable outside of the wrapper framework.
* Allow the msg_log() function to output to "stdout" or "stderr". If
you want to have all of the logging appear on the screen, then you
can now set WRAPPER_LOG=stderr.
* Make some of the script components not overridable on a per-wrapper
basis.
* Add a gen-transform.sh script that generates transformation sedfiles.
The "transform" script is used to transform arguments, while the
"untransform" script is used to unwrap files. Move the no-rpath
logic from buildlink3/gen-transform.sh into wrapper/gen-transform.sh
since it's not buildlink3-specific.
* Check for a non-empty blibpath before adding the option in
cmd-sink-aix-xlc.
* Extend the autodetection of the libtool mode to detect "compile" and
"uninstall".
* Add a cmd-sink-libtool script that doesn't pass linker options to
libtool unless we're in "link" mode.
* Set _USE_RPATH to "yes" for UnixWare so that the wrappers will see the
rpath options and convert them to a LD_RUN_PATH definition.
* Add more debugging code.
20040826
========
* Rewrite buildlink3/gen-transform.sh to produce more precise sed commands.
Drop some unused commands from the mini-language, and add a few more
that are more restrictive in their scope.
* Fix problem where repeated options weren't properly handled by some
of sed commands. It's not enough that they're "global replace",
since some patterns match separator characters before and after each
option. We must repeat those patterns twice to catch all instances
correctly.
* Fix problem noted in both PR pkg/24760 and PR pkg/25500, where
-L/usr/lib/* was being mangled improperly.
* Remove the top-level "buildlink" target; instead, make buildlinking
occur as part of the "wrapper" target.
* Add more debugging code.
20040828
========
* Added a head_queue function to shell-lib that returns the head of the
named queue without popping it off the front of the queue.
* Strip consecutive, repeated library options from the command line when
we read it in the logic script.
* Be more careful about not underflowing the argument buffer.
20040906
========
* shell-lib was moved into pkgsrc/mk/scripts; correct references to that
file in the wrapper code.
* Use opt-sub instead of sub-mangle when protecting -I/usr/include/*
and -L/usr/lib/* from buildlink transformations. This avoids adding
lines that look like "-I-I..." in the transformation sedfiles.
* mangle and sub-mangle are only meant to transform directories in
-I, -L, and rpath options, so remove the lines in
buildlink3/gen-transform.sh that transformed bare directories.
* Fix bug in strip-slashdot where the "." wasn't backquoted and thus
matched all characters instead of only the "." character.
* Change the libtool wrapper to use a modified buildcmd script that
doesn't rearrange any of the arguments. This should fix spurious
problems where libtool doesn't understand how to parse the command
line when the -l options are moved to the end of the argument list.
* Fix bug in the logic script where the $cachearg and $cachedarg
weren't being properly set at all times, which caused the cache to
contain the wrong transformed argument.
20040907
========
* Support automatically passing "ABI" flags to the compiler and linker
depending on the value of ${ABI}. Currently supports the SunPro
compiler with ${ABI} == 64 and the MIPSPro compiler with ${ABI} as
any of 32, n32, o32, and 64.
* Move back the code that splits absolute paths to shared libraries
from arg-source back into logic. This allows us to correctly skip
splitting those paths based on the previous option. Also add a
sanity check that the library name in the split argument doesn't
contain a "/" since shell globs are not as precise as REs.
* Don't transform the path given after --dynamic-linker (used by GNU
ld for ELF linkage).
* Add the ability for the libtool wrapper to be called just to unwrap
an existing libtool archive by running:
libtool --mode=unwrap -o libfoo.la
20040914
========
* Add a loop in libtool-fix-la to ensure that all of the options listed
in the dependency_libs lines of *.lai files are processed. This fixes
a buildlink3 leakage bug.
* Merge the gen-transform.sh scripts between buildlink3 and wrapper and
place them all in wrapper. This makes sense since the commands simply
allow for many types of transformations, which buildlink3 takes
advantage of, but there is nothing inherently buildlink-ish about
those commands.
* Don't directly manipulate SUBST_SED.unwrap. Instead, create the
value of SUBST_SED.unwrap by combining several other variables
(currently just _UNWRAP_SED) to ensure that the correct ordering is
preserved.
* Correct some confusing debugging messages.
which are the full option names used to set rpath directives for the
linker and the compiler, respectively. In places were we are invoking
the linker, use "${LINKER_RPATH_FLAG} <path>", where the space is
inserted in case the flag is a word, e.g. -rpath. The default values
of *_RPATH_FLAG are set by the compiler/*.mk files, depending on the
compiler that you use. They may be overridden on a ${OPSYS}-specific
basis by setting _OPSYS_LINKER_RPATH_FLAG and _OPSYS_COMPILER_RPATH_FLAG,
respectively. Garbage-collect _OPSYS_RPATH_NAME and _COMPILER_LD_FLAG.
for handling per-package build options.
Before including this file, the following variables should be defined:
PKG_OPTIONS_VAR
This is a list of the name of the make(1) variables that
contain the options the user wishes to select. This
variable should be set in a package Makefile. E.g.,
PKG_OPTIONS_VAR= WIBBLE_OPTIONS
or
PKG_OPTIONS_VAR= FOO_OPTIONS BAR_OPTIONS
PKG_SUPPORTED_OPTIONS
This is a list of build options supported by the package.
This variable should be set in a package Makefile. E.g.,
PKG_SUPPORTED_OPTIONS= kerberos ldap ssl
Optionally, the following variables may also be defined:
PKG_DEFAULT_OPTIONS
This is a list the options that should be built into
every package, if that option is supported. This
variable should be set in /etc/mk.conf.
${PKG_OPTIONS_VAR} (the variables named in PKG_OPTIONS_VAR)
These variables list the selected build options and
override any default options given in PKG_DEFAULT_OPTIONS.
If any of the options begin with a '-', then that option
is always removed from the selected build options, e.g.
PKG_DEFAULT_OPTIONS= kerberos ldap sasl
PKG_OPTIONS_VAR= WIBBLE_OPTIONS
WIBBLE_OPTIONS= ${PKG_DEFAULT_OPTIONS} -sasl
# implies PKG_OPTIONS == "kerberos ldap"
or
PKG_OPTIONS_VAR= WIBBLE_OPTIONS
WIBBLE_OPTIONS= kerberos -ldap ldap
# implies PKG_OPTIONS == "kerberos"
This variable should be set in /etc/mk.conf.
PKG_FAIL_UNSUPPORTED_OPTIONS
If this is set to "yes", then the presence of unsupported
options in PKG_OPTIONS.<pkg> (see below) causes the build
to fail. Set this to "no" to silently ignore unsupported
options. Default: "yes".
After including this file, the following variables are defined:
PKG_OPTIONS
This is the list of the selected build options, properly
filtered to remove unsupported and duplicate options.
Example usage:
-------------8<-------------8<-------------8<-------------8<-------------
# Global and legacy options
.if defined(USE_OPENLDAP) || defined(USE_SASL2)
. if !defined(PKG_OPTIONS.wibble)
. if defined(USE_OPENLDAP) && !empty(USE_OPENLDAP:M[yY][eE][sS])
PKG_OPTIONS.wibble+= ldap
. endif
. if defined(USE_SASL2) && !empty(USE_SASL2:M[yY][eE][sS])
PKG_OPTIONS.wibble+= sasl
. endif
. endif
.endif
PKG_OPTIONS_VAR= PKG_OPTIONS.wibble
PKG_SUPPORTED_OPTIONS= ldap sasl
.include "../../mk/bsd.options.mk"
# Package-specific option-handling
###
### LDAP support
###
.if !empty(PKG_OPTIONS:Mldap)
. include "../../databases/openldap/buildlink3.mk"
CONFIGURE_ARGS+= --enable-ldap=${BUILDLINK_PREFIX.openldap}
.endif
###
### SASL authentication
###
.if !empty(PKG_OPTIONS:Msasl)
. include "../../security/cyrus-sasl2/buildlink3.mk"
CONFIGURE_ARGS+= --enable-sasl=${BUILDLINK_PREFIX.sasl}
.endif
-------------8<-------------8<-------------8<-------------8<-------------
number is included in MACHINE_GNU_PLATFORM.
Fixes at least the build of wip/mingw-gcc.
Patch based upon the one provided by Michal Pasternak in PR 23856.
Package Makefiles may now directly include compiler.mk.
* Don't include compiler.mk within bsd.prefs.mk any longer. It was only
included for the purposes of defining CC_VERSION. Packages that want
to test the value of CC_VERSION should now first include
"../../mk/compiler.mk". Any GCC_REQD statements in package Makefiles
should be set before compiler.mk is included.
* Simpllfy pkgsrc/mk/compiler/*.mk files as a result of not needing to
be included indirectly by bsd.prefs.mk. We remove the special handling
associated with detecting whether the file was included from within
bsd.prefs.mk. These files are now much more straightforward to write
and understand.
* G/C the BSD_PREFS_MK stack mechanism as the only users (compiler/*)
no longer need it.
* Ensure that directories are prepended to the PATH only from within
bsd.pkg.mk.
new enough to support them. pkgsrc requires updated pkg_install for
older NetBSD releases and will ask the user to install them as the first
step to using pkgsrc, but the error messages about pkg_info not supporting
the "-K" option were confusing.
