the wrapped command and exit. This will be used by libtool which needs
to figure out the true command name. Yeah, it is ugly behaviour from
libtool, but I'm not going to rewrite libtool.
Change ok-ed by jlam.
may lead to failure for some packages, a note to the user is printed
when a wrapped command fails, including the "real" command line of the
wrapped command.
This change arose from the implicit addition of the -std=c99, -std=gnu99
or -c99 options, which made some packages fail with obscure error
messages. This change helps users get to the real cause more quickly.
When discarding arguments, split_arg is set to yes, so that no empty
arguments are added to the resulting command line. A more informative
debugging message is also printed in that case.
Added a debugging message to transform_to(), for consistency with the
other functions.
Added some code to check for unknown options.
Reindented the file so that the compiler options are better visible.
They had been hidden by the massive block comments before. Also, every
option has got its own line so that one only needs to read the first
column to find the relevant code.
make the wrapper code less frightening. For example, instead of
arg="${arg#-Wl,}"
$debug_log $wrapperlog " (transform-sunpro-cc) to: $arg"
addtocache=yes
one can simply write
transform_to "${arg-Wl,}"
The functions that are currently implemented are: transform_to,
transform_discard, transform_to_nocache, transform_discard,
transform_pass and transform_pass_unknown.
cache together with MAKE_FLAGS=-j. The cache file is now regenerated
into a temporary file and then renamed (atomically) to the "real" cache
file.
Fixed some code that would fail when run in "set -e" mode. If you want to
test the "set -e" mode, just uncomment the line in wrapper.sh.
command line
$debug_log $arguments
tries to execute the empty command. This results in error messages of
the form
cc[50]: : cannot execute (IRIX)
bash: : command not found (Bash)
: permission denied (NetBSD /bin/sh)
Setting IFS to the original value as soon as possible fixes these bugs.
Fixes PR 34135.
http://mail-index.netbsd.org/tech-pkg/2006/07/30/0005.html
2. Too much is logged in the .wrapper.log
The wrappers log quite a lot of detailed information by
default, with the result that my .work.log file is 176236028
bytes {for ghc}.
The wrapper functionality is mature, so this detailed
information is seldom needed, and you can in general determine
what happened by looking at the input/output-lines (i.e.
those lines marked as "[*]" and "<.>" in the .work.log).
I would therefore like to change the default behavior to
only log the input/output-lines.
Flip the switch for WRAPPER_DEBUG from "yes" to "no" by default so
that the detailed and rather verbose "push", "pop", "transform", etc.
messages are not printed. While here, also make WRAPPER_DEBUG a
pkgsrc-private variable by prepending with an underscore.
compiler toolchain until the configure stage, move the "wrapper" stage
back to between "patch" and "configure".
This makes "make extract NO_DEPENDS=" work again for all packages.
when passing through the barrier. This ensures the PATH (passed via
PKGSRC_MAKE_ENV) is correctly set for all phases after the barrier.
This fixes a bug in "interactive" pkgsrc use, where if you have no
work directory and type "make build && make install", then the "install"
step does not have a PATH set to include all the wrapper and tools
directories.
introducing the concept of a "barrier". We separate the user-invokable
targets into ones that must happen before the barrier, and ones that
must happen after the barrier. The ones that happen after the barrier
are run in a sub-make process. In this case, the targets that must
be run after the barrier are from the "wrapper" step and beyond. We
rewrite the various "flow" targets, e.g. wrapper, configure, build,
etc., so that they of the right form to use the barrier target.
This now completely removes the concept of PKG_PHASE from pkgsrc. It
is replaced with the concept of "before" and "after" the barrier, and
this state can be checked by testing for the existence of the barrier
cookie file. Because we've removed most of the recursive makes, there
is now nowhere to hook the PKG_ERROR_HANDLER.* commands, so remove
them for now.
As part of this commit, put back the logic that conditionalized the
sources for the various cookie files. Because the sources are all
"phony" targets, they were always run, regardless of whether or not
the cookie file already existed. Now, if a cookie file exists, then
that entire phase associated with that cookie file is skipped.
Lastly, fix a thinko in configure/bsd.configure.mk where setting
NO_CONFIGURE in a package Makefile would manage to skip the "wrapper"
step altogether. Fix this by correctly noting "wrapper" and not
"patch" as the preceding step to "configure".
of the logic from fetch/fetch.mk into flavor/pkg/check.mk, so that
check-vulnerable can be used as a source target.
Make check-vulnerable a source target for every phase of the build
workflow, which ensures that it is always run if the user starts a
new phase from the command line.
Fix the cookie-generation targets so that they don't append, only
overwrite to the cookie file. This works around potential problems
due to recursive makes.
Move the cookie checks so that they surround the corresponding phase
target. The presence of the cookie should now inform the make process
to avoid doing any processing of phases that occur before the phase
corresponding to the cookie.
and into their own directories. Also do some cleanups with build/_build
and pkginstall -- we get rid of _build and simply run pkginstall as
part of the "build" target.
Introduce a new mechanism to handle varying directory depths under
${WRKSRC} in which we find files to override, e.g. configure, config.*,
libtool, etc. OVERRIDE_DIRDEPTH is a package-settable variable that
specifies how far under ${WRKSRC} the various targets should look,
and it defaults to "2". We preserve the
meaning of the various *_OVERRIDE variables, so if they are defined,
then their values supersede the OVERRIDE_DIRDEPTH mechanism.
devel/tla will need to specially set OVERRIDE_DIRDEPTH to 3 (see log
for revision 1.1857 for bsd.pkg.mk -- to be done in a separate commit.
itools, intltool, diff3, sdiff, msgmerge
* Adding USE_TOOLS+=itools to a package Makefile will cause the
tool-directory versions of imake, makedepend, mkdirhier and xmkmf
to point to the ones from the devel/nbitools package.
This change will remove the need for nbitools/buildlink3.mk, which
currently does a bit of hackery to force the "right" imake tools to
be used by packages that need it.
* Adding USE_TOOLS+=intltool to a package Makefile will cause the
local versions of intltool-* inside ${WRKSRC} to be replaced by
copies from the textproc/intltool package. If "intltool" is not
specified as a tool, then we create "broken" intltool-* tools in
the tools directory to help highlight hidden dependencies on the
intltool package.
In addition, modify the tools framework so that if "perl" is not
specified as a tool, then we create a "broken" perl tool in the
tools directory for the same reason as for "intltool".
These two changes together will remove the need for
intltools/buildlink3.mk and should also catch all cases where the
sources' intltools may have been silently used because perl was
found on the system.
* Adding USE_TOOLS+=diff3, USE_TOOLS+=sdiff, or USE_TOOLS+=msgmerge
to a package Makefile will cause the corresponding tool to be pulled
into the tools directory.
These are convenience tools to help simplify dependencies for some
packages.
dollar character before the opening parenthesis. This should really have
been detected by bmake, but somehow the parser seems to be tolerant with
respect to syntax errors.
PIC code archive, but libtool isn't smart enough to realize it. Munge
the name within cmd-sink-libtool into something that will make libtool
recognize it as a PIC code archive. We unmunge it in arg-pp-main so
non-libtool wrappers see the right thing. This let's us use libtool
to link applications with an embedded Perl interpreter.
I feel so dirty...
standard C++ runtime libraries.
we need to explicitly do this because ${CXX} only links against the
C++ runtime libraries when building an executable, not a shared
object. this results in executables linked against libraries linked by
${CXX} unable to resolve C++ symbols at runtime.
fixes packages which link shared libraries using ${CXX}, including
graphics/glu, databases/db4, and packages which use shared libraries
provided by these packages.
-Wl,-rpath -Wl,/dir1:/dir2:/dir3
into:
-Wl,-rpath,/dir1 -Wl,-rpath,/dir2 -Wl,-rpath,/dir3
Do the same for -Wl,-R and -Wl,-rpath-link. This makes the wrapper
scripts pass the test in regress/buildlink-transform/rpath-merge.mk
and fixes PR pkg/27702.
variables that use the :sh modifier. This still causes expansion to only
happen when referenced, and has the advantage of being :Q-safe.
Bring back the changes from revision 1.19 of mk/subst.mk now that the
problem noted above has been fixed. This passes the buildlink-unwrap
regression test.
caches variable definitions that were computed by make. These variables
are specified by listing them in MAKE_VARS, e.g.,
.if !defined(FOO)
FOO!= very_time_consuming_command
.endif
MAKE_VARS+= FOO
bsd.pkg.mk will include only the one generated during the most recent
phase. A particular phase's makevars.mk file consists of variable
definitions that are a superset of all of the ones produced in previous
phases of the build.
The caching is useful because bsd.pkg.mk invokes make recursively,
which in the example above has the potential to run the very time-consuming
command each time unless we cause FOO to be defined for the sub-make
processes. We don't cache via MAKE_FLAGS because MAKE_FLAGS isn't
consistently applied to every invocation of make, and also because
MAKE_FLAGS can overflow the maximum length of a make variable very
quickly if we add many values to it.
One important and desirable property of variables cached via MAKE_VARS
is that they only apply to the current package, and not to any
dependencies whose builds may have been triggered by the current
package.
The makevars.mk files are generated by new targets fetch-vars,
extract-vars, patch-vars, etc., and these targets are built during
the corresponding real-* target to ensure that they are being invoked
with PKG_PHASE set to the proper value.
Also, remove the variables cache file that bsd.wrapper.mk was generating
since the new makevars.mk files provide the same functionality at a
higher level. Change all WRAPPER_VARS definitions that were used by
the old wrapper-phase cache file into MAKE_VARS definitions.
to tech-pkg:
=====
* USE_BUILDLINK3=YES will be unconditional. (In fact, USE_BUILDLINK3 will
be ignored altogether by mk/; but see below.)
* NO_BUILDLINK and NO_WRAPPER will be ignored by mk/. If a build happens,
these phases will happen.
* The existing NO_BUILD will imply the previous NO_BUILDLINK and NO_WRAPPER.
If no build happens, those phases are not needed.
* NO_TOOLS will be ignored by mk/. The tools phase, which provides much
more than just the C compiler, will always happen regardless of package.
This will make metapackage builds only slightly slower, in trade for far
less user error.
icc is a high performance compiler suite for Linux/i386 and ia64, and
is free for non-commercial use. see:
http://www.intel.com/software/products/compilers/clin/
for more information.
tested with icc version 8.0 and 8.1 on Slackware 10/i386.
so move the setting for _WRAP_ARG_PP.LIBTOOL out of conditional areas of
bsd.wrapper.mk and set it globally in bsd.buildlink3.mk with the rest of
the libtool wrapper variables.
the compiler wrapper already knows to do. This should protect
"-install_name ..." from wrapper transformations within the libtool
wrapper on Darwin and fixes PR pkg/29215.
Software that properly use libtool don't pass an explicit "-install_name
..." because libtool always generates its own such argument when
building the (Darwin) shared library. However, not every piece of
software, including the ones like lang/tcl that pkgsrc converted to
use libtool, will use libtool in the documented fashion. Longer term,
those packages should be fixed properly, but we will still need the
libtool wrapper to avoid transforming arguments that the compiler
wrapper itself doesn't transform, e.g. for MIPSpro and -LANG:*.
-L/some/where -lfoo so that the arguments we push on the stack are actually
used in logic.
Should solve the issue build bash with option 'static' enabled reported on
netbsd-users.
OK'd by jlam@.
arg-pp-main, and allow for wrapper-specific hooks to extend the
pre-processing through _WRAP_ARG_PP.<wrappee>. Move the Darwin
GCC-related preprocessing into arg-pp-darwin-gcc, and create a
arg-pp-mipspro-cc that understands -LANG:<feature> and -LIST:<feature>
so that they're not confused with the usual -Ldir options.
shell script instead. We can't avoid invoking sed at least once since
we need to protect the arguments against shell expansion, but we avoid
invoking sed for transformations unless there actually are transformations
to perform. The code is simpler, more straightforward, and logically
correct now.
This fixes problems with using a non-GCC compiler with packages that
haven't been converted to use buildlink3, noted in PR pkg/28896.
are handled. The idea now is to simply remove the paths in the values
of these variables, leaving behind only the basename plus any arguments,
e.g.:
CC= /usr/local/bin/gcc becomes CC= gcc
CPP= /usr/local/bin/gcc -E becomes CPP= gcc -E
The wrapper scripts are generated for every unique executable mentioned
by the toolchain variables, so for the example above, only a "gcc"
wrapper script is generated for ${CC} and ${CPP}. PKG_{CC,CPP,CXX,etc.}
are the paths to the executables wrapped by the wrapper scripts.
Note that it's now possible to set "CC" to something more than just the
path to the compiler, e.g.
CC= cc -ffast-math -funroll-loops -fomit-frame-pointer
and the full value of ${CC} will be passed through via CONFIGURE_ENV
and MAKE_ENV.
defaults within the wrapper script. This avoids requiring those two
variables to be defined in the shell environment for the correct defaults
to take effect.
The "untransform" file is used to unwrap *-config files and *.lai files.
The transform file can be much stricter about what command line arguments
should look like, so take advantage of that to fix the "libpath" and
"libpath-in-define" regression tests.
6.0 on Mac OS X 10.3.5.
to use XL C, set PKGSRC_COMPILER=xlc in mk.conf. XLCBASE defaults to
/opt/ibmcmp/vacpp/6.0 (the default installation location on OS X),
this can be overridden in mk.conf too.
this is a work in progress - some simple packages can be built, but
there are still lots of issues that need to be worked through.
the wrappers from adding extra flags to the command-line if "-v"
exists. This makes "g++ -v" return the same error code for both the
wrapper and the real executable and fixes the problem with the
libtool-base build not correctly detecting the C++ compiler and thus
not adding the CXX configuration tag to the final libtool script.
Downgrade LIBTOOL_REQD to 1.5.10 since the newest version isn't needed
with this change.
included by bsd.prefs.mk. This allows the following variables to be used
before bsd.wrapper.mk is included:
WRAPPER_DIR WRAPPER_SRCDIR
WRAPPER_BINDIR WRAPPER_SHELL
WRAPPER_TMPDIR
paths to static archives and libtool archives. This should fix the
problem where a mangled path sometimes makes its way into the the
executed command line.
is intended to be toggled by a $scan file. Simplify buildlink3 by removing
_BLNK_LIBTOOL_LDFLAGS and just setting _WRAP_EXTRA_ARGS.* like all of the
other wrappers.
the non-buildlink-related code and moves it out of mk/buildlink3 into
mk/wrapper. The buildlink3 code is modified to simply hook its
transformations into the wrapper script framework.
The wrapper script framework has some new features:
* Support automatically passing "ABI" flags to the compiler and linker
depending on the value of ${ABI}. Currently supports the SunPro
compiler with ${ABI} == 64 and the MIPSPro compiler with ${ABI} as
any of 32, n32, o32, and 64.
* making UnixWare GCC accept -rpath options and silently converting
them into an appropriate LD_RUN_PATH
* Add cmd-sink-interix-gcc and cmd-sink-interix-ld that errors out
when it sees -fpic/-fPIC and -shared/-Bshareable, respectively
(requested by <tv>).
* Much improved debugging output. It's possible to output the wrapper
work log in-line with normal output by setting WRAPPER_LOG to
"stderr".
Important differences in behaviour from the old buildlink3 code include:
* Only move the -l options to the end of the command line, leaving the
-L options in-place.
* Extend the autodetection of the libtool mode to detect "compile" and
"uninstall".
* Fix problem noted in both PR pkg/24760 and PR pkg/25500, where
-L/usr/lib/* was being mangled improperly.
* Remove the top-level "buildlink" target; instead, make buildlinking
occur as part of the "wrapper" target.
* mangle and sub-mangle are only meant to transform directories in
-I, -L, and rpath options, so remove the lines in
buildlink3/gen-transform.sh that transformed bare directories.
* Add the ability for the libtool wrapper to be called just to unwrap
an existing libtool archive by running:
libtool --mode=unwrap -o libfoo.la
The old --fix-la syntax no longer works.
20040818
========
* Initial release of a new wrapper script framework that encapsulates
the non-buildlink-related code and moves it out of mk/buildlink3.
These features include:
* making MIPSpro accept GCC options
* making MIPSpro "ucode" accept GCC options
* making SunPro accept GCC options
* making "ld" accept -Wl,* options and silently removing the "-Wl,"
* (NEW) making UnixWare GCC accept -rpath options and silently
converting them into an appropriate LD_RUN_PATH
One major benefit of this is that the buildlink3 code is now much
tighter and easier to understand since it concerns itself solely
with buildlink-related details. I haven't yet optimized the wrapper
cache, so the new wrapper scripts may take slightly longer to execute
than the old buildlink3 wrapper scripts, but I'll be improving this
over time.
20040821
========
* Move the inclusion of $cmd_sink outside of the main loop in wrapper.sh
so that the $cmd_sink script can be used to globally scan and process
the arguments. Move the LD_RUN_PATH code to a cmd-sink-unixware-gcc
script. Garbage-collect the now unused export_vars-related code.
* Add cmd-sink-aix-xlc for AIX xlc that munges -Wl,-R* into an
appropriate -blibpath option.
* Add cmd-sink-interix-gcc and cmd-sink-interix-ld that errors out
when it sees -fpic/-fPIC and -shared/-Bshareable, respectively
(requested by <tv>).
* Move the code that converts full paths to shared libraries into the
"-Ldir -llib" equivalents from the buildlink3 code into wrapper/logic.
Remove the same from bsd.buildlink3.mk and gen-transform.sh.
* Move the code that checks for absolute rpaths from the buildlink3
code into wrapper/arg-source. Remove the same from bsd.buildlink3.mk
and gen-transform.sh.
* Only move the -l options to the end of the command line, leaving the
-L options in-place.
* Add more debugging code.
20040824
========
* Fix quoting problems after arguments are transformed. Remove the
hack that was inserted that magically made almost everything work
because we do it the right way now.
* Move the inclusion of $logic outside of the main loop in wrapper.sh
so that the $logic script doesn't have to worry about underflowing
the argument buffer.
* Encapsulate the loop in wrapper.sh that fills the argument buffer
entirely within the arg-source script.
* Move from the logic script into the arg-source script the
transformations that merge or split arguments.
* Fix bug where skipargs was effectively being ignored if it was more
than 1.
* Handle the whitespace in transformations in the logic script that
turn one library option into multiple library options, e.g.
"-lreadline" -> "-ledit -ltermcap".
* Allow you to specify an environment variable WRAPPER_SKIP_TRANSFORM
for whether you wish to skip the transformation step in the logic
script. This is intended for testing purposes.
* Added check_prog() and init_lib() functions to the shell code library
to make it more reusable outside of the wrapper framework.
* Allow the msg_log() function to output to "stdout" or "stderr". If
you want to have all of the logging appear on the screen, then you
can now set WRAPPER_LOG=stderr.
* Make some of the script components not overridable on a per-wrapper
basis.
* Add a gen-transform.sh script that generates transformation sedfiles.
The "transform" script is used to transform arguments, while the
"untransform" script is used to unwrap files. Move the no-rpath
logic from buildlink3/gen-transform.sh into wrapper/gen-transform.sh
since it's not buildlink3-specific.
* Check for a non-empty blibpath before adding the option in
cmd-sink-aix-xlc.
* Extend the autodetection of the libtool mode to detect "compile" and
"uninstall".
* Add a cmd-sink-libtool script that doesn't pass linker options to
libtool unless we're in "link" mode.
* Set _USE_RPATH to "yes" for UnixWare so that the wrappers will see the
rpath options and convert them to a LD_RUN_PATH definition.
* Add more debugging code.
20040826
========
* Rewrite buildlink3/gen-transform.sh to produce more precise sed commands.
Drop some unused commands from the mini-language, and add a few more
that are more restrictive in their scope.
* Fix problem where repeated options weren't properly handled by some
of sed commands. It's not enough that they're "global replace",
since some patterns match separator characters before and after each
option. We must repeat those patterns twice to catch all instances
correctly.
* Fix problem noted in both PR pkg/24760 and PR pkg/25500, where
-L/usr/lib/* was being mangled improperly.
* Remove the top-level "buildlink" target; instead, make buildlinking
occur as part of the "wrapper" target.
* Add more debugging code.
20040828
========
* Added a head_queue function to shell-lib that returns the head of the
named queue without popping it off the front of the queue.
* Strip consecutive, repeated library options from the command line when
we read it in the logic script.
* Be more careful about not underflowing the argument buffer.
20040906
========
* shell-lib was moved into pkgsrc/mk/scripts; correct references to that
file in the wrapper code.
* Use opt-sub instead of sub-mangle when protecting -I/usr/include/*
and -L/usr/lib/* from buildlink transformations. This avoids adding
lines that look like "-I-I..." in the transformation sedfiles.
* mangle and sub-mangle are only meant to transform directories in
-I, -L, and rpath options, so remove the lines in
buildlink3/gen-transform.sh that transformed bare directories.
* Fix bug in strip-slashdot where the "." wasn't backquoted and thus
matched all characters instead of only the "." character.
* Change the libtool wrapper to use a modified buildcmd script that
doesn't rearrange any of the arguments. This should fix spurious
problems where libtool doesn't understand how to parse the command
line when the -l options are moved to the end of the argument list.
* Fix bug in the logic script where the $cachearg and $cachedarg
weren't being properly set at all times, which caused the cache to
contain the wrong transformed argument.
20040907
========
* Support automatically passing "ABI" flags to the compiler and linker
depending on the value of ${ABI}. Currently supports the SunPro
compiler with ${ABI} == 64 and the MIPSPro compiler with ${ABI} as
any of 32, n32, o32, and 64.
* Move back the code that splits absolute paths to shared libraries
from arg-source back into logic. This allows us to correctly skip
splitting those paths based on the previous option. Also add a
sanity check that the library name in the split argument doesn't
contain a "/" since shell globs are not as precise as REs.
* Don't transform the path given after --dynamic-linker (used by GNU
ld for ELF linkage).
* Add the ability for the libtool wrapper to be called just to unwrap
an existing libtool archive by running:
libtool --mode=unwrap -o libfoo.la
20040914
========
* Add a loop in libtool-fix-la to ensure that all of the options listed
in the dependency_libs lines of *.lai files are processed. This fixes
a buildlink3 leakage bug.
* Merge the gen-transform.sh scripts between buildlink3 and wrapper and
place them all in wrapper. This makes sense since the commands simply
allow for many types of transformations, which buildlink3 takes
advantage of, but there is nothing inherently buildlink-ish about
those commands.
* Don't directly manipulate SUBST_SED.unwrap. Instead, create the
value of SUBST_SED.unwrap by combining several other variables
(currently just _UNWRAP_SED) to ensure that the correct ordering is
preserved.
* Correct some confusing debugging messages.
