packages that use a sub-directory for prefix, like
/usr/pkg/qt3/.
(I have been using this for a couple years but recently
merged my patch over to new flavor mk file.)
Okayed by jlam on tech-pkg.
_WRKDIR_LOCKFILE and never _PREFIX_LOCKFILE, so use that to keep things
simple.
Replaced ECHO with ECHO_MSG in the message for creating the WRKDIR
symlink.
WRKDIR against foreign processes. Instead, the PKG_DBDIR is locked,
preventing other (pkgsrc) processes from modifying the set of installed
packages while another one is doing that.
This allows to use "bmake update" efficiently on multi-processor
machines by just running it in four different package directories.
Note that the pkg_add and pkg_delete tools may still interfere with this
locking when they are directly run.
have been empty to qualify as a comment. Otherwise "make help
TOPIC=configure" shows a lot of misleading contents.
Removed the duplicate appearance of RCS Ids in the help texts, which
often occured when the help text was the header comment of a file.
word after the leading "#" for the comment. This format is also
recognized. The RCS Id of the current file is printed together with the
help text, so that the user can see which file defines the variable.
Removed the :M* operator for _HELP_AWK, since bmake has a bug when "" is
passed to the :M* operator. I've sent a bug report for this.
Since more and more files follow the convention of documenting variables
in a common format, it makes sense to be able to get the documentation
of a single variable by simple means, instead of searching through the
whole mk/ directory.
Try "make help TOPIC=CONFIGURE_DIRS" for an example.
While here, ...
- Added stricter checking by using "set -eu".
- The bulk build configuration file is properly included, and the
MAKECONF definition that it may contain is properly exported.
- All progress messages and error messages are prefixed by "upload>",
so that it is obvious where the messages come from.
- Since extracting the make(1) variables takes quite a long time, print
an informational message before doing that.
- Removed the use of the error-prone lintpkgsrc to detect whether a
package is restricted or vulnerable.
- If an error occurs, the upload program returns an exitcode of 1,
which is common among Unix utilities.
- Removed almost all pipe operators, since they tend to hide program
failures.
- All error messages are redirected to stderr instead of stdout.
- add a -n|--no-upload flag which does everything but actually executing
the upload to help see what would happen
- add a -d|--debug flag to preserve the temporary files to help with
debugging
- add a -V|--version flag
http://mail-index.netbsd.org/tech-pkg/2006/07/30/0005.html
2. Too much is logged in the .wrapper.log
The wrappers log quite a lot of detailed information by
default, with the result that my .work.log file is 176236028
bytes {for ghc}.
The wrapper functionality is mature, so this detailed
information is seldom needed, and you can in general determine
what happened by looking at the input/output-lines (i.e.
those lines marked as "[*]" and "<.>" in the .work.log).
I would therefore like to change the default behavior to
only log the input/output-lines.
Flip the switch for WRAPPER_DEBUG from "yes" to "no" by default so
that the detailed and rather verbose "push", "pop", "transform", etc.
messages are not printed. While here, also make WRAPPER_DEBUG a
pkgsrc-private variable by prepending with an underscore.
show-all-depends-dirs
show-all-depends-dirs-excl
Replace them with calls to pkgsrc/mk/scripts/depends-depth-first.awk
which does the same thing but without the extra overhead of invoking
a recursive make.
environment ${PKGSRC_MAKE_ENV} is also passed along. Create a
convenience variable RECURSIVE_MAKE that does exactly this and that
can be used in place of MAKE when invoking make recursively.
Use RECURSIVE_MAKE everywhere in pkgsrc/mk that we invoke make
recursively.
If ${FILESDIR}/getsite.sh exists, then use it to determine the fetch
URL for each of the distfiles for the package. Otherwise, use
SITE_<file> and MASTER_SITES, in order, to determine the URL for each
distfile.
If the script path differs from ${FILESDIR}/getsite.sh, then set
DYNAMIC_SITE_SCRIPT to the full path to that script.
Remove the need to set DYNAMIC_MASTER_SITES explicitly in the package
Makefile for:
graphics/ns-cult3d
wm/sawfish-themes
www/apache-tomcat55
www/jakarta-tomcat4
www/jakarta-tomcat5
compiler toolchain until the configure stage, move the "wrapper" stage
back to between "patch" and "configure".
This makes "make extract NO_DEPENDS=" work again for all packages.
that does not exist, it stopped.
Now it instead adds an entry to the main broken file and marks
it as not package with maintainer "directory_does_not_exist"
in the HTML output. (Not packaged directories are not mentioned
in the text version.)
when passing through the barrier. This ensures the PATH (passed via
PKGSRC_MAKE_ENV) is correctly set for all phases after the barrier.
This fixes a bug in "interactive" pkgsrc use, where if you have no
work directory and type "make build && make install", then the "install"
step does not have a PATH set to include all the wrapper and tools
directories.
to all internal recursive make processes. As such, rename it to
"PKGSRC_MAKE_ENV".
XXX Note, some of the usage of this variable in package Makefiles seems
XXX incorrect. They probably want "MAKE_ENV", which is the environment
XXX passed to the make process when running "make" within ${WRKSRC}.
-- instead, we just check "pkg_info", which should exist on all pkgsrc
platforms.
XXX Note that this may need to change when we later support other package
XXX system flavors.
Back out revision 1.79 of pkgtools/x11-links/Makefile which was only to
work around needing a C compiler for shlib-type to work.
it for plurals support, but that is already handled correctly (FSVO
"correctly") by the pkgsrc/mk/tools/msgfmt.sh script.
Also remove _USE_GNU_GETTEXT definitions from pkgsrc/mk/platform/*.mk
files as that value has been unused by pkgsrc for quite some time
(going back several branches).
package said USE_LANGUAGES=#none or USE_LANGUAGES=fortran. Added a
c-fail-wrapper that works like the other fail-wrappers. The default
value for USE_LANGUAGES is still "c". Some problems are expected with
packages that say USE_LANGUAGES+=c++ or with packages containing GNU
configure scripts and setting USE_LANGUAGES=c++, as those scripts always
need a working C compiler.
symlink in the pkgsrc tree in case it's actually a stale work directory.
This gets rid of this error:
===> Cleaning for pkg_install-20060701
rm: work: is a directory
*** Error code 1 (ignored)
currently exist. This triggers a bug in the Bourne shell of at least
DragonFly and Irix, when set -e is also active. Remove set -e to avoid
this. Discussed with jlam@, shown by DragonFly bulk builds, complains on
DragonFly's user list and PR 34036.
A test case for this shell bug is, which should return 0:
set -e
testf () {
test -f /nonexistent || return 1
return 0
}
if testf; then
exit 1
fi
exit 0
had actually been ignoring LTCONFIG_OVERRIDE anyway and just using
the default LIBTOOL_OVERRIDE to replace libtool scripts in packages.
This just formalizes the fact that LTCONFIG_OVERRIDE is not used
meaningfully by pkgsrc.
the old bsd.pkg.mk behavior (noted by adrianp in private email). Keep
"makedistinfo" as an alias for "distinfo".
While here, sprinkle some .PHONY declarations for correctness.
in do-fetch-file (formerly they were inserted by the old _FETCH_FILE
inserting some quotes that are no longer added by the fetch command list
macro). This makes fetching with MASTER_SORT* set work again.
* All the smarts is now encapsulated in the "fetch" script. The fetch
script understands how to use the distinfo file (if specified) to
look up the size and checksums of the file to fetch and will use
that information to verify checksums of the fetched files or resume
transfers of interrupted fetches.
* Move the default settings for FETCH_RESUME_ARGS and FETCH_OUTPUT_ARGS
for "ftp" from mk/defaults/mk.conf into mk/fetch/fetch.mk. We rewrite
it to avoid needing conditional statements.
* Avoid spawning a new make(1) process just to mirror a distfile.
* Split out fetch-list targets into a separate file fetch-list.mk.
These targets should probably be moved into a standalone script.
* Fix distclean target to properly remove partial downloads.
fetch.mk. This script currently completely replaces the functionality
in _FETCH_FILE. I will eventually add the ability to resume a file
transfer to this script.
"fetch" target, but it does affect the "checksum" target's shell
script, which errors out on the second occurrence of a file. The
shell script should perhaps also be fixed, but it seems sensible
regardless for ${ALLFILES} not to contain duplicate filenames. As
a side effect, the file list is sorted.
Regression found by building mail/qmail with the "qmail-netqmail"
option, which adds to PATCHFILES a file that's already in ${DISTFILES}.
Arguably this is gross, but it worked before, and now works again.
Tested on my usual pkg_comp(8) build of 200+ packages, with an
initially empty ${DISTDIR} and ${PACKAGES}. Thanks seb@ for the
more idiomatic make(1) construction.
and those that are defined by the infrastructure (_BUILD_DEFS). This
allows the build-defs-message target to be moved to the end of
bsd.pkg.mk. Now it prints the correct result even in unprivileged
builds, which had been wrong due to the order in which the files have
been included. For example, ${UNPRIVILEGED_USER} was displayed as (not
defined) although its value was defined, which could be checked with
"bmake show-var".
Tested with one package that _does_ define BUILD_DEFS and with one that
doesn't. The behavior stays the same.
replaced automatically by defining REPLACE_AWK, REPLACE_BASH or
REPLACE_SH and an appropriate addition to USE_TOOLS.
The replace-interpreter.mk file is included unconditionally because the
number of variables that would trigger the inclusion of this file is
quite large.
This fixes problems where a package sets PKG_*_REASON, which causes
bsd.pkg.mk to define its own "checksum" replacement, which causes a
"duplicate script" make error to occur.
* Avoid shell differences between /bin/sh and Korn shell by using:
while read line; do list; done < FILE
instead of
cat FILE | while read line; do list; done
stages, and that installs dependencies listed in BOOTSTRAP_DEPENDS.
The bootstrap-depends step works just like the normal depends step
and honors the value of DEPENDS_TARGET. It's now possible to add
dependencies solely to facilitate fetching the distfiles, e.g.
BOOTSTRAP_DEPENDS+= curl-[0-9]*:../../www/curl
* Teach the tools framework about ":bootstrap" as a tools modifier
which indicates the tool should be added as a dependency via
BOOTSTRAP_DEPENDS.
* Add "digest" to the tools framework.
* Use USE_TOOLS+=digest:bootstrap to force pkgsrc to install digest
before anything else. Get rid of unused "uptodate-digest" target
and related digest version-checking code.
* Finish the refactoring work: split checksum-related code out of
bsd.pkg.mk and into pkgsrc/mk/checksum and replace the "checksum"
target command list with a script that does all the real work.
* Make DIGEST_ALGORITHMS and PATCH_DIGEST_ALGORITHM into private
variables by prepending them with an underscore. Also, rename
_PATCH_DIGEST_ALGORITHM to _PATCH_DIGEST_ALGORITHMS and adjust the
makepatchsum target to allow that variable to contain a list of
algorithms, all of which are used when creating the patch checksums
for ${DISTINFO_FILE}.
allow IMAKE to be set by anything other than the tools framework.
Modify the IRIX files so that the native imake is listed as a built-in
tool in the case where X11_TYPE is "native". Also, move the include
of tools/default.mk a bit lower in bsd.prefs.mk so that tools.${OPSYS}.mk
files can use the value of X11_TYPE. This should properly set and
point IMAKE to the right binary on IRIX without destroying the
configuration for platforms where IMAKE was not explicitly set, i.e.
every non-IRIX platform.
USE_TOOLS+=perl was necessary. Therefore, added a new class of tools,
TOOLS_FAIL, which records the call in a .warnings file, which is later
printed to the user. At least when the tool is first called in the
"configure" phase; I didn't test other phases.
and add a new helper target and script, "show-buildlink3", that outputs
a listing of the buildlink3.mk files included as well as the depth at
which they are included.
For example, "make show-buildlink3" in fonts/Xft2 displays:
zlib
fontconfig
iconv
zlib
freetype2
expat
freetype2
Xrender
renderproto
Dar is built by default with an arbitrary-size-integer library for managing
all file length/timestamp details. If 32-bit or 64-bit integers (with
overflow protection) are sufficient for requirements, the dar-int32 and
dar-int64 options can significantly reduce the run-time memory and CPU
overheads of Dar.
semantics in pkgsrc. Because libtool-override is run by default
whenever USE_LIBTOOL is specified, LIBTOOL_OVERRIDE never needs to be
defined, and some packages set it to nothing to avoid running
libtool-override. However, shlibtool-override is only run if
SHLIBTOOL_OVERRIDE is defined and non-empty.
Split the code for libtool-override and shlibtool-override to reflect
these differing semantics. This should make the PHP packages build
again by not overriding libtool.
overwritten in the case where LTCONFIG_OVERRIDE was defined.
As a side note, after analyzing the way that the original code in
bsd.pkg.mk worked, I think we can nuke LTCONFIG_OVERRIDE completely,
but we'll need a bulk build to verify this. The original code always
replaced the libtool scripts because LIBTOOL_OVERRIDE is always defined
in bsd.pkg.use.mk, so LTCONFIG_OVERRIDE essentially had no effect.
target in that it installs the currently-built software into the
filesystem. In that case where "replace" is specified as a target on
the command line, make "replace" and not "install" be the source target
for "package".
Also, place the "replace" target between the "install" and "package"
targets in _BARRIER_POST_TARGETS as it should be legal to do those
steps in that order (but not in another order).
These changes make the following work:
make replace package
In this example, the currently installed package will be replaced and
the newly-installed software will be packaged, all within the same
make process.
move .MAIN all the way to the top of the file. bsd.pkg.barrier.mk
(currently) needs to be included before bsd.wrapper.mk since it defines
_BARRIER_COOKIE, which is expanded and used in place within bsd.wrapper.mk.
This makes the "wrapper" phase run again.
bsd.pkg.barrier.mk uses the "make()" test expression. Also, include
"all" as a post-barrier target since it is implicitly the ".MAIN" target
when a user just types "make" in a package directory.
introducing the concept of a "barrier". We separate the user-invokable
targets into ones that must happen before the barrier, and ones that
must happen after the barrier. The ones that happen after the barrier
are run in a sub-make process. In this case, the targets that must
be run after the barrier are from the "wrapper" step and beyond. We
rewrite the various "flow" targets, e.g. wrapper, configure, build,
etc., so that they of the right form to use the barrier target.
This now completely removes the concept of PKG_PHASE from pkgsrc. It
is replaced with the concept of "before" and "after" the barrier, and
this state can be checked by testing for the existence of the barrier
cookie file. Because we've removed most of the recursive makes, there
is now nowhere to hook the PKG_ERROR_HANDLER.* commands, so remove
them for now.
As part of this commit, put back the logic that conditionalized the
sources for the various cookie files. Because the sources are all
"phony" targets, they were always run, regardless of whether or not
the cookie file already existed. Now, if a cookie file exists, then
that entire phase associated with that cookie file is skipped.
Lastly, fix a thinko in configure/bsd.configure.mk where setting
NO_CONFIGURE in a package Makefile would manage to skip the "wrapper"
step altogether. Fix this by correctly noting "wrapper" and not
"patch" as the preceding step to "configure".
of the logic from fetch/fetch.mk into flavor/pkg/check.mk, so that
check-vulnerable can be used as a source target.
Make check-vulnerable a source target for every phase of the build
workflow, which ensures that it is always run if the user starts a
new phase from the command line.
Fix the cookie-generation targets so that they don't append, only
overwrite to the cookie file. This works around potential problems
due to recursive makes.
Move the cookie checks so that they surround the corresponding phase
target. The presence of the cookie should now inform the make process
to avoid doing any processing of phases that occur before the phase
corresponding to the cookie.
and into their own directories. Also do some cleanups with build/_build
and pkginstall -- we get rid of _build and simply run pkginstall as
part of the "build" target.
Introduce a new mechanism to handle varying directory depths under
${WRKSRC} in which we find files to override, e.g. configure, config.*,
libtool, etc. OVERRIDE_DIRDEPTH is a package-settable variable that
specifies how far under ${WRKSRC} the various targets should look,
and it defaults to "2". We preserve the
meaning of the various *_OVERRIDE variables, so if they are defined,
then their values supersede the OVERRIDE_DIRDEPTH mechanism.
devel/tla will need to specially set OVERRIDE_DIRDEPTH to 3 (see log
for revision 1.1857 for bsd.pkg.mk -- to be done in a separate commit.
itools, intltool, diff3, sdiff, msgmerge
* Adding USE_TOOLS+=itools to a package Makefile will cause the
tool-directory versions of imake, makedepend, mkdirhier and xmkmf
to point to the ones from the devel/nbitools package.
This change will remove the need for nbitools/buildlink3.mk, which
currently does a bit of hackery to force the "right" imake tools to
be used by packages that need it.
* Adding USE_TOOLS+=intltool to a package Makefile will cause the
local versions of intltool-* inside ${WRKSRC} to be replaced by
copies from the textproc/intltool package. If "intltool" is not
specified as a tool, then we create "broken" intltool-* tools in
the tools directory to help highlight hidden dependencies on the
intltool package.
In addition, modify the tools framework so that if "perl" is not
specified as a tool, then we create a "broken" perl tool in the
tools directory for the same reason as for "intltool".
These two changes together will remove the need for
intltools/buildlink3.mk and should also catch all cases where the
sources' intltools may have been silently used because perl was
found on the system.
* Adding USE_TOOLS+=diff3, USE_TOOLS+=sdiff, or USE_TOOLS+=msgmerge
to a package Makefile will cause the corresponding tool to be pulled
into the tools directory.
These are convenience tools to help simplify dependencies for some
packages.
so it can be overridden on the command line. This allows to check for
example ${WRKSRC} instead of the installed files. Of course, since the
variable starts with an underscore, this feature is not meant to be
official.
dollar character before the opening parenthesis. This should really have
been detected by bmake, but somehow the parser seems to be tolerant with
respect to syntax errors.
the cache files left by the README.html generation. This is indended
to be used to monitor a ftp server, not for generating a file list for
uploading to a ftp server.
9.1 as legacy option.
SUSE 10.0 supports more architectures (i386, powerpc, x86_64) and is
already required for some packages (e.g., acroread7). It will help to
get more testing so we can phase out 9.1 before the next stable branch.
see \c, where c is anything but a legal character as defined by
msgfmt-0.10.35, then replace the backslash with a '?'. Yes, this is
a hack, but it works around a bug in the way that older msgfmt
mis-identifies some "control" sequences. This fixes building of
zh_TW.po in x11/matchbox-panel as noted in the bulk build results:
http://mail-index.netbsd.org/pkgsrc-bulk/2006/06/20/0000.html
While here, note in a header comment which packages' *.po files to
use for regression tests whenever changes to this file are made.
PKG_FAIL_REASON in that case. It didn't have an effect anyway for normal
builds, since subst.mk is included after checking PKG_FAIL_REASON.
Discussed with jlam.
some extra processing to ensure that we have a list of unique directories.
Otherwise we end up with two problems:
- cache files get rebuilt all the time because they get built once for each
path to the directory in question and since the path ends up in the cache,
it is always declared out of date.
- we end up with multiple links to the same binary package in the README.html
files.
Committed during the freeze becuase this is a real bug which is encountered
daily.
processing. Allow for this possiblity by falling through both names
when given input files. This fixes the builds of sysutils/dfuibe_installer
and sysutils/dfuife_curses noted in the bulk build results:
http://mail-index.netbsd.org/pkgsrc-bulk/2006/06/20/0000.html
evaluated. Now the SUBST_MESSAGE is only printed once when the
substitution is actually done. Before this change it had been printed
also when the subst-<class> target had been invoked a second time, but
the substitution didn't take place again, which had confused me. Also,
converted the code to use ${WARNING_MSG} and ${STEP_MSG}.
lossage when building security/openpam). Utilize a tools cookie file
to ensure that the post-tools target is only ever run once to avoid
tricky coding requirements for the post-tools target. Also document
some more of the targets.
the pkglint warning:
As {INSTALL,DEINSTALL}_TEMPLATE is modified using "+=", its name
should indicate plural.
This does make the variables a bit more suggestive of the fact that they
hold lists of values.
standard output. These are error or warning messages, so they shouldn't
be "seen" by anything expecting the output of a "make" command to make
sense. This addresses PR pkg/32239 by following the suggestions by
Roland Illig.
if PKG_{FAIL,SKIP}_REASON is set. This fixes the behavior when one
invokes "make build" in a package that sets a fail or skip reason to
stop as soon as the reason is printed.
error and warning messages that are picked up by the error-check
target. Use them instead of using a bare ${ECHO} for more code clarity.
Implemented as suggested by Roland Illig.
which are invoked in recursive make calls for the "install" and
"package" targets respectvely. These recursive make calls prevent
the top-level make from seeing all of the targets and computing a full
dependency graph, so it becomes possible for some targets to be invoked
more than once. This change passes enough information along to the
recursive make calls and ensures that the source targets for the real-*
targets are only invoked once.
to install dependencies looked roughly like this:
${CAT} ${_DEPENDS_FILE} |
while read type pattern dir; do
cd $$dir && ${MAKE} install
done
In the code above, tghe recursive make invoked to install each dependency
does a just-in-time su to acquire root privileges for the installation,
but the su tries to get terminal settings for standard input (from
the pipe) using tcgetattr(), which fails and subsequently causes su
to exit with a puzzling "conversation failure" error. Rewrite the
loop to look (roughly) like this:
set -- `${CAT} ${_DEPENDS_FILE}`
while test $# -gt 0; do
type=$1; pattern=$2; dir=$3; shift 3
cd $$dir && ${MAKE} install
done
Note that this is potentially bad for shells with very low limits on
the maximum command line length, but at least this preserves file
descriptor 1 to reference the controlling tty unless the user does
something weird with input redirection when invoking make.
author has presumably given us valid *.po files, so skip performing
validity checks on the *.po file. This fixes building software where
the author has actually *not* produced proper *.po files, e.g.
net/gtk-gnutella, where the de.po files have msgid/msgstr pairs that
do not have matching numbers of format specifiers (%[a-z]).
