environment ${PKGSRC_MAKE_ENV} is also passed along. Create a
convenience variable RECURSIVE_MAKE that does exactly this and that
can be used in place of MAKE when invoking make recursively.
Use RECURSIVE_MAKE everywhere in pkgsrc/mk that we invoke make
recursively.
stages, and that installs dependencies listed in BOOTSTRAP_DEPENDS.
The bootstrap-depends step works just like the normal depends step
and honors the value of DEPENDS_TARGET. It's now possible to add
dependencies solely to facilitate fetching the distfiles, e.g.
BOOTSTRAP_DEPENDS+= curl-[0-9]*:../../www/curl
* Teach the tools framework about ":bootstrap" as a tools modifier
which indicates the tool should be added as a dependency via
BOOTSTRAP_DEPENDS.
* Add "digest" to the tools framework.
* Use USE_TOOLS+=digest:bootstrap to force pkgsrc to install digest
before anything else. Get rid of unused "uptodate-digest" target
and related digest version-checking code.
* Finish the refactoring work: split checksum-related code out of
bsd.pkg.mk and into pkgsrc/mk/checksum and replace the "checksum"
target command list with a script that does all the real work.
* Make DIGEST_ALGORITHMS and PATCH_DIGEST_ALGORITHM into private
variables by prepending them with an underscore. Also, rename
_PATCH_DIGEST_ALGORITHM to _PATCH_DIGEST_ALGORITHMS and adjust the
makepatchsum target to allow that variable to contain a list of
algorithms, all of which are used when creating the patch checksums
for ${DISTINFO_FILE}.
allow IMAKE to be set by anything other than the tools framework.
Modify the IRIX files so that the native imake is listed as a built-in
tool in the case where X11_TYPE is "native". Also, move the include
of tools/default.mk a bit lower in bsd.prefs.mk so that tools.${OPSYS}.mk
files can use the value of X11_TYPE. This should properly set and
point IMAKE to the right binary on IRIX without destroying the
configuration for platforms where IMAKE was not explicitly set, i.e.
every non-IRIX platform.
bsd.pkg.mk. They didn't actually need to be defined in bsd.prefs.mk,
just somewhere before the "main" bsd.<phase>.mk files were included.
This moves some conditional (?=) definitions back into bsd.pkg.mk so
they won't conflict with any conditional definitions in package
Makefiles.
This should fix the "checksum" problems in lang/php-gd as noted here:
http://mail-index.netbsd.org/pkgsrc-users/2006/06/05/0012.html
where EXTRACT_SUFX had the wrong value due to the order in while *.mk
files were included.
into a new file pkgsrc/mk/tools/create.mk. This leaves bsd.tools.mk
as a file that pulls in all of the other ones. Also move the
tools-related targets from bsd.pkg.mk into bsd.tools.mk.
The tools cookie file has been removed, as well as hooks for
{pre,do,post}-tools. Instead, there is now only a single public target
"tools" which may be invoked. Invoking "tools" will always cause all
of the tools in ${TOOLS_DIR} to be created.
The "tools" step has been moved and is now just after the "depends"
step and before sources are extracted. This is the earliest place
where the "tools" step can be taken, and it allows the created tools
to be used in all steps/phases after it, starting with "extract". As
a consequence, we should just invoke tools by their bare names in
targets, e.g. awk, sed, patch, etc., instead of with the ${VARIABLE}
names, e.g. ${AWK}, ${SED}, ${PATCH}, etc.
pkgsrc/mk. Also get rid of the recursive make for the "patch" target.
This basically merges the "patch" phase into the "tools" phase.
There should eventually be a standalone script that can be used to
verify checksums listed in distinfo that should be used instead of
the roll-your-own code in the do-pkgsrc-patch target.
subdirectories of pkgsrc/mk. Move the following files around for
locality:
pkgsrc/mk/scripts/extract -> pkgsrc/mk/extract/extract
pkgsrc/mk/bsd.sites.mk -> pkgsrc/mk/fetch/sites.mk
Also get rid of the recursive make for the "fetch" and "extract"
targets. This basically merges the "fetch" and "extract" phases into
the "patch" phase.
There is still much more work to do to simplify the fetch code, but
this is a good start.
than pkgsrc's current one. This is an important lead-up to any project
that redesigns the pkg_* tools in that it doesn't tie us to past design
(mis)choices. This commit mostly deals with rearranging code, although
there was a considerable amount of rewriting done in cases where I
thought the code was somewhat messy and was difficult to understand.
The design I chose for supporting multiple package system flavors is
that the various depends, install, package, etc. modules would define
default targets and variables that may be overridden in files from
pkgsrc/mk/flavor/${PKG_FLAVOR}. The default targets would do the
sensible thing of doing nothing, and pkgsrc infrastructure would rely
on the appropriate things to be defined in pkgsrc/mk/flavor to do the
real work. The pkgsrc/mk/flavor directory contains subdirectories
corresponding to each package system flavor that we support. Currently,
I only have "pkg" which represents the current pkgsrc-native package
flavor. I've separated out most of the code where we make assumptions
about the package system flavor, mostly either because we directly
use the pkg_* tools, or we make assumptions about the package meta-data
directory, or we directly manipulate the package meta-data files, and
placed it into pkgsrc/mk/flavor/pkg.
There are several new modules that have been refactored out of bsd.pkg.mk
as part of these changes: check, depends, install, package, and update.
Each of these modules has been slimmed down by rewriting them to avoid
some recursive make calls. I've also religiously documented which
targets are "public" and which are "private" so that users won't rely
on reaching into pkgsrc innards to call a private target.
The "depends" module is a complete overhaul of the way that we handle
dependencies. There is now a separate "depends" phase that occurs
before the "extract" phase where dependencies are installed. This
differs from the old way where dependencies were installed just before
extraction occurred. The reduce-depends.mk file is now replaced by
a script that is invoked only once during the depends phase and is
used to generate a cookie file that holds the full set of reduced
dependencies. It is now possible to type "make depends" in a package
directory and all missing dependencies will be installed.
Future work on this project include:
* Resolve the workflow design in anticipation of future work on
staged installations where "package" conceptually happens before
"install".
* Rewrite the buildlink3 framework to not assume the use of the
pkgsrc pkg_* tools.
* Rewrite the pkginstall framework to provide a standard pkg_*
tool to perform the actions, and allowing a purely declarative
file per package to describe what actions need to be taken at
install or deinstall time.
* Implement support for the SVR4 package flavor. This will be
proof that the appropriate abstractions are in place to allow
using a completely different set of package management tools.
using it in a test to set _MAKE. With this change pkgsrc works on
NetBSD/i386 3.0 to build with an empty environment (env -i sh).
Tested with my ~100 favourite server packages. Does not affect
the case when PATH is already set. To have a per OPSYS default path
the include of platform/${OPSYS}.mk will probably need to be at
the top of bsd.prefs.mk - arguably it should be there already.
There are bound to be assumptions made by some packages which will
be broken by an empty env, but the bulk of pkgsrc and in particular
the infrastructure works fine.
ALLOW_VULNERABLE_PACKAGES settings that applies to all packages, there can
now be per-package lists of allowed vulnerability ids:
ALLOW_VULNERABILITIES.<pkgname>=<space separated list of vulnids>
To avoid duplication of code, audit-packages is now used to do these checks.
It can be skipped altogether by setting:
SKIP_AUDIT_PACKAGES=yes
blocks can override it without running the commands at all.
Move Interix [LOWER_]OS_VERSION speedup hack into bsd.prefs.mk, since it
must happen early at runtime.
While here, speed up the OS_VERSION calculation slightly for OSF1.
tools listed in USE_TOOLS -- some of them are required by the pkgsrc
infrastructure in variable assignment statements that look like:
VARIABLE!= ${AWK} ...
These tools are actually *required* by pkgsrc to be installed on the
system before it can even work (bootstrap situation). For these tools,
only override the "TOOL" name representing the tool if we're really
using the pkgsrc version of the tool.
We accomplish this by adding a new :pkgsrc modifier that is appended
to these tools listed in USE_TOOLS. We also list these tools in
bsd.prefs.mk so that all packages pick them up fairly early on.
that a package needs. Tools that pkgsrc needs are listed in
PKGSRC_USE_TOOLS, and tools that a package needs on top of that are
listed in USE_TOOLS.
Define "TOOL" variables, e.g. SED, AWK, MKDIR, etc. for each of the
tools that pkgsrc needs, and "TOOLS_TOOL" variables, e.g. TOOLS_SED,
TOOLS_AWK, TOOLS_MKDIR, etc. for each of the tools that a package
needs. These variables contain the full command line to the real
command and arguments needed to invoke the tool.
caches variable definitions that were computed by make. These variables
are specified by listing them in MAKE_VARS, e.g.,
.if !defined(FOO)
FOO!= very_time_consuming_command
.endif
MAKE_VARS+= FOO
bsd.pkg.mk will include only the one generated during the most recent
phase. A particular phase's makevars.mk file consists of variable
definitions that are a superset of all of the ones produced in previous
phases of the build.
The caching is useful because bsd.pkg.mk invokes make recursively,
which in the example above has the potential to run the very time-consuming
command each time unless we cause FOO to be defined for the sub-make
processes. We don't cache via MAKE_FLAGS because MAKE_FLAGS isn't
consistently applied to every invocation of make, and also because
MAKE_FLAGS can overflow the maximum length of a make variable very
quickly if we add many values to it.
One important and desirable property of variables cached via MAKE_VARS
is that they only apply to the current package, and not to any
dependencies whose builds may have been triggered by the current
package.
The makevars.mk files are generated by new targets fetch-vars,
extract-vars, patch-vars, etc., and these targets are built during
the corresponding real-* target to ensure that they are being invoked
with PKG_PHASE set to the proper value.
Also, remove the variables cache file that bsd.wrapper.mk was generating
since the new makevars.mk files provide the same functionality at a
higher level. Change all WRAPPER_VARS definitions that were used by
the old wrapper-phase cache file into MAKE_VARS definitions.
* Get rid of an explicit check for ${_IMAKE_MAKE} == ${GMAKE} in
bsd.pkg.mk to check for whether we need to depend on gmake or not.
Instead, we now note in Linux.mk that packages that need imake will
also need to use gmake by setting _IMAKE_TOOLS+=gmake.
* Push the definition of MAKE_PROGRAM from bsd.pkg.mk into make.mk where
it's closer to related code.
to provide "TOOL" definitions for tools used by a top-level make process
(usually because it uses them in a != variable definition). This allows
USE_TOOLS to be defined before bsd.prefs.mk is included by a package
Makefile, where USE_TOOLS lists the additional (non-default) tools that
are required to build the package.
Also, drop the fallback to existing "TOOL" definitions because we now
have TOOLS_PLATFORM.* for each platform in pkgsr/mk/tools/tools.*.mk.
