variable is much more comfortable than specifying all the directories in
INSTALLATION_DIRS that using such a long name for it feels like a
punishment. It also doesn't look nice in the package Makefiles, for
example:
old:
INSTALLATION_DIRS= bin ${PKGMANDIR}/man1 ${PKGMANDIR}/man8
new:
AUTO_MKDIRS= yes
The variable INSTALLATION_DIRS_FROM_PLIST will be deprecated slowly.
twice: First against the required package pattern (PKGNAME_REQD), and
then against the current version (PKGNAME). When only a binary package
for an old (but sufficient) version of a package was available, that
version had been installed and the current version been built
afterwards, which was unnecessary.
This problem was mentioned in PR 36146, and is hereby fixed.
By the way, the code was really ugly: The messages that had been printed
didn't reflect what the code was actually doing. This is fixed, too.
in the definition of INSTALLATION_DIRS, where a short man/ would be much
shorter. Since we already have that man-transforming magic in some other
places of pkgsrc, it's also here.
While here, documented INSTALLATION_DIRS and
INSTALLATION_DIRS_FROM_PLIST so they are found by "bmake help".
- check-perms.mk was not DESTDIR aware, prefix files before passing it
to the directory extraction
- PKG_FILELIST_CMD was calling pkg_info, which is fine for normal
installation, but fails of course for DESTDIR. Just drop the @ lines
from ${_DEPENDS_PLIST} and use that.
- To make he former work, ensure that _flavor-generate-metadata is part
of _INSTALL_ALL_TARGETS. It was normally a dependency of
_flavor-register, but that is skipped for DESTDIR.
- Remove ${_DEPENDS_PLIST} when running install-clean.
contains files that are generated during the "install" phase. This
should fix the problem where PLIST modifications were ignored if the
PLIST was modified between a "make deinstall" and a "make reinstall".
- USE_CROSS_COMPILATION activates it, CROSS_DESTDIR specifies root of
the target filesystem
- derive _CROSS_DESTDIR from CROSS_DESTDIR or MAKEOBJDIR
- buildlink3.mk prefixes the files to symlink with _CROSS_DESTDIR
- compiler/gcc.mk knows about the target prefix (e.g. i386--netbsdelf)
- PKG_DBDIR is prefixed with _CROSS_DESTDIR
- package-install and bin-install are not called with su
- install and strip are redirected to the tool version
- links for the target specific ar, as, ld, nm, objdump, ranlib and
strip are added
- compiler wrapper detect if linking is requested or not
- special command sinks for CPP and CC/CXX add the cross-compile magic:
- modify include dirs to get the target /usr/include
- modify linker dirs and runpath to use target /usr/lib at link time,
but keep correct rpath entries
Supported-by: Google SoC 2007
Basic tests by he@ on Sparc. Review from jlam@.
- Targets that are not intended to be defined here (replace-pkg and
undo-replace-pkg) are not defined here. (Sounds obvious, but apparently
isn't.)
- Targets that are intended to be defined here and only here (replace
and undo-replace) do not need a multiple-definition protection.
- Moved all comments to the top of the file.
install/install.mk to check/check-shlibs.mk.
Renamed check-shlibs to _check-shlibs. Since it had been declared as
"private" before, this will not break anything.
don't use mkdir, as the latter uses the group permissions of the
parent directory under BSDs, resulting in group leaks for
USE_DESTDIR=yes. Discussed with agc@.
need to specify INSTALLATION_DIRS itself. Instead, the list is generated
from the static PLIST files. Like for INSTALLATION_DIRS, mtree is
disabled when this variable is set.
are identical. For DESTDIR builds, the package is not installed to
PREFIX as part of the build, so package-install does exactly that after
package is done. Change bin-install to call package-install.
"make install". That way, packages don't have to say:
INSTALL_MAKE_FLAGS+= ${MAKE_FLAGS} foo=bar,
instead it suffices to say
INSTALL_MAKE_FLAGS+= foo=bar,
which is also more appropriate since multiple files (Makefile,
Makefile.common, options.mk, *.mk) don't need to care about whether
INSTALL_MAKE_FLAGS already includes MAKE_FLAGS or not.
Note: It is no longer possible to override MAKE_FLAGS completely. But
except for some exotic platforms, the default value is empty anyway.
Packages may set PKG_DESTDIR_SUPPORT to either "destdir" or
"user-destdir" to flag support for this, following the same
rules as PKG_INSTALLATION_TYPES (e.g. define before first include
of bsd.prefs.mk).
The user activates it via USE_DESTDIR. When set to "yes",
packages with "user-destdir" are handled as "destdir".
The installation of the package will not go to ${LOCALBASE},
but a subdirectory of ${WRKDIR} instead. pre/post install scripts are
not run and the package is not registered either. A binary package
can be created instead to be installed normally with pkg_add.
For "user-destdir" packages, everything is run as normal user and
ownership is supposed to be correctled by pkg_create later. Since
the current pkg_install code uses pax and it doesn't allow overwriting
owners, this does not work yet.
For "destdir" packages, installation, packaging and cleaning is run as
root.
This commit does not change the handling of DEPENDS_TARGET or
bin-install to allow recursive usage.
INSTALLATION_DIRS but a regular file of the same name already exists,
the BSD install(1) program doesn't care but exits successfully.
To avoid much more confusing error messages, this is checked here, and a
_good_ error message is printed.
recursively. This bug was reported by Hans Rosenfeld in
http://mail-index.netbsd.org/tech-pkg/2006/10/03/0002.html
As a side effect, when bin-install fails to install a binary package,
the package is still built from source, but not as a privileged user.
Added two targets acquire-localbase-lock and release-localbase-lock,
which mark the complete LOCALBASE directory as locked, so that multiple
packages cannot run the install, deinstall or bin-install targets at the
same time.
The install target aquires locks in both WRKSRC and LOCALBASE, the other
two targets only need the LOCALBASE lock, since they may be run without
WRKSRC being present on the system.
locking.mk must be included before tools.mk and the PKG_FAIL_REASON
check.
WRKDIR against foreign processes. Instead, the PKG_DBDIR is locked,
preventing other (pkgsrc) processes from modifying the set of installed
packages while another one is doing that.
This allows to use "bmake update" efficiently on multi-processor
machines by just running it in four different package directories.
Note that the pkg_add and pkg_delete tools may still interfere with this
locking when they are directly run.
environment ${PKGSRC_MAKE_ENV} is also passed along. Create a
convenience variable RECURSIVE_MAKE that does exactly this and that
can be used in place of MAKE when invoking make recursively.
Use RECURSIVE_MAKE everywhere in pkgsrc/mk that we invoke make
recursively.
when passing through the barrier. This ensures the PATH (passed via
PKGSRC_MAKE_ENV) is correctly set for all phases after the barrier.
This fixes a bug in "interactive" pkgsrc use, where if you have no
work directory and type "make build && make install", then the "install"
step does not have a PATH set to include all the wrapper and tools
directories.
stages, and that installs dependencies listed in BOOTSTRAP_DEPENDS.
The bootstrap-depends step works just like the normal depends step
and honors the value of DEPENDS_TARGET. It's now possible to add
dependencies solely to facilitate fetching the distfiles, e.g.
BOOTSTRAP_DEPENDS+= curl-[0-9]*:../../www/curl
* Teach the tools framework about ":bootstrap" as a tools modifier
which indicates the tool should be added as a dependency via
BOOTSTRAP_DEPENDS.
* Add "digest" to the tools framework.
* Use USE_TOOLS+=digest:bootstrap to force pkgsrc to install digest
before anything else. Get rid of unused "uptodate-digest" target
and related digest version-checking code.
* Finish the refactoring work: split checksum-related code out of
bsd.pkg.mk and into pkgsrc/mk/checksum and replace the "checksum"
target command list with a script that does all the real work.
* Make DIGEST_ALGORITHMS and PATCH_DIGEST_ALGORITHM into private
variables by prepending them with an underscore. Also, rename
_PATCH_DIGEST_ALGORITHM to _PATCH_DIGEST_ALGORITHMS and adjust the
makepatchsum target to allow that variable to contain a list of
algorithms, all of which are used when creating the patch checksums
for ${DISTINFO_FILE}.
introducing the concept of a "barrier". We separate the user-invokable
targets into ones that must happen before the barrier, and ones that
must happen after the barrier. The ones that happen after the barrier
are run in a sub-make process. In this case, the targets that must
be run after the barrier are from the "wrapper" step and beyond. We
rewrite the various "flow" targets, e.g. wrapper, configure, build,
etc., so that they of the right form to use the barrier target.
This now completely removes the concept of PKG_PHASE from pkgsrc. It
is replaced with the concept of "before" and "after" the barrier, and
this state can be checked by testing for the existence of the barrier
cookie file. Because we've removed most of the recursive makes, there
is now nowhere to hook the PKG_ERROR_HANDLER.* commands, so remove
them for now.
As part of this commit, put back the logic that conditionalized the
sources for the various cookie files. Because the sources are all
"phony" targets, they were always run, regardless of whether or not
the cookie file already existed. Now, if a cookie file exists, then
that entire phase associated with that cookie file is skipped.
Lastly, fix a thinko in configure/bsd.configure.mk where setting
NO_CONFIGURE in a package Makefile would manage to skip the "wrapper"
step altogether. Fix this by correctly noting "wrapper" and not
"patch" as the preceding step to "configure".
of the logic from fetch/fetch.mk into flavor/pkg/check.mk, so that
check-vulnerable can be used as a source target.
Make check-vulnerable a source target for every phase of the build
workflow, which ensures that it is always run if the user starts a
new phase from the command line.
Fix the cookie-generation targets so that they don't append, only
overwrite to the cookie file. This works around potential problems
due to recursive makes.
Move the cookie checks so that they surround the corresponding phase
target. The presence of the cookie should now inform the make process
to avoid doing any processing of phases that occur before the phase
corresponding to the cookie.
which are invoked in recursive make calls for the "install" and
"package" targets respectvely. These recursive make calls prevent
the top-level make from seeing all of the targets and computing a full
dependency graph, so it becomes possible for some targets to be invoked
more than once. This change passes enough information along to the
recursive make calls and ensures that the source targets for the real-*
targets are only invoked once.
are generated for a target and output them all at once at the conclusion
of the target's invocation. The implementation is in bsd.pkg.error.mk,
which defines a macro target "error-check" that will print out any
non-empty warning and error files in ${WARNING_DIR} and ${ERROR_DIR}
and exit appropriately if there were errors.
Convert some targets that were just long sequences of ${ERROR_MSG} or
${WARNING_MSG} within a single shell statement to use the new delayed
error output via error-check.
Modify the compiler "fail" wrappers for C++ and Fortran to be less
verbose during invocation. Instead collect the warnings and only
print them at the end of the completed phase, e.g. after "configure"
and/or "build" completes.
in error message like 'WARNING: Warning: ...'.
2.) Replace "WARN_MSG" with "WARNING_MSG" which makes the "make package"
target work again for restricted packages like "acroread7".
bsd.pkg.mk. They didn't actually need to be defined in bsd.prefs.mk,
just somewhere before the "main" bsd.<phase>.mk files were included.
This moves some conditional (?=) definitions back into bsd.pkg.mk so
they won't conflict with any conditional definitions in package
Makefiles.
This should fix the "checksum" problems in lang/php-gd as noted here:
http://mail-index.netbsd.org/pkgsrc-users/2006/06/05/0012.html
where EXTRACT_SUFX had the wrong value due to the order in while *.mk
files were included.
if PKG_SKIP_REASON or PKG_FAIL_REASON is defined. This commit adds
!target(...) guards around those target definitions to avoid "duplicate
target definition" warnings.
than pkgsrc's current one. This is an important lead-up to any project
that redesigns the pkg_* tools in that it doesn't tie us to past design
(mis)choices. This commit mostly deals with rearranging code, although
there was a considerable amount of rewriting done in cases where I
thought the code was somewhat messy and was difficult to understand.
The design I chose for supporting multiple package system flavors is
that the various depends, install, package, etc. modules would define
default targets and variables that may be overridden in files from
pkgsrc/mk/flavor/${PKG_FLAVOR}. The default targets would do the
sensible thing of doing nothing, and pkgsrc infrastructure would rely
on the appropriate things to be defined in pkgsrc/mk/flavor to do the
real work. The pkgsrc/mk/flavor directory contains subdirectories
corresponding to each package system flavor that we support. Currently,
I only have "pkg" which represents the current pkgsrc-native package
flavor. I've separated out most of the code where we make assumptions
about the package system flavor, mostly either because we directly
use the pkg_* tools, or we make assumptions about the package meta-data
directory, or we directly manipulate the package meta-data files, and
placed it into pkgsrc/mk/flavor/pkg.
There are several new modules that have been refactored out of bsd.pkg.mk
as part of these changes: check, depends, install, package, and update.
Each of these modules has been slimmed down by rewriting them to avoid
some recursive make calls. I've also religiously documented which
targets are "public" and which are "private" so that users won't rely
on reaching into pkgsrc innards to call a private target.
The "depends" module is a complete overhaul of the way that we handle
dependencies. There is now a separate "depends" phase that occurs
before the "extract" phase where dependencies are installed. This
differs from the old way where dependencies were installed just before
extraction occurred. The reduce-depends.mk file is now replaced by
a script that is invoked only once during the depends phase and is
used to generate a cookie file that holds the full set of reduced
dependencies. It is now possible to type "make depends" in a package
directory and all missing dependencies will be installed.
Future work on this project include:
* Resolve the workflow design in anticipation of future work on
staged installations where "package" conceptually happens before
"install".
* Rewrite the buildlink3 framework to not assume the use of the
pkgsrc pkg_* tools.
* Rewrite the pkginstall framework to provide a standard pkg_*
tool to perform the actions, and allowing a purely declarative
file per package to describe what actions need to be taken at
install or deinstall time.
* Implement support for the SVR4 package flavor. This will be
proof that the appropriate abstractions are in place to allow
using a completely different set of package management tools.
used by the various scriptlets. Also, consistently use the idiom of
creating a temporary directory with mode 700 and creating temporary
files underneath that directory to avoid race conditions.
${PKG_SYSCONFDIR} if PKG_SYSCONFSUBDIR is defined and non-empty. A
package may now set PKG_SYSCONFDIR_PERMS to an "owner group mode"
triplet, which defaults to "${ROOT_USER} ${ROOT_GROUP} 755".
where they don't verify that any pre-existing config files and
directories have the correct permissions. For example, if you are
upgrading a package to a newer version and the config files and
directories used by the package need to have different permissions
than in previous versions of the package, then the new package may
fail to work because it can't access pre-existing files and directories.
This commit improves on this by doing the following:
(1) Teach the +FILES and +DIRS scriptlets two new actions "PERMS" and
"CHECK-PERMS". "PERMS" fixes permissions on existing files and
directories. "CHECK-PERMS" will verify those same bits and warn
the user when they are wrong. The "CHECK-PERMS" actions for the
two scriptlets are run immediately after the "ADD" actions.
(2) Add a new variable PKG_CONFIG_PERMS that controls whether the
"PERMS" action will automatically fix permissions. PKG_CONFIG_PERMS
is only consulted if PKG_CONFIG is "yes". PKG_CONFIG_PERMS can
be set in the shell environment when running pkg_add, e.g.:
export PKG_CONFIG=yes
export PKG_CONFIG_PERMS=yes
pkg_add /path/to/binary/package.tgz
The default value of PKG_CONFIG_PERMS embedded into the +INSTALL
script may also be set in /etc/mk.conf. This value defaults to
"no", so that by default, the +INSTALL script will not modify or
destroy any existing configuration files or directories.
The +INSTALL script will now always warn you if there are files or
directories whose permissions differ from what the package is expecting
to use, and if PKG_CONFIG_PERMS is set to "yes", then it will go ahead
and fix those permissions for you automatically.
(1) Allow specifying the numeric UID and GID for users and groups in
/etc/mk.conf by setting PKG_UID.<user> and PKG_GID.<group> to
those values. If these values are specified, then the +USERGROUP
script will verify that existing users and groups match the
requested UIDs and GIDs for the package, and otherwise create them
with these UIDs and GIDs. For example:
PKG_UID.courier= 10001
PKG_GID.mail= 6
In this example, the courier-authlib binary package will be created
to use uid 10001 for the "courier" user and gid 6 for the "mail"
group.
(2) Allow a package to request that users and groups be created prior
to configuring or building a package by setting USERGROUP_PHASE
to "configure" or "build". Because the reason for this is typically
to hardcode the UIDs and GIDs of requested users and groups directly
into the package's executables, these hardcoded values will be
automatically determined and put into the +USERGROUP script. For
example:
USERGROUP_PHASE= configure
PKG_GROUPS= qmail nofiles
PKG_USERS+= qmaill:nofiles
PKG_USERS+= qmailq:qmail
In this example, the users and groups are created before the
configure phase when building qmail, and the qmail binary package's
+INSTALL script will try to create (or verify) users and groups
with the same UIDs and GIDs that were used during the build.
As part of these changes, the format for PKG_USERS and PKG_GROUPS has
changed -- the optional parts of the corresponding entries are no
longer used and cannot be specified. Instead, the following variables
should be set:
PKG_GID.<group> is the group's numeric GID.
PKG_UID.<user> is the user's numeric UID.
PKG_GECOS.<user> is the user's description.
PKG_HOME.<user> is the user's home directory.
PKG_SHELL.<user> is the user's login shell.
A separate commit will follow which will fix all packages that set
PKG_USERS and PKG_GROUPS to use the new syntax and variables.
(1) Don't claim to be adding and removing the info file if it does
exist at all.
(2) Always try to create and remove the directory containing the "dir"
index file to avoid failures.
directory containing the "dir" file that is updated. This allows
packages to install info files in one place but update the dir file
in a separate location.
that is a purely user-settable variable to represent the relative
path under ${PREFIX} where info files are stored and "dir" files
are managed. PKGINFODIR defaults to "info". INFO_DIR still works,
but will be obsoleted after the 2006Q1 branch.
* Modify GNU_CONFIGURE_INFODIR to only honor ${PKGINFODIR} if the
package installs directly into ${PREFIX} and not some subdirectory
under ${PREFIX}. This fixes packages that don't really honor
$(infodir) all that well, and also avoids PLIST problems relating
to directory removal for those packages.
* Since the majority of Emacs Lisp packages use GNU_CONFIGURE, just
set GNU_CONFIGURE_INFODIR directly to ${EMACS_INFOPREFIX}, which is
the Emacs-distro-specific location for info files. Also pass
EMACS_INFOPREFIX through PLIST_SUBST for PLIST substitution.
* INFO_FILES should be defined if the package installs info files.
If the info files are not listed in the PLIST, then INFO_FILES
must list the filenames for the info files installed by the package,
which are assumed to be located in ${PREFIX}/${PKGINFODIR}.
* The plist module can now better detect info files listed in PLISTs
and exports a command to the pkginstall module to append info file
names to the +INFO_FILES scriptlet at install-time.
* The print-PLIST target is updated to properly list info files in
the auto-generated PLIST.
* The check-files code is updated to skip all "dir" Info database files.
and rc.d scripts so that we source the latest makevars.mk file and
get the latest set of cached variables. This fixes problems where
BUILDLINK_PREFIX.* wasn't being expanded correctly when substituting
into an INSTALL script.
INSTALL/DEINSTALL and rc.d scripts, regardless of whether NO_BUILD is
defined or not.
We do this by renaming the main "build" target to "_build", and creating
a new "build" target that has "_build" and "pkginstall" as dependencies.
This allows the "build" and "install" targets to be consecutive, so
no changes in behavior are visible to the user.
Because the pkginstall targets are no longer run within the protection
of the locks during the build phase, we need to manage locking within
a new "pkginstall" target.
scripts are generated. The various scriptlets in pkgsrc/mk/install
are now full templates that can be combined to form the INSTALL and
DEINSTALL scripts. All of the templates have either leading or trailing
blank lines so that when they are concatentated, the full INSTALL and
DEINSTALL scripts will still be easy to read.
All of the generated template sources for the INSTALL and DEINSTALL
scripts are now stored in ${WRKDIR}/.pkginstall for local consistency
and have been renamed so that they are not dot-files.
The "data" for the +* scriptlets are now generated in a separate step
and instead of being appended to the unpacked scriptlet are now appended
to the INSTALL script itself. When the scriptlets are unpacked, the
corresponding data lines for each scriptlet are pulled out of the
INSTALL script and put into the unpacked scriptlets. This makes it
easy to append more data lines during the install phase without needing
to regenerate the INSTALL script.
