this port confusing bsd.pkgng.mk into thinking
that WITH_PKGNG is defined and pkg(8) shoul be
used to register.
Pointyhat to: bdrewery
Feature safe: yes
- Add PKGNG option to apply pkgng support patch. This will
be fully integrated upstream in the next version. This
functionality is only enabled when WITH_PKGNG is defined
in make.conf (or a recent CURRENT by default, see
UPDATING 20121010). Due to the compat OPTION handling,
defining WITH_PKGNG will also apply the patch by default.
The pkgng patch is contributed by avilla@
Feature safe: yes
- Take maintainership
- Convert to new options framework
- Use USE_GITHUB for fetching
- Add DISTNAME to avoid conflict with existing distfile as
the checksum of the distfile has changed now that it is
no longer modified for release.
- Remove use of bsd.port.pre.mk
Minor improvements:
===================
* Improve messages for processing of dependent ports when -r
* If there is an installed version, always print that message, not just
when we're in the parent
* Optimize determination of $new_port:
- When first set, let it be global
- Use a function to incorporate/optimize the various tests from the
2 different places that it was being set
* Use a more robust method of determining portdir from $PWD. This also
works for bare 'portmaster' with no arguments now, if you're in a port
directory.
* Various additional code cleanups
Bug fix:
========
Do read_distinfos() in the background for multiport() too, oops
Minor fixes/improvements
========================
* Turns out there are a non-zero number of distinfo files that have SIZE
listed first, so in the code that parses them for the data to put into
/var/db/pkg/$port/distfiles, handle this situation gracefully.
* If we trap while working on a child port do not remove that child from
the PM_NEEDS_UPDATE list so that when it gets printed in the parent
that child is still on the list.
* Use the count of total ports as $num_of_deps for -Faf
* For -av, add num_{roots|trunks|branches|leaves} values to the headers
* In multiport(), decrement the number of ports to work on and print a -v
message if we have already seen a port in the worklist as a dependency
of another port in the list.
* Many other minor code cleanups
Performance enhancement
=======================
* In read_distinfos*() use while read instead of grep. Avoids the fork
and reduces the total time for building the list(s) of distfiles by 25%
Major updates/fixes to the distfile handling code
=================================================
* Make all parts of the code SUBDIR-safe (most were already)
- Fix a few places where the relativity of the cwd was not respected,
most often resulting in files not being deleted that should be
* Move the storage of the distfile info back to /var/db/pkg where it
should have been all along. Mostly for correctness, but also to deal
with the problem of non-UNIQUENAMEs causing the wrong files to be
deleted with -d.
* Clean up the old /var/db/ports/*/distfiles files after reading them,
and delete the directory if it's empty. Since we are doing this
unconditionally now, there is no longer a need for delete_dist_list().
The new ones will live and die with the rest of the package directory.
* Update some of the messages printed out for distfile-related things
to make it more clear what's going on.
* Check DI_FILES to see if the distfile could be valid for another port
in more places, so add a function to deal with the common ones.
* When checking DI_FILES, anchor the grep pattern with a ^ to
avoid false positives
Other code changes
==================
* Make the whitespace more consistent
* Various minor improvements, mostly to avoid unnecessary forking
UI Change
=========
* When using -i and there is an +IGNOREME file, the default should be no
==================
* Restore the line from -av that prints each port as we work on it.
* Print more useful information about ports that don't have distfiles,
or distfiles we're skipping due to duplicates when -F (especially
with -a).
Bug fixes
=========
* Consolidate all writing to the IPC_SAVE file into safe_exit()
so that it always gets called. Also, zero out the file after
reading it so that a parent process doesn't inadvertently
read stale data during a trap (unlikely, but not impossible).
============
Print the same progress update that goes to the term title in-line,
and apply various improvements to that code.
Move 'make clean' to the background. There is no point waiting around
for it to finish. In the case of very large work directories this may
cause some I/O contention during the unpacking of the next port to
build, but in most cases this should be lost in the noise.
Other Improvements
==================
Speed up the determination of WRKDIRPREFIX, and do not bother looking
for empty directories if we exited abnormally.
Add code to detect dependency loops created by moving part of a port
into another port, and CONFLICT'ing on previous versions of that port
(ala textproc/p5-XML-SAX -> p5-XML-SAX-Base). When a loop is detected
suggest that the user try pkg_updating.
Bug Fixes
=========
Improve error handling for mktemp
Improve handling of abnormal exits (such as ^C, failed port, etc.),
especially when the first exit happens in a child process.
If the second argument to -o is something that is completely non-existent,
report in the error message what the user typed on the command line.
Reported by: avilla
In clean_build_only_list() remove ports that are already installed
when using --delete-build-only to prevent them from being spuriously
removed.
Reported by: avilla
============
1. Add a feature to allow users to list files that should be
preserved across upgrades.
Various Improvements
====================
1. Add a verbose message for cleaning WRKDIRPREFIX
2. Improve the handling of the -F option with packages, especially with
--packages-only:
* In update_port() do better at saying what we're launching a child to do
* In post_first_pass() make clear that we are fetch'ing, not building,
installing, etc.
* In multiport() call check_fetch_only when we're done
* In the main body add fetch'ed ports to the CUR_DEPS list so that they
are only fetched once.
3. Various typo fixes for the man page.
Bug Fixes
=========
1. If an update fails in a child port the parent needs to source the
$IPC_SAVE file before it bails out to make sure that various lists are
updated in the parent. This is particularly important for the list(s)
of work already completed, and especially to update PM_NEEDS_UPDATE for
the message of how to resume from the point of failure.
2. For multiple -r, don't add a port to the list of dependents to rebuild
if it was itself specified as a -r port.
3. In parent_exit() run the code to kill background processes if we're
called with a non-zero exit code. That will trigger it (for example)
where the user answers no to the confirmation prompt and is either
using -D, or the generation of the list of valid distfiles did not
complete.
4. Until I can revamp the way dependencies are tracked, include all 3
forms of CONFLICTS to make sure we catch alternates.
============
1. Add the ability to retry package creation. This is mostly useful for
PM_SU_CMD after a build that took so long that your sudo ticket has
expired.
2. Add support for multiple -r options on the same command line. [1]
Various Internal Cleanups and Improvements
==========================================
1. s/printf/echo -e/ for consistency, and use it to full advantage
2. In the code that derives the proper value of PACKAGESITE remove 6.x
and add 9.x. Also add a 10-current entry.
3. Add a more-or-less painless test to deal with the issue of the current
working directory no longer existing. This problem usually manifests as
a "The value of DISTDIR cannot be empty" error because 'make -f' does
not handle this condition gracefully.
4. In find_and_delete_distfiles() (run after a successful install) change
the default pattern to be foo-[0-9]* instead of foo* [2]
When the new pattern doesn't match any existing distfiles, fall back to
the old one.
5. Also in find_and_delete_distfiles(), cache matches to $DI_FILES so that
we don't have to re-test that same file again.
6. Change the method used to print the prompt in get_answer_g() to echo -e
so that I can be more flexible in formatting prompt messages.
7. For --clean-packages if a package is up to date but not installed
it should still be deleted if -y is in use.
8. Avoid duplicate word in a composite sentence [3]
9. If a user has non-standard FETCH_ARGS it could circumvent the intent
of --always-fetch, so forcibly unlink the file before proceeding to
the fetch routine.
Sponsored by: Henk van Oers [1]
Suggested by: Lapo Luchini <lapo@lapo.it> [2]
Submitted by: Dan Allen <danallen46@airwired.net> [3]
In shuffling around the updating of +REQUIRED_BY I forgot to delete
empty grep_deps files. [1]
While I'm looking at unlink'ing files, bring some sanity to the various
places that it's done which haven't been reviewed in a while:
* Fix the pm_unlink* functions to use the builtin test
* Only/always call the function when we're not sure if the file exists
* Just use /bin/unlink when we're sure it does
* In parent_exit the glob pattern in the for loop will return 'pattern-*'
if there are no matches, so test for that specifically rather than
returning to the conditional version.
The mere presence of IS_INTERACTIVE in the Makefile is not enough, the
condition needs to be verified in case it is conditional [2]
We want to make sure that $TMPDIR exists before trying to use it. The
cheapest way to do this is to just call mkdir -p. Move the umask call
before this to be safe.
Split the test for the existence of /usr/X11R6 and whether or not it's
a link for one more micro-optimization.
Submitted by: olgeni [1]
PR: ports/157481 [2]
Submitted by: Carl <k0802647@telus.net> [2]
=============
* The -F option now works with --packages and --packages-only
o If -F --packages-only and there is no package to fetch,
give the user a menu of options
o If -F and --packages lack of a package is a soft fail
* Revamp the code in check_fetch_only() and where we wait for b/g
distfile fetching to complete to be a lot smarter:
o In check_fetch_only() simplify the code to count fetch processes which
both makes it faster by avoiding ps, and also makes it smarter because
it will find fewer false positives.
o In both, poll every 2 seconds, but only output to the console every 6.
This way we'll be able to exit a lot sooner, but still avoid spamming
the user with lots of messages.
* In the event that an invalid combination of configuration options causes
a port to be BROKEN, etc. allow the user to run --force-config and try
again
* Add the ability to specify the -t option to -L and --index-only to
check every installed port instead of using the shortcut. This is
slow, but useful on an old system where there is no ports tree and
one or more of the ports is so old that it no longer exists in INDEX.
Bug Fixes:
==========
Fix an unfortunately long-standing bug in the --package code, we need to
update the +REQUIRED_BY files for ports that the installed port depends on.
When I tested pkg_add originally I confirmed that it did this, but I
neglected to re-test when adding the --no-deps option, which suppresses
this behavior. Mea culpa.
So take this opportunity to fix the bug, and improve some other things in
the process:
* Add code to the section that updates dependencies for the newly installed
package to walk the +CONTENTS file and runs the +REQUIRED_BY update for
every port that the newly installed depends on.
* Various non-user-visible optimizations to the dependency udpating code
* When called from gen_dep_list, which also uses temp_list as a local
variable, uniquify_list() needs its own set of local variables otherwise
temp_list ends up being duplicated by uniquify_list
Minor Optimizations:
====================
* Micro-optimize a few more cases where we can avoid going out to disk
to test stuff by separating the test for a variable first.
* Continue the campaign to unset things to keep the environment small
* Various other micro-optimizations
* Fix a long-standing bug with EUID detection. If the UID was > 4 digits
the code would fail. Fix this by using a more efficient ps command.
While I'm at it, use the same more efficient method for determining
the process group id for the trap cleanup routine.
* Fix a buglet in the new --update-if-newer code, exit if all listed
ports are actually up to date.
* Add --update-if-newer which will operate on a list of ports given
on the command line. If a port list is already up to date, it
will be skipped.
* Tweak the way INDEX files are downloaded and installed to only
require root privileges for the minimum possible number of actions,
and do things in a more secure way:
1. If necessary to download the new INDEX.bz2, as a non-root user, do
it in a temporary directory created with mktemp
2. If necessary to update the actual INDEX file unzip it in that same
temporary directory
3. Only use $PM_SU_CMD to install the file if necessary
4. Unlink both INDEX and INDEX.bz2 in their real locations before
installing the new versions
* Add the ability to store the portmaster.rc file in the same directory
as the script itself.
Many improvements to the --index-only/--packages-only code, particularly
in the absence of a ports tree.
* Make the --index* options more friendly to being duplicated in the rc
file and command line. Mixing different options will still trigger an
error, however having the same option both places will not.
* Don't do the work of handling the INDEX file if we won't need it
(-l, --list-origins, -e, -s)
* Reorganize the final check of $pd (PORTSDIR) to properly handle the
various combinations
* For multiple ports and -o, only test for the origin directory if
we are not using the INDEX. If we are, check it there instead.
* Add some code to --check-depends to explicitly handle --index-only.
It worked before by ignoring the origin test if there was no ports
directory, but failed if --index-only was also used.
This is better both because it doesn't fail with --index-only, but
also because it now uses the index to validate $origin.
* Since --check-depends now works with --index-only, update usage()
Also:
* Add some whitespace to the "Installing new +CONTENTS" message
* In --clean-packages show the path in the delete prompt for stale
packages that are not installed like we do for ones that are
* Various internal cleanups and micro-optimizations
New Features
============
1. The PT_NO_INSTALL_PACKAGE make variable will cause a port to always
be compiled, instead of being installed from a package. It cannot
be used in combination with the -PP/--packages-only option. [1]
2. All user input is now wrapped in a verification routine to make
sure that yes means yes, etc.
3. The -e, -s, --clean-distfiles, --clean-packages, --check-depends
and --check-port-dbdir features can now take -y and -n options to
answer "yes" or "no" respectively to all user dialogs. This also
allows me to deprecate the horribly hacky --clean-distfiles-all and
--clean-packages-all in favor of -y.
4. If using a separate WRKDIRPREFIX, when the run is over use find to
clean out the empty directories that are left behind by the ports
infrastructure.
5. If PM_PACKAGES is set, don't bother with build depends [2]
Small Fixes And Improvements
============================
1. Update usage() in the script and the man page for the new features,
and improve the consistency between the two.
2. When we enter -PRERELEASE phase the package-finding code should
still be looking at -stable. [3]
3. Fix some bugs with -r:
* In certain situations dependent ports that were otherwise
up to date would be missed.
* For the purpose of not rebuilding the parent port when -R
is used we only care about the dependencies of the parent
port being up to date.
4. Error handling for INDEX fetch
5. The -e, -s, -l, -L, --list-origins, and --check-depends features
all work fine without a ports tree
* In -e and -s, delete_dist_list() requires the ports tree, so
if the user didn't use -D imply it when the ports tree is
missing, and hide delete_dist_list() behind that test
6. pdb and port_dbdir need default values for usage()
Inspired by: Marco van Lienen <marco+freebsd-ports@lordsith.net> [1]
Matthew Seaman <m.seaman@infracaninophile.co.uk> [1]
Reminded by: ale [2]
Reported by: Jakub Lach <jakub_lach@mailplus.pl> [3]
When specifying 'portmaster port-1.23 dependency-of-port-2.34'
the dependency would be built twice because the PM_NEEDS_UPDATE
list is kept by ORIGIN. So use the same technique to validate
the port in the build phase that we are using in the first pass.
Also, update the documentation on full version upgrades to encourage
users to back up any vital file in /usr/local before "making sure that
it is empty."
=========
1. Since $build_l is no longer being incremented prior to the user's
+IGNOREME decision it's no longer necessary to decrement it if the
user chooses not to update that port.
2. While I was reading the trace to find this bug I noticed that we are
running update_build_l() (which in turn updates PM_NEEDS_UPDATE) prior
to running check_state() (FORBIDDEN|DEPRECATED|BROKEN|IGNORE), so move
that check up; and move the "currently installed" and "port directory"
messages up as well to give context to any messages from check_state().
3. Only print the fast-resume message if we're building. Otherwise we
might not know all the ports that need updating.
4. Fix the zsh completion file for --clean-packages[-all]
Minor Fixes
===========
1. In read_distinfos_all() a missing DISTINFO_FILE doesn't need to be fatal
like it does in read_distinfos() since it could be caused by something
uninteresting like a MASTERDIR that exists in part of the tree that has
not been downloaded (e.g., a c[v]sup refuse file). Add some whitespace
around the message in case make also emits an error.
2. For --clean-distfiles* add a line after the "Checking" message to
make the spacing consistent
3. In usage() fill in $pd if it's empty (which it will be for -h|--help)
4. Give a more helpful error message for -o when the port to be replaced
isn't installed or listed as a dependency
in -a as dependencies, then built again because they were pre-scheduled for
update.
Use the same method to handle the same problem in multiport(), which allows
me to remove one more list.
Bug Fixes
=========
1. Properly handle +IGNOREME files again
2. In safe_exit() only print the restart hint if -n PM_NEEDS_UPDATE
Enhancements
============
1. For -F, a better message for spawned children, and continue the campaign
to do less irrelevant stuff.
2. Add some whitespace to the "All ports are up to date" message for -a
3. Make it clear in the docs that -e only handles single ports, and
use the enhanced features of find_glob_dirs() to return better error
messages if a glob doesn't match one and only one port. (Problem
mentioned in PR.)
PR: ports/151824
Submitted by: Ben Grimm <freebsd-pr@bengrimm.net>
numerous internal reorganizations of various sections of code; and several
informative messages have been updated to be more informative. :)
If you're interested in all the details you can review the svn logs at
http://svn.freebsd.org/viewvc/base/user/dougb/portmaster/
Catch Up With Ports Infrastructure
==================================
1. Switch to using DISTINFO_FILE instead of MD5_FILE
2. Use the SHA256 line from distinfo instead of MD5
3. The BEFOREPORTMK trick no longer works to list MASTER_SITES for all ports
so switch to the slightly slower but more reliable pm_make without it. This
will solve the problem of portmaster not fetching all distfiles in the
background.
4. In the case statement to select the appropriate release for package
fetching glob 9.0-CURRENT to handle snapshots as well.
Various Improvements
====================
1. If we fail, offer a suggestion for a shortcut to restart
2. In messages that refer to the INDEX file, use the variable to make it
clear what we're working with.
3. In dependency_check() be clear when we're using the INDEX
4. Dramatic (9-10 times) speedup in iport_from_origin() by making an
intelligent guess about what director{y|ies] to search in based on
the origin directory name which means we're almost always
looking in either exactly the right directory the first time, or
at worst (such as the py-* ports) a dramatically smaller subset.
5. If using --index-only and neither INDEXDIR nor PORTSDIR are set, set
INDEXDIR to $TMPDIR so that we can proceed. That makes PACKAGES the
only variable the user has to set to use --index-only and --packages-only
with no $PORTSDIR and no ports directory.
Bug Fixes
=========
1. For the determination of LOCALBASE:
a. Don't try to use INDEX if PM_INDEX is not set
b. Give a more meaningful error message
2. Be more careful to check that there is a line for the port in the INDEX
when working --index-only.
3. In the package fetch routine use a better string match to see if we have
already downloaded the directory listing. The previous one worked but would
match too many files for net* and x11*, especially when installing a lot of
ports. So add a '.' to differentiate these.
4. The --clean-packages and --clean-packages-all features were added just
a little too hastily, so overhaul them.
New Features
============
1. Add an option to log actions after the run is complete. To enable
it define PM_LOG in your rc file as the path of the file you want
to log to. If using PM_SU_CMD make sure that the directory or file
is writable by the unprivileged user.
2. Add the --clean-packages[-all] feature
This works similarly to --clean-distfiles[-all] in that it searches
the $PACKAGES directory for out of date packages, and either offers
to delete them, or with -all deletes them without prompting. It
also cleans out stale symlinks and empty directories.
This necessitates moving init_packages_var() and parse_index() so
they can be used by the new feature.
The idea for grabbing the ORIGIN from the package's +CONTENTS file
and checking to see if it is up to date came from:
Alexey Markov <a.markov@complitex.ru>
3. Add an option to --clean-distfiles[-all] to allow a distfile to be
valid if it matches ANY port in the tree, not just those installed.
This requires building a distinfo list from all ports, so duplicate
the relevant bits of read_distinfos() into read_distinfos_all(). The
common elements are split into during the loop and after the loop, so
making 4 functions total doesn't seem worth it.
Add logic to --clean-distfiles to handle the -t option and call the
right read_distinfos* function accordingly.
4. Add the ability to ignore failed backup package creation by setting
PM_IGNORE_FAILED_BACKUP_PACKAGE in the environment (rc file, etc.)
5. Add the ability to skip the rebuild of the -r port by adding -R
Cleanups and Tweaks
===================
1. Finalize the deprecations of /etc/portmaster.rc, -u and -p
2. If --local-packagedir and ${LOCAL_PACKAGEDIR}/All/${new_port}.tbz
exists just use it, don't bother dereferencing $LATEST_LINK. This
is faster, but it also solves the problem of (for example)
packages/Latest/perl links to perl-5.10.x, even with 5.12.x
installed and the 5.12.x package in packages/All
3. Change the behavior of the -G option.
In the original version of portmaster the concept of 2 distinct
passes through the dependency tree, the first to do all the
'make config's and the second to build everything was intrinsic to
the design. When released publicly there were a non-trivial number
of users who objected to the idea of 'make config' being mandatory,
so I added the -G option to disable the whole first pass. However
as time went on, more and more features were added that depended on
the first pass, so it was necessary to rethink how -G works.
In this version, the 2-pass design is now universal, and the only
thing -G does is suppress the actual OPTIONS dialog. This has
several benefits:
a. Now all features are available to -G users
b. The new method of suppressing the OPTIONS dialog is 100%
effective (thus it's no longer necessary to use BATCH to truly
suppress them)
c. The --force-config option now overrides -G so users who really
hate OPTIONS can have -G in their rc file, but still have it
available by using --force-config on the command line
Lots of variables, functions, and text have been modified to
substitute out references to "config" in favor of "first pass"
4. Use nice for the make commands that actually build stuff
5. Improve the messages for fetching and updating INDEX
6. Add some whitespace in checkdepends() after removing stale data
7. If we're doing -F don't bother to initialize $PACKAGES
8. Clean up whitespace in the code
9. Use a better, more generic method of signaling an abnormal exit
and call safe_exit with a status of 1 from more places.
This is likely the cause of some of the vague problem reports I have
received in the past that include complaints of, "I used -r/f and -R
but portmaster still rebuilt certain ports a whole bunch of times."
10. In parent_exit() if INSTALLED_LIST has content but ilist does not,
move it over so we can display the right information about work done
11. Make parse_index() more forgiving of PORTSDIR != /usr/ports
12. If PORTSDIR exists, but isn't a ports tree, warn the user, and when
setting PACKAGES run tests against $pd, not literal /usr/ports.
Thanks to bf@ for these 2.
ports to update it can get quite large.
If there are a very large number of ports installed the grep in
iport_from_origin() can fail with "Argument list too long." It's also
theoretically possible that the grep would fail for other reasons. So
if it does, fall back to 'pkg_info -O'. My method is faster which is
why I'm still trying it first, but this is a sort of "belt and
suspenders" situation.
and doing so can make debugging more difficult.
Add code to post_config() to make sure that things in the
build-only dependency list are not listed as run dependencies for
something already installed.
Apparently, in some circumstances it's necessary to run ldconfig -R
when one moves shared libs from /usr/local/lib to
/usr/local/lib/compat/pkg. So add that to the -w code.
has moved) but there IS a /var/db/ports/<portname>/distfiles, we do not
want to prompt the user if they are using -d, we just want to delete
the files.
In the code to strip the port from $build_l if the user chose not to
update a port that has an +IGNOREME file, the backslash in "\t" needs
to be escaped.
=============================
1. In iport_from_origin() if we don't find a match (which is normal for
ports that are not yet installed) then return immediately.
2. A little more white space after the user input for --check-port-dbdir
3. In check_for_updates() if a port has moved we should also check for
an +IGNOREME file before forcing the update.
4. For distfile fetching:
a. Limit the scope of some of the variables
b. If a distfile has a / in the file name creation of the flag file
for background fetching will fail, so swap that for a _ instead.
