Commit graph

82 commits

Author SHA1 Message Date
jlam
817fd9ec0d All of the other *SITES* variables are plural, so rename DYNAMIC_SITE_*
to DYNAMIC_SITES_* for consistency.
2006-07-27 16:06:27 +00:00
jlam
e7af7b2e05 Apply the "convention over configuration" principle:
If ${FILESDIR}/getsite.sh exists, then use it to determine the fetch
URL for each of the distfiles for the package.  Otherwise, use
SITE_<file> and MASTER_SITES, in order, to determine the URL for each
distfile.

If the script path differs from ${FILESDIR}/getsite.sh, then set
DYNAMIC_SITE_SCRIPT to the full path to that script.

Remove the need to set DYNAMIC_MASTER_SITES explicitly in the package
Makefile for:

	graphics/ns-cult3d
	wm/sawfish-themes
	www/apache-tomcat55
	www/jakarta-tomcat4
	www/jakarta-tomcat5
2006-07-27 15:16:44 +00:00
jlam
d5f750aa08 The ORDERED_SITES variable defined in fetch.mk was made private in
revision 1.14, so reflect that in the usage in fetch-list.mk.  This
fixes PR pkg/34075.
2006-07-27 13:51:21 +00:00
rillig
ac35deefb7 Documented DIST_PATH and fixed two typos (${TARGET} is not ${.TARGET}). 2006-07-27 07:41:40 +00:00
wiz
078ec42655 Remove mirrors.sunsite.dk because it causes hangs in the
weekly distfiles fetch.
2006-07-23 14:11:52 +00:00
wiz
0b9a3594c5 Remove cran.dk -- causes hangs for weekly distfiles mirror job. 2006-07-23 13:28:14 +00:00
joerg
eeaca99630 verify_file() returns a non-zero result whenever the distfile does not
currently exist. This triggers a bug in the Bourne shell of at least
DragonFly and Irix, when set -e is also active. Remove set -e to avoid
this. Discussed with jlam@, shown by DragonFly bulk builds, complains on
DragonFly's user list and PR 34036.

A test case for this shell bug is, which should return 0:

set -e

testf () {
	test -f /nonexistent || return 1
	return 0
}

if testf; then
	exit 1
fi
exit 0
2006-07-20 13:31:08 +00:00
jlam
c88a32be7d Make "mirror-distfiles" less verbose by not printing out anything if no
bootstrap dependencies need to be built and installed.
2006-07-19 16:01:40 +00:00
jlam
ec6f4ba413 Comment the code a bit more. 2006-07-19 15:13:40 +00:00
jlam
ae7cf87ca5 Only pass -v to the fetch script if PKG_VERBOSE is set. 2006-07-19 14:25:59 +00:00
jlam
cbcfa6d0b5 Restore original quoting from old fetch.mk for _ORDERED_SITES to resolve
the problem with MASTER_SORT*.

XXX The whole MASTER_SORT* thing should just be a standalone script.
2006-07-19 14:22:47 +00:00
jlam
2936f01adf Add a verbose flag to the fetch script to output the actual fetch command. 2006-07-19 14:16:29 +00:00
jlam
d2df49806c Make some variables "private":
MASTER_SORT_AWK
	SORT_SITES_CMD
	ORDERED_SITES

These variables are purely internal to pkgsrc.
2006-07-19 14:03:59 +00:00
jlam
16335bb8a4 Work around incomprehensible quoting mess surrounding MASTER_SORT* by
in do-fetch-file (formerly they were inserted by the old _FETCH_FILE
inserting some quotes that are no longer added by the fetch command list
macro).  This makes fetching with MASTER_SORT* set work again.
2006-07-19 14:01:29 +00:00
jlam
1302d118e8 PASSIVE_FETCH only affects "ftp", so move it to the ftp-related FETCH_*
variables.
2006-07-19 13:33:42 +00:00
jlam
4c09a7046a Convert _FETCH_MESSAGE to plain old FETCH_MESSAGE. Remove _FETCH_MESSAGE
processing from mk/fetch/*.mk.
2006-07-18 23:27:14 +00:00
jlam
833691e2fe Overhaul fetch module:
* All the smarts is now encapsulated in the "fetch" script.  The fetch
  script understands how to use the distinfo file (if specified) to
  look up the size and checksums of the file to fetch and will use
  that information to verify checksums of the fetched files or resume
  transfers of interrupted fetches.

* Move the default settings for FETCH_RESUME_ARGS and FETCH_OUTPUT_ARGS
  for "ftp" from mk/defaults/mk.conf into mk/fetch/fetch.mk.  We rewrite
  it to avoid needing conditional statements.

* Avoid spawning a new make(1) process just to mirror a distfile.

* Split out fetch-list targets into a separate file fetch-list.mk.
  These targets should probably be moved into a standalone script.

* Fix distclean target to properly remove partial downloads.
2006-07-18 22:41:06 +00:00
jlam
e65b7c1371 First cut at a fetch script to replace the humungous fetch "macros" in
fetch.mk.  This script currently completely replaces the functionality
in _FETCH_FILE.  I will eventually add the ability to resume a file
transfer to this script.
2006-07-17 15:34:22 +00:00
schmonz
0a53ed9007 Remove duplicate files from ${ALLFILES}. This doesn't affect the
"fetch" target, but it does affect the "checksum" target's shell
script, which errors out on the second occurrence of a file. The
shell script should perhaps also be fixed, but it seems sensible
regardless for ${ALLFILES} not to contain duplicate filenames. As
a side effect, the file list is sorted.

Regression found by building mail/qmail with the "qmail-netqmail"
option, which adds to PATCHFILES a file that's already in ${DISTFILES}.
Arguably this is gross, but it worked before, and now works again.

Tested on my usual pkg_comp(8) build of 200+ packages, with an
initially empty ${DISTDIR} and ${PACKAGES}. Thanks seb@ for the
more idiomatic make(1) construction.
2006-07-17 02:13:11 +00:00
rillig
2221827892 Split the variable BUILD_DEFS into those that are defined by packages
and those that are defined by the infrastructure (_BUILD_DEFS). This
allows the build-defs-message target to be moved to the end of
bsd.pkg.mk. Now it prints the correct result even in unprivileged
builds, which had been wrong due to the order in which the files have
been included. For example, ${UNPRIVILEGED_USER} was displayed as (not
defined) although its value was defined, which could be checked with
"bmake show-var".

Tested with one package that _does_ define BUILD_DEFS and with one that
doesn't. The behavior stays the same.
2006-07-15 23:58:52 +00:00
jlam
b532d92d10 DIGEST is no longer defined, so use TOOLS_DIGEST instead. 2006-07-13 15:23:18 +00:00
jlam
7a2c97d53f * Add a new stage "bootstrap-depends" that happens before all other
stages, and that installs dependencies listed in BOOTSTRAP_DEPENDS.
  The bootstrap-depends step works just like the normal depends step
  and honors the value of DEPENDS_TARGET.  It's now possible to add
  dependencies solely to facilitate fetching the distfiles, e.g.

	BOOTSTRAP_DEPENDS+=	curl-[0-9]*:../../www/curl

* Teach the tools framework about ":bootstrap" as a tools modifier
  which indicates the tool should be added as a dependency via
  BOOTSTRAP_DEPENDS.

* Add "digest" to the tools framework.

* Use USE_TOOLS+=digest:bootstrap to force pkgsrc to install digest
  before anything else.  Get rid of unused "uptodate-digest" target
  and related digest version-checking code.

* Finish the refactoring work: split checksum-related code out of
  bsd.pkg.mk and into pkgsrc/mk/checksum and replace the "checksum"
  target command list with a script that does all the real work.

* Make DIGEST_ALGORITHMS and PATCH_DIGEST_ALGORITHM into private
  variables by prepending them with an underscore.  Also, rename
  _PATCH_DIGEST_ALGORITHM to _PATCH_DIGEST_ALGORITHMS and adjust the
  makepatchsum target to allow that variable to contain a list of
  algorithms, all of which are used when creating the patch checksums
  for ${DISTINFO_FILE}.
2006-07-13 14:02:34 +00:00
rillig
1dd9c5c90e Removed the leading "-" of a shell command. The unexpected error message
that it had produced had confused me.
2006-07-12 12:34:13 +00:00
jlam
990d0b5377 Make the check-vulnerable target more self-sufficient, by moving some
of the logic from fetch/fetch.mk into flavor/pkg/check.mk, so that
check-vulnerable can be used as a source target.

Make check-vulnerable a source target for every phase of the build
workflow, which ensures that it is always run if the user starts a
new phase from the command line.

Fix the cookie-generation targets so that they don't append, only
overwrite to the cookie file.  This works around potential problems
due to recursive makes.

Move the cookie checks so that they surround the corresponding phase
target.  The presence of the cookie should now inform the make process
to avoid doing any processing of phases that occur before the phase
corresponding to the cookie.
2006-07-05 09:08:35 +00:00
jdolecek
d35e963055 add .cz KDE mirror 2006-06-25 18:02:38 +00:00
tron
be3eaae91c Remove "apache.towardex.com" (down) and "sunsite.tus.ac.jp" (redirects
to new version) from the list of Apache master sites.
2006-06-23 07:45:49 +00:00
jlam
8f71bbc4ec If FAILOVER_FETCH is defined, then we need the digest binary to already
be installed, so let uptodate-digest do that crazy thing it does.
2006-06-20 14:54:03 +00:00
jlam
fce460b04b Fix FAILOVER_FETCH usage to actually match documentation -- it's
supposed to be just defined/undefined, but was previously being used
as non-empty/emtpy.
2006-06-20 14:50:27 +00:00
jlam
3058ace76f Move some variable definitions out of bsd.prefs.mk and back into
bsd.pkg.mk.  They didn't actually need to be defined in bsd.prefs.mk,
just somewhere before the "main" bsd.<phase>.mk files were included.
This moves some conditional (?=) definitions back into bsd.pkg.mk so
they won't conflict with any conditional definitions in package
Makefiles.

This should fix the "checksum" problems in lang/php-gd as noted here:

    http://mail-index.netbsd.org/pkgsrc-users/2006/06/05/0012.html

where EXTRACT_SUFX had the wrong value due to the order in while *.mk
files were included.
2006-06-06 15:28:51 +00:00
jlam
4a1c92e775 Improve the warning message if the vulnerabilities file is not found. 2006-06-06 03:36:00 +00:00
jlam
eee5c6a41f Add documentation about which are the public targets for the fetch module. 2006-06-06 03:35:42 +00:00
jlam
c78510391e Refactor "fetch" and "extract" code into correspondingly named
subdirectories of pkgsrc/mk.  Move the following files around for
locality:

	pkgsrc/mk/scripts/extract  -> pkgsrc/mk/extract/extract
	pkgsrc/mk/bsd.sites.mk     -> pkgsrc/mk/fetch/sites.mk

Also get rid of the recursive make for the "fetch" and "extract"
targets.  This basically merges the "fetch" and "extract" phases into
the "patch" phase.

There is still much more work to do to simplify the fetch code, but
this is a good start.
2006-06-06 03:05:48 +00:00