Commit graph

315 commits

Author SHA1 Message Date
nikita
9980e6202c gitlab: underscore is a valid character in gitlab project/user names. 2022-05-24 13:05:37 +00:00
nikita
24852d9fda Add USE_GITLAB, based on github.mk, to make gitlab MASTER_SITE
handling less repetitive.

 To use, set in Makefile:
 DISTNAME=     exampleproject-1.2
 MASTER_SITES= ${MASTER_SITE_GITLAB:=accountname/}

 The following variables alter gitlab.mk behavior:

 GITLAB_PROJECT        defaults to PKGBASE
 GITLAB_TAG            defaults to PKGVERSION_NOREV
                       sometimes you want to override with v${PKGVERSION_NOREV}
                       SHA-1 commit ids are also acceptable
 GITLAB_RELEASE        defaults to not defined, set this to ${DISTNAME}
                       when packaging a release not based on a git tag.
 GITLAB_SUBMODULES     manually set submodule information:
                       values should be:
                       GitLab_user GitLab_project tag_or_hash submodule_path
 GITLAB_TYPE           overrides the autodetected MASTER_SITE URL scheme:
2022-05-23 21:45:45 +00:00
gutteridge
510571864b sites.mk: update primary Apache sites 2022-05-21 00:44:58 +00:00
schmonz
318812e1f2 The non-writability check for DISTDIR is intended to prevent wasting
resources when fetchers wouldn't be able to save what they download. On
my build farm with pkgsrc (and pkgsrc/distfiles) mounted over NFS,
however, the check gives false positives for NetBSD 9.2 and -current.
Downgrade it to a warning so that these fetches can succeed, while
leaving a breadcrumb in case someone encounters a true positive.

It would of course be interesting to sort out why, in my environment, a
wide variety of other OSes get 1 for "${TEST} ! -w $fetchdir" while
NetBSD gets 0. In the meantime, joerg@ suggested this workaround and
gdt@ agrees it's reasonable to try.
2021-12-16 14:10:46 +00:00
markd
3579b1988a mk: download.kde.org redirects to https 2021-08-31 11:33:04 +00:00
nia
e309f9522a Remove all MySQL mirrors that 404 or fail to respond 2021-06-21 17:22:18 +00:00
nia
18ff2d6077 Remove broken MySQL mirror 2021-06-21 17:15:15 +00:00
nia
9f6b4e65a3 Remove extremely slow MySQL mirror 2021-06-21 17:14:22 +00:00
wiz
b0ee303e9c mk: master site xorg now prefers https 2021-04-28 09:54:24 +00:00
wiz
9945ccab7a mk: remove dead download site 2021-04-21 15:38:17 +00:00
nia
9c68a24386 correct documentation 2021-04-10 09:03:13 +00:00
ryoon
e8aa763230 mk: Introduce GITHUB_SUBMODULES to get and place Git submodules for GitHub 2021-03-30 16:47:03 +00:00
wiz
7c4e93e02a mk: remove ftp.dante.de, it does not offer FTP any longer 2021-02-14 11:46:13 +00:00
fcambus
7c23c72a84 Switch to HTTP for cpan.pair.com, the FTP service has been discontinued. 2021-01-07 09:18:31 +00:00
nia
0d650e3ac1 ftp.gnu.org redirects to https. 2020-11-11 09:26:32 +00:00
nia
d1eb683552 sites: add some working GNOME mirrors 2020-11-05 09:16:00 +00:00
nia
96bc199167 sites: remove GNOME master sites that now either fail or redirect 2020-11-05 09:07:24 +00:00
kim
b3f2f0abe1 Use http with ftp.funet.fi 2020-09-02 22:16:48 +00:00
jperkin
dc189ce30d mk: Rewrite the checksum script in awk.
The previous shell script version's runtime was quadratic against the
number of distfiles to verify.  Historically this has not been an issue,
with usually only a handful of files per package.  However, with the
introduction of Go modules the number of distfiles used by a single
package can be very high.

For example, in an upcoming update of www/grafana to version 7.1.5, the
number of GO_MODULE_FILES is 821.  Running 'bmake checksum' takes:

  real    18m20.743s
  user    17m27.975s
  sys     0m49.239s

With the awk code, this is reduced to a far more sensible:

  real    0m4.330s
  user    0m3.241s
  sys     0m0.875s

The script has been written to emulate the previous version precisely,
preserving the same output and error messages and supporting all of its
behaviour, with the one exception that previous exit values of 128 have
been changed to 3, in order to avoid any potential signed 8-bit issues.

The one change in the pkgsrc infrastructure is that the mk/fetch/fetch
script no longer sets a working default value for ${CHECKSUM}.  This is
not a problem in a pkgsrc environment as all of the required variables
are set correctly, but if there happen to be any users who are using
this script in a standalone environment, they will need to set it
accordingly.  This was probably required in many situations previously
anyway, as none of the script's environment variables were set, and
trying to support this would be fragile at best.
2020-08-27 11:45:45 +00:00
brook
fb1eef408e Add URLs to the MASTER_SITE_R_CRAN list
At some point CRAN added the https protocol to its repositories, but
this was never reflected in MASTER_SITE_R_CRAN.  Add analogues for
all the http sites with responsive https servers.
2020-08-09 16:50:13 +00:00
rillig
6fb4f703ff mk/fetch/fetch.mk: add keywords for "bmake help" 2020-08-04 21:54:46 +00:00
wiz
e597787c88 mk: do not check vulnerability when just fetching distfiles
This is intended to reduce the log output on ftp.NetBSD.org when
fetching all distfiles.

Also, we call the target in basically every step of package creation
(extract, patch, configure, build, install, package) - perhaps we should trim
this down some more.
2020-06-18 17:05:55 +00:00
js
9da7e8bc30 mk/fetch/fetch.mk: Use -f with ofhttp
pkgsrc wants to be able to override files when the checksum didn't
match.
2020-04-25 11:36:20 +00:00
js
c982f1310b mk/fetch/fetch.mk: Add ofhttp as fetcher 2020-04-25 11:21:06 +00:00
mef
c4fe655544 (mk/fetch/site.mk) ftp.cse.buffalo.edu has not been responded for a week 2020-04-21 23:00:14 +00:00
rillig
55e7cdf4de mk/fetch/fetch.mk: don't print double-slash in pathname 2020-03-30 22:44:26 +00:00
rillig
276bc87b64 mk/fetch: add SITES.* to the variables for the show-all-fetch target 2019-11-22 19:30:23 +00:00
rillig
af0890f6c6 mk/fetch: sort variables for bmake show-all-github 2019-11-04 19:05:18 +00:00
rillig
629d9c85db doc/guide: fill in the MASTER_SITE variables automatically
Keeping these two lists in sync is not something that humans should do.
2019-05-05 18:36:05 +00:00
wiz
ffb5aa8ddc *: mirror.switch.ch does not provide ftp/http service any longer 2019-02-04 09:36:41 +00:00
tnn
22ac324162 mk/sites: cpan.perl.org advertises Strict-Transport-Security 2019-01-14 09:46:58 +00:00
rillig
858c6cfd69 mk/misc: make configuration for show-all targets more configurable
Up to now, there was a central list of variable name patterns that
defined whether a variable was printed as a sorted list, as a list or as
a single value.

Now each variable group decides on its own which of the variables are
printed in which way, using the usual glob patterns. This is more
flexible since different files sometimes differ in their naming
conventions.

Two variable groups are added: license (for everything related to
LICENSE) and go (for lang/go).
2018-11-30 18:38:19 +00:00
maya
8d0e197859 Remove dead mozilla mirrors. ESR mirrors are not used. 2018-10-23 23:22:31 +00:00
maya
637d08c6eb drop ftp://ftp.gnu.org, they have a notice stating they intend to shut down
ftp services. (http remains)
2018-10-13 02:46:11 +00:00
maya
6c25dec844 Remove clauses 3,4 from TNF-only copyright blocks.
This is based on the decision The NetBSD Foundation made in 2008 to
do so, which was already applied to src.

This change has been applied to code which is likely not in other
repositories.

ok board@, reviewed by riastradh@
2018-08-22 20:48:36 +00:00
rillig
3866b420a6 mk/fetch/github.mk: document that PKGBASE influences GitHub variables
See https://mail-index.netbsd.org/pkgsrc-users/2018/07/thread1.html#027251
2018-08-01 18:01:32 +00:00
wiz
f367007762 *: gd.tuwien.ac.at/ftp.tuwien.ac.at is gone, remove it from various mastersites 2018-04-21 13:38:04 +00:00
rillig
81c4fbd48d mk/fetch: add documentation for show-all-github 2018-04-08 20:39:47 +00:00
maya
192d0ba9af Reorder MASTER_SITES_GNU:
- Put the main mirror at the lowest priority. it seems to be struggling.
- Add two mirrors that seem decent at the highest priority.
2018-02-18 19:31:56 +00:00
alnsn
cbe853ca6d Add FETCH_PROXY and FETCH_PROXY.{ftp,http,https}. 2018-01-11 08:54:37 +00:00
wiz
e9d95506d5 mk: Remove ftp.kddlabs.co.jp from sites, does not provide ftp service 2017-12-03 11:01:19 +00:00
tnn
e5b08a6647 define MASTER_SITE_CRATESIO (for downloading rust crates) 2017-11-08 11:58:02 +00:00
maya
58176b4461 remove ftp variant when http server already exists
pretty sure these are the same server but ftp is considerably slower on
a high latency connection
2017-09-29 06:17:12 +00:00
wiz
371e3139fb fetch/sites.mk: ftp.tuwien.ac.at times out, remove it 2017-09-17 10:55:17 +00:00
hauke
2711324a88 Update MASTER_SITE_XEMACS list. 2017-08-10 12:21:18 +00:00
jperkin
007ae9858c Do not perform checksum tests when explicitly calling a make target that
creates or updates the checksum.  Allows users to set FAILOVER_FETCH in
mk.conf unconditionally and be able to update distinfo files without
first having to remove them.
2017-08-01 13:15:32 +00:00
maya
2b5f2f1662 Reorder so cpan http mirrors are first
at least with nbftp, ftp downloads involve several slow back and forth 'cd'
commands and end up being noticeably slower for lots of little files
2017-06-12 07:13:35 +00:00
maya
e5900c2b7d replace ftp at kernel.org with http. it will shut down soon. 2017-01-28 00:23:35 +00:00
hauke
16ed67bd37 Update MASTER_SITE_XEMACS:
- remove three defunct mirrors
- remove xemacs.org - its hoster tux.org went down,
  and the master site as well as all the *.xemacs.org
  DNS entries are gone
- add two mirrors that run under their own domain name
2016-12-11 19:19:36 +00:00
wiz
db05d33e02 Remove ex-CPAN mirror. 2016-12-04 18:26:44 +00:00