pkgsrc/doc/guide/files/components.xml

715 lines
27 KiB
XML

<!-- $NetBSD: components.xml,v 1.50 2014/11/28 12:11:24 bsiegert Exp $ -->
<chapter id="components"> <?dbhtml filename="components.html"?>
<title>Package components - files, directories and contents</title>
<para>Whenever you're preparing a package, there are a number of
files involved which are described in the following
sections.</para>
<sect1 id="components.Makefile">
<title><filename>Makefile</filename></title>
<para>Building, installation and creation of a binary package are all
controlled by the package's <filename>Makefile</filename>.
The <filename>Makefile</filename> describes various things about
a package, for example from where to get it, how to configure,
build, and install it.</para>
<para>A package <filename>Makefile</filename> contains several
sections that describe the package.</para>
<para>In the first section there are the following variables, which
should appear exactly in the order given here. The order and
grouping of the variables is mostly historical and has no further
meaning.</para>
<itemizedlist>
<listitem><para><varname>DISTNAME</varname> is the basename of the
distribution file to be downloaded from the package's
website.</para></listitem>
<listitem><para><varname>PKGNAME</varname> is the name of the
package, as used by pkgsrc. You need to provide it if
<varname>DISTNAME</varname> (which is the default) is not a good
name for the package in pkgsrc or <varname>DISTNAME</varname> is not
provided (no distribution file is required). Usually it is the pkgsrc
directory name together with the version number. It must match the
regular expression
<varname>^[A-Za-z0-9][A-Za-z0-9-_.+]*$</varname>, that is, it
starts with a letter or digit, and contains only letters, digits,
dashes, underscores, dots and plus signs.</para></listitem>
<listitem><para><varname>CATEGORIES</varname> is a list of categories
which the package fits in. You can choose any of the top-level
directories of pkgsrc for it.</para>
<para>Currently the following values are available for
<varname>CATEGORIES</varname>. If more than
one is used, they need to be separated by spaces:</para>
<programlisting>
archivers cross geography meta-pkgs security
audio databases graphics misc shells
benchmarks devel ham multimedia sysutils
biology editors inputmethod net textproc
cad emulators lang news time
chat finance mail parallel wm
comms fonts math pkgtools www
converters games mbone print x11
</programlisting>
</listitem>
<listitem><para><varname>MASTER_SITES</varname>,
<varname>DYNAMIC_MASTER_SITES</varname>,
<varname>DIST_SUBDIR</varname>, <varname>EXTRACT_SUFX</varname>
and <varname>DISTFILES</varname> are discussed in detail in
<xref linkend="build.fetch"/>.</para></listitem>
</itemizedlist>
<para>The second section contains information about separately
downloaded patches, if any.
<itemizedlist>
<listitem><para><varname>PATCHFILES</varname>:
Name(s) of additional files that contain distribution patches.
There is no default. pkgsrc will look for them at
<varname>PATCH_SITES</varname>.
They will automatically be uncompressed before patching if
the names end with <filename>.gz</filename> or
<filename>.Z</filename>.</para>
</listitem>
<listitem><para><varname>PATCH_SITES</varname>:
Primary location(s) for distribution patch files (see
<varname>PATCHFILES</varname> above) if not found locally.</para>
</listitem>
<listitem><para><varname>PATCH_DIST_STRIP</varname>:
an argument to &man.patch.1; that sets the pathname strip count to
help find the correct files to patch. It defaults to
<command>-p0</command>.</para>
</listitem>
</itemizedlist></para>
<para>The third section contains the following variables.
<itemizedlist>
<listitem><para><varname>MAINTAINER</varname> is the email
address of the person who feels responsible for this package,
and who is most likely to look at problems or questions regarding
this package which have been reported with &man.send-pr.1;.
Other developers may contact the <varname>MAINTAINER</varname>
before making changes to the package, but are not required to
do so. When packaging a new program, set <varname>MAINTAINER</varname>
to yourself. If you really can't maintain the package for future
updates, set it to
<email>pkgsrc-users@NetBSD.org</email>.</para></listitem>
<listitem><para><varname>OWNER</varname> should be used instead
of <varname>MAINTAINER</varname> when you do not want other
developers to update or change the package without contacting
you first. A package Makefile should contain one of
<varname>MAINTAINER</varname> or <varname>OWNER</varname>, but
not both. </para></listitem>
<listitem><para><varname>HOMEPAGE</varname> is a URL where users can
find more information about the package.</para></listitem>
<listitem><para><varname>COMMENT</varname> is a one-line
description of the package (should not include the package
name).</para></listitem>
</itemizedlist></para>
<para>Other variables that affect the build:
<itemizedlist>
<listitem>
<para><varname>WRKSRC</varname>: The directory where the
interesting distribution files of the package are found. The
default is <filename>${WRKDIR}/${DISTNAME}</filename>, which
works for most packages.</para>
<para>If a package doesn't create a subdirectory for itself
(most GNU software does, for instance), but extracts itself in
the current directory, you should set
<varname>WRKSRC=${WRKDIR}</varname>.</para>
<para>If a package doesn't create a subdirectory with the
name of <varname>DISTNAME</varname> but some different name,
set <varname>WRKSRC</varname> to point to the proper name in
<filename>${WRKDIR}</filename>, for example
<varname>WRKSRC=${WRKDIR}/${DISTNAME}/unix</varname>. See
<filename role="pkg">lang/tcl</filename> and <filename
role="pkg">x11/tk</filename> for other examples.</para>
<para>The name of the working directory created by pkgsrc is
taken from the <varname>WRKDIR_BASENAME</varname>
variable. By default, its value is
<filename>work</filename>. If you want to use the same
pkgsrc tree for building different kinds of binary packages,
you can change the variable according to your needs. Two
other variables handle common cases of setting
<varname>WRKDIR_BASENAME</varname> individually. If
<varname>OBJHOSTNAME</varname> is defined in
&mk.conf;, the first component of
the host's name is attached to the directory name. If
<varname>OBJMACHINE</varname> is defined, the platform name
is attached, which might look like
<filename>work.i386</filename> or
<filename>work.sparc</filename>.</para>
</listitem>
</itemizedlist></para>
<para>Please pay attention to the following gotchas:</para>
<itemizedlist>
<listitem>
<para>Add <varname>MANCOMPRESSED</varname> if man pages are
installed in compressed form by the package. For packages using
BSD-style makefiles which honor MANZ, there is
<varname>MANCOMPRESSED_IF_MANZ</varname>.</para>
</listitem>
<listitem>
<para>Replace <filename>/usr/local</filename> with
<quote>${PREFIX}</quote> in all files (see patches,
below).</para>
</listitem>
<listitem>
<para>If the package installs any info files, see <xref
linkend="faq.info-files"/>.</para>
</listitem>
</itemizedlist>
</sect1>
<sect1 id="components.distinfo">
<title><filename>distinfo</filename></title>
<para>The <filename>distinfo</filename> file contains the message
digest, or checksum, of each distfile needed for the package. This
ensures that the distfiles retrieved from the Internet have not been
corrupted during transfer or altered by a malign force to introduce
a security hole. Due to recent rumor about weaknesses of digest
algorithms, all distfiles are protected using both SHA1 and RMD160
message digests, as well as the file size.</para>
<para>The <filename>distinfo</filename> file also contains the
checksums for all the patches found in the
<filename>patches</filename> directory (see <xref
linkend="components.patches"/>).</para>
<para>To regenerate the <filename>distinfo</filename> file, use the
<command>make makedistinfo</command> or <command>make mdi</command>
command.</para>
<para>Some packages have different sets of distfiles depending on
the platform, for example <filename
role="pkg">lang/openjdk7</filename>. These are kept in the same
<filename>distinfo</filename> file and care should be taken when
upgrading such a package to ensure distfile information is not
lost.</para>
</sect1>
<sect1 id="components.patches">
<title>patches/*</title>
<para>Many packages still don't work out-of-the box on the various
platforms that are supported by pkgsrc. Therefore, a number of custom
patch files are needed to make the package work. These patch files are
found in the <filename>patches/</filename> directory.</para>
<para>In the <emphasis>patch</emphasis> phase, these patches are
applied to the files in <varname>WRKSRC</varname> directory after
extracting them, in <ulink
url="http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_13_03">alphabetic
order</ulink>.</para>
<sect2 id="components.patch.structure">
<title>Structure of a single patch file</title>
<para>The <filename>patch-*</filename> files should be in
<command>diff -bu</command> format, and apply without a fuzz to avoid
problems. (To force patches to apply with fuzz you can set
<varname>PATCH_FUZZ_FACTOR=-F2</varname>). Furthermore, each patch
should contain only changes for a single file, and no file should be
patched by more than one patch file. This helps to keep future
modifications simple.</para>
<para>Each patch file is structured as follows: In the first line,
there is the RCS Id of the patch itself. The second line should be
empty for aesthetic reasons. After that, there should be a comment for
each change that the patch does. There are a number of standard
cases:</para>
<itemizedlist>
<listitem><para>Patches for commonly known vulnerabilities should
mention the vulnerability ID (CAN, CVE).</para></listitem>
<listitem><para>Patches that change source code should mention the
platform and other environment (for example, the compiler) that the
patch is needed for.</para></listitem>
</itemizedlist>
<para>In all, the patch should be commented so that any
developer who knows the code of the application can make some use of
the patch. Special care should be taken for the upstream developers,
since we generally want that they accept our patches, so we have less
work in the future.</para>
</sect2>
<sect2 id="components.patches.caveats">
<title>Creating patch files</title>
<para>One important thing to mention is to pay attention that no RCS
IDs get stored in the patch files, as these will cause problems when
later checked into the &os; CVS tree. Use the
<command>pkgdiff</command> command from the <filename
role="pkg">pkgtools/pkgdiff</filename> package to avoid these
problems.</para>
<para>For even more automation, we recommend using
<command>mkpatches</command> from the same package to make a
whole set of patches. You just have to backup files before you
edit them to <filename>filename.orig</filename>, e.g. with
<command>cp -p filename filename.orig</command> or, easier, by
using <command>pkgvi</command> again from the same package. If
you upgrade a package this way, you can easily compare the new
set of patches with the previously existing one with
<command>patchdiff</command>. The files in <filename>patches</filename>
are replaced by new files, so carefully check if you want to take all
the changes.</para>
<para>When you have finished a package, remember to generate
the checksums for the patch files by using the <command>make
makepatchsum</command> command, see <xref
linkend="components.distinfo"/>.</para>
<para>When adding a patch that corrects a problem in the
distfile (rather than e.g. enforcing pkgsrc's view of where
man pages should go), send the patch as a bug report to the
maintainer. This benefits non-pkgsrc users of the package,
and usually makes it possible to remove the patch in future
version.</para>
<para>The file names of the patch files are usually of the form
<filename>patch-<replaceable>path_to_file__with__underscores.c</replaceable></filename>.
Many packages still use the previous convention
<filename>patch-<replaceable>[a-z][a-z]</replaceable></filename>,
but new patches should be of the form containing the filename.
<command>mkpatches</command> included in <filename
role="pkg">pkgtools/pkgdiff</filename> takes care of the name
automatically.</para>
</sect2>
<sect2 id="components.patches.sources">
<title>Sources where the patch files come from</title>
<para>If you want to share patches between multiple packages
in pkgsrc, e.g. because they use the same distfiles, set
<varname>PATCHDIR</varname> to the path where the patch files
can be found, e.g.:</para>
<programlisting>
PATCHDIR= ${.CURDIR}/../xemacs/patches
</programlisting>
<para>Patch files that are distributed by the author or other
maintainers can be listed in
<varname>PATCHFILES</varname>.</para>
<para>If it is desired to store any patches that should not be
committed into pkgsrc, they can be kept outside the pkgsrc
tree in the <filename>$LOCALPATCHES</filename> directory. The
directory tree there is expected to have the same
<quote>category/package</quote> structure as pkgsrc, and
patches are expected to be stored inside these dirs (also
known as <filename>$LOCALPATCHES/$PKGPATH</filename>). For
example, if you want to keep a private patch for
<filename>pkgsrc/graphics/png</filename>, keep it in
<filename>$LOCALPATCHES/graphics/png/mypatch</filename>. All
files in the named directory are expected to be patch files,
and <emphasis>they are applied after pkgsrc patches are
applied</emphasis>.</para>
</sect2>
<sect2 id="components.patches.guidelines">
<title>Patching guidelines</title>
<para>When fixing a portability issue in the code do not use
preprocessor magic to check for the current operating system nor
platform. Doing so hurts portability to other platforms because
the OS-specific details are not abstracted appropriately.</para>
<para>The general rule to follow is: instead of checking for the
operating system the application is being built on, check for the
specific <emphasis>features</emphasis> you need. For example,
instead of assuming that kqueue is available under NetBSD and
using the <varname>__NetBSD__</varname> macro to conditionalize
kqueue support, add a check that detects kqueue itself &mdash;
yes, this generally involves patching the
<command>configure</command> script. There is absolutely nothing
that prevents some OSes from adopting interfaces from other OSes
(e.g. Linux implementing kqueue), something that the above checks
cannot take into account.</para>
<para>Of course, checking for features generally involves more
work on the developer's side, but the resulting changes are
cleaner and there are chances they will work on many other
platforms. Not to mention that there are higher chances of being
later integrated into the mainstream sources. Remember:
<emphasis>It doesn't work unless it is right!</emphasis></para>
<para>Some typical examples:</para>
<table id="patch-examples">
<title>Patching examples</title>
<tgroup cols="3">
<thead>
<row>
<entry>Where</entry>
<entry>Incorrect</entry>
<entry>Correct</entry>
</row>
</thead>
<tbody>
<row>
<entry>configure script</entry>
<entry>
<programlisting>
case ${target_os} in
netbsd*) have_kvm=yes ;;
*) have_kvm=no ;;
esac
</programlisting>
</entry>
<entry>
<programlisting>
AC_CHECK_LIB(kvm, kvm_open, have_kvm=yes, have_kvm=no)
</programlisting>
</entry>
</row>
<row>
<entry>C source file</entry>
<entry>
<programlisting>
#if defined(__NetBSD__)
# include &lt;sys/event.h&gt;
#endif
</programlisting>
</entry>
<entry>
<programlisting>
#if defined(HAVE_SYS_EVENT_H)
# include &lt;sys/event.h&gt;
#endif
</programlisting>
</entry>
</row>
<row>
<entry>C source file</entry>
<entry>
<programlisting>
int
monitor_file(...)
{
#if defined(__NetBSD__)
int fd = kqueue();
...
#else
...
#endif
}
</programlisting>
</entry>
<entry>
<programlisting>
int
monitor_file(...)
{
#if defined(HAVE_KQUEUE)
int fd = kqueue();
...
#else
...
#endif
}
</programlisting>
</entry>
</row>
</tbody>
</tgroup>
</table>
<para>For more information, please read the <emphasis>Making
packager-friendly software</emphasis> article (<ulink
url="http://www.onlamp.com/pub/a/onlamp/2005/03/31/packaging.html">part
1</ulink>, <ulink
url="http://www.oreillynet.com/pub/a/onlamp/2005/04/28/packaging2.html">part
2</ulink>). It summarizes multiple details on how to make
software easier to package; all the suggestions in it were
collected from our experience in pkgsrc work, so they are possibly
helpful when creating patches too.</para>
</sect2>
<sect2 id="components.patches.feedback">
<title>Feedback to the author</title>
<para>Always, always, <emphasis role="strong">always</emphasis>
feed back any <emphasis>portability fixes</emphasis> or
improvements you do to a package to the mainstream developers.
This is the only way to get their attention on portability issues
and to ensure that future versions can be built out-of-the box on
NetBSD. Furthermore, any user that gets newer distfiles will get
the fixes straight from the packaged code.</para>
<para>This generally involves cleaning up the patches
(because sometimes the patches that are
added to pkgsrc are quick hacks), filing bug reports in the
appropriate trackers for the projects and working with the
mainstream authors to accept your changes. It is
<emphasis>extremely important</emphasis> that you do it so that
the packages in pkgsrc are kept simple and thus further changes
can be done without much hassle.</para>
<para>When you have done this, please add a URL to the upstream
bug report to the patch comment.</para>
<para>Support the idea of free software!</para>
</sect2>
</sect1>
<sect1 id="other-mandatory-files">
<title>Other mandatory files</title>
<variablelist>
<varlistentry>
<term><filename>DESCR</filename></term>
<listitem>
<para>A multi-line description of the piece of software. This should include
any credits where they are due. Please bear in mind that others do not
share your sense of humour (or spelling idiosyncrasies), and that others
will read everything that you write here.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><filename>PLIST</filename></term>
<listitem>
<para>This file governs the files that are installed on your
system: all the binaries, manual pages, etc. There are other
directives which may be entered in this file, to control the
creation and deletion of directories, and the location of
inserted files. See <xref linkend="plist"/> for more
information.</para>
</listitem>
</varlistentry>
</variablelist>
</sect1>
<sect1 id="components.optional">
<title>Optional files</title>
<sect2 id="components.optional.bin">
<title>Files affecting the binary package</title>
<variablelist>
<varlistentry>
<term><filename>INSTALL</filename></term>
<listitem>
<para>This shell script is invoked twice by &man.pkg.add.1;.
First time after package extraction and before files are
moved in place, the second time after the files to install
are moved in place. This can be used to do any custom
procedures not possible with @exec commands in
<filename>PLIST</filename>. See &man.pkg.add.1; and
&man.pkg.create.1; for more information. See also <xref
linkend="files-and-dirs-outside-prefix" />.
Please note that you can modify variables in it easily by using
<varname>FILES_SUBST</varname> in the package's
<filename>Makefile</filename>:</para>
<programlisting>
FILES_SUBST+= SOMEVAR="somevalue"
</programlisting>
<para>replaces "@SOMEVAR@" with <quote>somevalue</quote> in the
<filename>INSTALL</filename>. By default, substitution is
performed for <varname>PREFIX</varname>,
<varname>LOCALBASE</varname>, <varname>X11BASE</varname>,
<varname>VARBASE</varname>, and a few others, type
<command>make help topic=FILES_SUBST</command> for a
complete list.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><filename>DEINSTALL</filename></term>
<listitem>
<para>This script is executed before and after any files are removed. It is
this script's responsibility to clean up any additional messy details
around the package's installation, since all pkg_delete knows is how to
delete the files created in the original distribution.
See &man.pkg.delete.1;
and &man.pkg.create.1; for more information.
The same methods to replace variables can be used as for
the <filename>INSTALL</filename> file.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><filename>MESSAGE</filename></term>
<listitem>
<para>This file is displayed after installation of the package.
Useful for things like legal notices on almost-free
software and hints for updating config files after
installing modules for apache, PHP etc.
Please note that you can modify variables in it easily by using
<varname>MESSAGE_SUBST</varname> in the package's
<filename>Makefile</filename>:</para>
<programlisting>
MESSAGE_SUBST+= SOMEVAR="somevalue"
</programlisting>
<para>replaces "${SOMEVAR}" with <quote>somevalue</quote> in
<filename>MESSAGE</filename>. By default, substitution is
performed for <varname>PKGNAME</varname>,
<varname>PKGBASE</varname>, <varname>PREFIX</varname>,
<varname>LOCALBASE</varname>, <varname>X11PREFIX</varname>,
<varname>X11BASE</varname>,
<varname>PKG_SYSCONFDIR</varname>,
<varname>ROOT_GROUP</varname>, and
<varname>ROOT_USER</varname>.</para>
<para>You can display a different or additional files by
setting the <varname>MESSAGE_SRC</varname> variable. Its
default is <filename>MESSAGE</filename>, if the file
exists.</para>
</listitem>
</varlistentry>
<varlistentry>
<term><filename>ALTERNATIVES</filename></term>
<listitem>
<para>This file is used by the alternatives framework.
It creates, configures, and destroys generic wrappers used to
run programs with similar interfaces.
See pkg_alternatives(8) from pkgtools/pkg_alternatives
for more information.</para>
<para>Each line of the file contains two filenames, first
the wrapper and then the alternative provided by the package.
Both paths are relative to <varname>PREFIX</varname>.</para>
</listitem>
</varlistentry>
</variablelist>
</sect2>
<sect2 id="components.optional.build">
<title>Files affecting the build process</title>
<variablelist>
<varlistentry><term><filename>Makefile.common</filename></term>
<listitem><para>This file contains arbitrary things that could
also go into a <filename>Makefile</filename>, but its purpose is
to be used by more than one package. This file should only be
used when the packages that will use the file are known in
advance. For other purposes it is often better to write a
<filename>*.mk</filename> file and give it a good name that
describes what it does.</para></listitem></varlistentry>
<varlistentry><term><filename>buildlink3.mk</filename></term>
<listitem><para>This file contains the dependency information
for the buildlink3 framework (see <xref
linkend="buildlink"/>).</para></listitem></varlistentry>
<varlistentry><term><filename>hacks.mk</filename></term>
<listitem><para>This file contains workarounds for compiler bugs
and similar things. It is included automatically by the pkgsrc
infrastructure, so you don't need an extra
<literal>.include</literal> line for
it.</para></listitem></varlistentry>
<varlistentry><term><filename>options.mk</filename></term>
<listitem><para>This file contains the code for the
package-specific options (see <xref linkend="options"/>) that can be
selected by the user. If a package has only one or two options,
it is equally acceptable to put the code directly into the
<filename>Makefile</filename>.</para></listitem></varlistentry>
</variablelist>
</sect2>
<sect2 id="components.optional.none">
<title>Files affecting nothing at all</title>
<variablelist>
<varlistentry><term><filename>README*</filename></term>
<listitem><para>These files do not take place in the creation of
a package and thus are purely informative to the package
developer.</para></listitem></varlistentry>
<varlistentry><term><filename>TODO</filename></term>
<listitem><para>This file contains things that need to be done
to make the package even
better.</para></listitem></varlistentry>
</variablelist>
</sect2>
</sect1>
<sect1 id="work-dir">
<title><filename>work*</filename></title>
<para>When you type <command>make</command>, the distribution files are
unpacked into the directory denoted by
<varname>WRKDIR</varname>. It can be removed by running
<command>make clean</command>. Besides the sources, this
directory is also used to keep various timestamp files.
The directory gets <emphasis>removed completely</emphasis> on clean.
The default is <filename>${.CURDIR}/work</filename>
or <filename>${.CURDIR}/work.${MACHINE_ARCH}</filename>
if <varname>OBJMACHINE</varname> is set.</para>
</sect1>
<sect1 id="files-dir">
<title><filename>files/*</filename></title>
<para>If you have any files that you wish to be placed in the package prior
to configuration or building, you could place these files here and use
a <command>${CP}</command> command in the
<quote>pre-configure</quote> target to achieve
this. Alternatively, you could simply diff the file against
<filename>/dev/null</filename> and use the patch mechanism to manage
the creation of this file.</para>
<para>If you want to share files in this way with other
packages, set the <varname>FILESDIR</varname> variable to point
to the other package's <filename>files</filename> directory,
e.g.:</para>
<programlisting>
FILESDIR=${.CURDIR}/../xemacs/files
</programlisting>
</sect1>
</chapter>