pkgsrc/doc/guide/files/buildlink.xml
rillig 041e06d511 Changed all <programlisting> environments so that the first relevant
column starts in column 1. Everything else is left to the style sheet.
2007-06-01 11:07:24 +00:00

565 lines
22 KiB
XML

<!-- $NetBSD: buildlink.xml,v 1.24 2007/06/01 11:07:24 rillig Exp $ -->
<chapter id="buildlink">
<title>Buildlink methodology</title>
<para>Buildlink is a framework in pkgsrc that controls what headers and libraries
are seen by a package's configure and build processes. This is implemented
in a two step process:</para>
<orderedlist>
<listitem>
<para>Symlink headers and libraries for dependencies into
<varname>BUILDLINK_DIR</varname>, which by default is a subdirectory
of <varname>WRKDIR</varname>.</para>
</listitem>
<listitem>
<para>Create wrapper scripts that are used in place of the normal compiler
tools that translate <option>-I${LOCALBASE}/include</option> and
<option>-L${LOCALBASE}/lib</option> into references to
<varname>BUILDLINK_DIR</varname>. The wrapper scripts also make
native compiler on some operating systems look like GCC, so that
packages that expect GCC won't require modifications to build with
those native compilers.</para>
</listitem>
</orderedlist>
<para>This normalizes the environment in which a package is built so that the
package may be built consistently despite what other software may be
installed. Please note that the normal system header and library paths,
e.g. <filename>/usr/include</filename>,
<filename>/usr/lib</filename>, etc., are always searched -- buildlink3 is
designed to insulate the package build from non-system-supplied
software.</para>
<sect1 id="converting-to-buildlink3">
<title>Converting packages to use buildlink3</title>
<para>The process of converting packages to use the buildlink3
framework (<quote>bl3ifying</quote>) is fairly straightforward.
The things to keep in mind are:</para>
<orderedlist>
<listitem>
<para>Ensure that the build always calls the wrapper scripts
instead of the actual toolchain. Some packages are tricky,
and the only way to know for sure is the check
<filename>${WRKDIR}/.work.log</filename> to see if the
wrappers are being invoked.</para>
</listitem>
<listitem>
<para>Don't override <varname>PREFIX</varname> from within
the package Makefile, e.g. Java VMs, standalone shells,
etc., because the code to symlink files into
<filename>${BUILDLINK_DIR}</filename> looks for files
relative to <quote>pkg_info -qp <replaceable>pkgname</replaceable></quote>.
</para>
</listitem>
<listitem>
<para>Remember that <emphasis>only</emphasis> the
<filename>buildlink3.mk</filename> files that you list in a
package's Makefile are added as dependencies for that package.
</para>
</listitem>
</orderedlist>
<para>If a dependency on a particular package is required for its libraries and
headers, then we replace:</para>
<programlisting>
DEPENDS+= foo>=1.1.0:../../category/foo
</programlisting>
<para>with</para>
<programlisting>
.include "../../category/foo/buildlink3.mk"
</programlisting>
<para>The buildlink3.mk files usually define the required dependencies.
If you need a newer version of the dependency when using buildlink3.mk
files, then you can define it in your Makefile; for example:</para>
<programlisting>
BUILDLINK_API_DEPENDS.foo+= foo>=1.1.0
.include "../../category/foo/buildlink3.mk"
</programlisting>
<para>There are several <filename>buildlink3.mk</filename>
files in <filename>pkgsrc/mk</filename>
that handle special package issues:</para>
<itemizedlist>
<listitem>
<para><filename>bdb.buildlink3.mk</filename> chooses either
the native or a pkgsrc Berkeley DB implementation based on
the values of <varname>BDB_ACCEPTED</varname> and
<varname>BDB_DEFAULT</varname>.</para>
</listitem>
<listitem>
<para><filename>curses.buildlink3.mk</filename>: If the system
comes with neither Curses nor NCurses, this will take care
to install the <filename
role="pkg">devel/ncurses</filename> package.</para>
</listitem>
<listitem>
<para><filename>krb5.buildlink3.mk</filename> uses the value
of <varname>KRB5_ACCEPTED</varname> to choose between
adding a dependency on Heimdal or MIT-krb5 for packages that
require a Kerberos 5 implementation.</para>
</listitem>
<listitem>
<para><filename>motif.buildlink3.mk</filename> checks for a
system-provided Motif installation or adds a dependency on
<filename role="pkg">x11/lesstif</filename> or <filename
role="pkg">x11/openmotif</filename>. The user can set
<varname>MOTIF_TYPE</varname> to <quote>dt</quote>,
<quote>lesstif</quote>, or <quote>openmotif</quote> to choose
which Motif version will be used.</para>
</listitem>
<listitem>
<para><filename>oss.buildlink3.mk</filename> defines several
variables that may be used by packages that use the
Open Sound System (OSS) API.</para>
</listitem>
<listitem>
<para><filename>pgsql.buildlink3.mk</filename> will accept
either Postgres 8.0, 8.1, or 8.2, whichever is found installed. See
the file for more information.</para>
</listitem>
<listitem>
<para><filename>pthread.buildlink3.mk</filename> uses the value of
<varname>PTHREAD_OPTS</varname> and checks for native pthreads or adds
a dependency on <filename
role="pkg">devel/pth</filename> as needed.</para>
</listitem>
<listitem>
<para><filename>xaw.buildlink3.mk</filename> uses the value of
<varname>XAW_TYPE</varname> to choose a particular Athena widgets
library.</para>
</listitem>
</itemizedlist>
<para>The comments in those <filename>buildlink3.mk</filename>
files provide a more complete
description of how to use them properly.</para>
</sect1>
<sect1 id="creating-buildlink3.mk">
<title>Writing <filename>buildlink3.mk</filename> files</title>
<para>A package's <filename>buildlink3.mk</filename> file is
included by Makefiles to indicate the need to compile and link
against header files and libraries provided by the package. A
<filename>buildlink3.mk</filename> file should always provide
enough information to add the correct type of dependency
relationship and include any other
<filename>buildlink3.mk</filename> files that it needs to find
headers and libraries that it needs in turn.</para>
<para>To generate an initial <filename>buildlink3.mk</filename>
file for further editing, Rene Hexel's <filename
role="pkg">pkgtools/createbuildlink</filename>
package is highly recommended. For most packages, the following
command will generate a good starting point for
<filename>buildlink3.mk</filename> files:</para>
<screen>
&cprompt; <userinput>cd pkgsrc/<replaceable>category</replaceable>/<replaceable>pkgdir</replaceable>
&cprompt; createbuildlink &gt;buildlink3.mk</userinput>
</screen>
<sect2 id="anatomy-of-bl3">
<title>Anatomy of a buildlink3.mk file</title>
<para>The following real-life example
<filename>buildlink3.mk</filename> is taken
from <filename>pkgsrc/graphics/tiff</filename>:</para>
<programlisting>
# &#36;NetBSD: buildlink3.mk,v 1.7 2004/03/18 09:12:12 jlam Exp &#36;
BUILDLINK_DEPTH:= ${BUILDLINK_DEPTH}+
TIFF_BUILDLINK3_MK:= ${TIFF_BUILDLINK3_MK}+
.if ${BUILDLINK_DEPTH} == "+"
BUILDLINK_DEPENDS+= tiff
.endif
BUILDLINK_PACKAGES:= ${BUILDLINK_PACKAGES:Ntiff}
BUILDLINK_PACKAGES+= tiff
.if ${TIFF_BUILDLINK3_MK} == "+"
BUILDLINK_API_DEPENDS.tiff+= tiff>=3.6.1
BUILDLINK_PKGSRCDIR.tiff?= ../../graphics/tiff
.endif # TIFF_BUILDLINK3_MK
.include "../../devel/zlib/buildlink3.mk"
.include "../../graphics/jpeg/buildlink3.mk"
BUILDLINK_DEPTH:= ${BUILDLINK_DEPTH:S/+$//}
</programlisting>
<para>The header and footer manipulate
<varname>BUILDLINK_DEPTH</varname>, which is common across all
<filename>buildlink3.mk</filename> files and is used to track
at what depth we are including
<filename>buildlink3.mk</filename> files.</para>
<para>The first section controls if the dependency on
<replaceable>pkg</replaceable> is added.
<varname>BUILDLINK_DEPENDS</varname> is the global list of
packages for which dependencies are added by
buildlink3.</para>
<para>The second section advises pkgsrc that the
<filename>buildlink3.mk</filename> file for
<replaceable>pkg</replaceable> has been included at some point.
<varname>BUILDLINK_PACKAGES</varname> is the global list of
packages for which <filename>buildlink3.mk</filename> files
have been included. It must <emphasis>always</emphasis> be
appended to within a <filename>buildlink3.mk</filename>
file.</para>
<para>The third section is protected from multiple inclusion
and controls how the dependency on <replaceable>pkg</replaceable> is
added. Several important variables are set in the section:</para>
<itemizedlist>
<listitem>
<para><varname>BUILDLINK_API_DEPENDS.<replaceable>pkg</replaceable></varname>
is the actual dependency recorded in the installed
package; this should always be set using
<command>+=</command> to ensure that
we're appending to any pre-existing list of values. This
variable should be set to the first version of the
package that had an API change.
</para>
</listitem>
<listitem>
<para><varname>BUILDLINK_PKGSRCDIR.<replaceable>pkg</replaceable></varname>
is the location of the <replaceable>pkg</replaceable>
pkgsrc directory.</para>
</listitem>
<listitem>
<para><varname>BUILDLINK_DEPMETHOD.<replaceable>pkg</replaceable></varname>
(not shown above) controls whether we use
<varname>BUILD_DEPENDS</varname> or
<varname>DEPENDS</varname> to add the dependency on
<replaceable>pkg</replaceable>. The build dependency is
selected by setting
<varname>BUILDLINK_DEPMETHOD.<replaceable>pkg</replaceable></varname>
to <quote>build</quote>. By default, the full dependency is
used.</para>
</listitem>
<listitem>
<para><varname>BUILDLINK_INCDIRS.<replaceable>pkg</replaceable></varname>
and
<varname>BUILDLINK_LIBDIRS.<replaceable>pkg</replaceable></varname>
(not shown above) are lists of subdirectories of
<filename>${BUILDLINK_PREFIX.<replaceable>pkg</replaceable>}</filename>
to add to the header and library search paths. These
default to <quote>include</quote> and <quote>lib</quote>
respectively.</para>
</listitem>
<listitem>
<para><varname>BUILDLINK_CPPFLAGS.<replaceable>pkg</replaceable></varname>
(not shown above) is the list of preprocessor flags to add
to <varname>CPPFLAGS</varname>, which are passed on to the
configure and build phases. The <quote>-I</quote> option
should be avoided and instead be handled using
<varname>BUILDLINK_INCDIRS.<replaceable>pkg</replaceable></varname> as
above.</para>
</listitem>
</itemizedlist>
<para>The following variables are all optionally defined within
this second section (protected against multiple inclusion) and
control which package files are symlinked into
<filename>${BUILDLINK_DIR}</filename> and how their names are
transformed during the symlinking:</para>
<itemizedlist>
<listitem>
<para><varname>BUILDLINK_FILES.<replaceable>pkg</replaceable></varname>
(not shown above) is a shell glob pattern relative to
<filename>${BUILDLINK_PREFIX.<replaceable>pkg</replaceable>}</filename>
to be symlinked into
<filename>${BUILDLINK_DIR}</filename>,
e.g. <filename>include/*.h</filename>.</para>
</listitem>
<listitem>
<para><varname>BUILDLINK_FILES_CMD.<replaceable>pkg</replaceable></varname>
(not shown above) is a shell pipeline that
outputs to stdout a list of files relative to
<filename>${BUILDLINK_PREFIX.<replaceable>pkg</replaceable>}</filename>.
The resulting files are to be symlinked
into <filename>${BUILDLINK_DIR}</filename>. By default,
this takes the <filename>+CONTENTS</filename> of a
<replaceable>pkg</replaceable> and filters it through
<varname>${BUILDLINK_CONTENTS_FILTER.<replaceable>pkg</replaceable>}</varname>.</para>
</listitem>
<listitem>
<para><varname>BUILDLINK_CONTENTS_FILTER.<replaceable>pkg</replaceable></varname>
(not shown above) is a filter command that filters
<filename>+CONTENTS</filename> input into a list of files
relative to
<filename>${BUILDLINK_PREFIX.<replaceable>pkg</replaceable>}</filename>
on stdout. By default for overwrite packages,
<varname>BUILDLINK_CONTENTS_FILTER.<replaceable>pkg</replaceable></varname>
outputs the contents of the <filename>include</filename>
and <filename>lib</filename> directories in the package
<filename>+CONTENTS</filename>, and for pkgviews packages,
it outputs any libtool archives in
<filename>lib</filename> directories.</para>
</listitem>
<listitem>
<para><varname>BUILDLINK_FNAME_TRANSFORM.<replaceable>pkg</replaceable></varname>
(not shown above) is a list of sed arguments used to
transform the name of the source filename into a
destination filename, e.g. <command>-e
"s|/curses.h|/ncurses.h|g"</command>.</para>
</listitem>
</itemizedlist>
<para>The last section includes any
<filename>buildlink3.mk</filename> needed for
<replaceable>pkg</replaceable>'s library dependencies.
Including these <filename>buildlink3.mk</filename> files
means that the headers and libraries for these
dependencies are also symlinked into
<filename>${BUILDLINK_DIR}</filename>
whenever the <replaceable>pkg</replaceable>
<filename>buildlink3.mk</filename>
file is included.</para>
</sect2>
<sect2 id="updating-buildlink-depends">
<title>Updating <varname>BUILDLINK_API_DEPENDS.<replaceable>pkg</replaceable></varname> in <filename>buildlink3.mk</filename> files</title>
<para>The situation that requires increasing the dependency listed in
<varname>BUILDLINK_API_DEPENDS.<replaceable>pkg</replaceable></varname>
after a package update is when the API or interface to the
header files change.</para>
<para>In this case,
<varname>BUILDLINK_API_DEPENDS.<replaceable>pkg</replaceable></varname>
should be adjusted to require at least the new package
version. In some cases, the packages that depend on this new
version may need their <varname>PKGREVISION</varname>s
increased and, if they have <filename>buildlink3.mk</filename>
files, their
<varname>BUILDLINK_API_DEPENDS.<replaceable>pkg</replaceable></varname>
adjusted, too. This is needed so pkgsrc will require the
correct package dependency and not settle for an older one
when building the source.</para>
<para><varname>BUILDLINK_ABI_DEPENDS.<replaceable>pkg</replaceable></varname>
should be increased when the binary interface or sonames
(major number of the library version) of any installed
shared libraries change. This is needed so that binary
packages made using it will require the correct package
dependency and not settle for an older one which will not
contain the necessary shared libraries.</para>
<para>See <xref linkend="dependencies"/> for
more information about dependencies on other packages,
including the <varname>BUILDLINK_ABI_DEPENDS</varname> and
<varname>ABI_DEPENDS</varname> definitions.</para>
<para>Please take careful consideration before adjusting
<varname>BUILDLINK_API_DEPENDS.<replaceable>pkg</replaceable></varname>
or
<varname>BUILDLINK_ABI_DEPENDS.<replaceable>pkg</replaceable></varname>
as we don't want to cause unneeded package deletions and
rebuilds. In many cases, new versions of packages work just
fine with older dependencies.</para>
<para>Also it is not needed to set
<varname>BUILDLINK_ABI_DEPENDS.<replaceable>pkg</replaceable></varname>
when it is identical to
<varname>BUILDLINK_API_DEPENDS.<replaceable>pkg</replaceable></varname>. </para>
</sect2>
</sect1>
<sect1 id="writing-builtin.mk">
<title>Writing <filename>builtin.mk</filename> files</title>
<para>Some packages in pkgsrc install headers and libraries that
coincide with headers and libraries present in the base system.
Aside from a <filename>buildlink3.mk</filename> file, these
packages should also include a <filename>builtin.mk</filename>
file that includes the necessary checks to decide whether using
the built-in software or the pkgsrc software is
appropriate.</para>
<para>The only requirements of a builtin.mk file for
<replaceable>pkg</replaceable> are:</para>
<orderedlist>
<listitem>
<para>It should set
<varname>USE_BUILTIN.<replaceable>pkg</replaceable></varname>
to either <quote>yes</quote> or <quote>no</quote>
after it is included.</para>
</listitem>
<listitem>
<para>It should <emphasis>not</emphasis> override any
<varname>USE_BUILTIN.<replaceable>pkg</replaceable></varname>
which is already set before the
<filename>builtin.mk</filename> file is included.</para>
</listitem>
<listitem>
<para>It should be written to allow multiple inclusion. This
is <emphasis>very</emphasis> important and takes careful
attention to <filename>Makefile</filename> coding.</para>
</listitem>
</orderedlist>
<sect2 id="anatomy-of-builtin.mk">
<title>Anatomy of a <filename>builtin.mk</filename> file</title>
<para>The following is the recommended template for builtin.mk
files:</para>
<programlisting>
.if !defined(IS_BUILTIN.foo)
#
# IS_BUILTIN.foo is set to "yes" or "no" depending on whether "foo"
# genuinely exists in the system or not.
#
IS_BUILTIN.foo?= no
# BUILTIN_PKG.foo should be set here if "foo" is built-in and its package
# version can be determined.
#
. if !empty(IS_BUILTIN.foo:M[yY][eE][sS])
BUILTIN_PKG.foo?= foo-1.0
. endif
.endif # IS_BUILTIN.foo
.if !defined(USE_BUILTIN.foo)
USE_BUILTIN.foo?= ${IS_BUILTIN.foo}
. if defined(BUILTIN_PKG.foo)
. for _depend_ in ${BUILDLINK_API_DEPENDS.foo}
. if !empty(USE_BUILTIN.foo:M[yY][eE][sS])
USE_BUILTIN.foo!= \
${PKG_ADMIN} pmatch '${_depend_}' ${BUILTIN_PKG.foo} \
&amp;&amp; ${ECHO} "yes" || ${ECHO} "no"
. endif
. endfor
. endif
.endif # USE_BUILTIN.foo
CHECK_BUILTIN.foo?= no
.if !empty(CHECK_BUILTIN.foo:M[nN][oO])
#
# Here we place code that depends on whether USE_BUILTIN.foo is set to
# "yes" or "no".
#
.endif # CHECK_BUILTIN.foo
</programlisting>
<para>The first section sets
<varname>IS_BUILTIN.<replaceable>pkg</replaceable></varname>
depending on if <replaceable>pkg</replaceable> really exists
in the base system. This should not be a base system software
with similar functionality to <replaceable>pkg</replaceable>;
it should only be <quote>yes</quote> if the actual package is
included as part of the base system. This variable is only
used internally within the <filename>builtin.mk</filename>
file.</para>
<para>The second section sets
<varname>BUILTIN_PKG.<replaceable>pkg</replaceable></varname>
to the version of <replaceable>pkg</replaceable> in the base
system if it exists (if
<varname>IS_BUILTIN.<replaceable>pkg</replaceable></varname>
is <quote>yes</quote>). This variable is only used internally
within the <filename>builtin.mk</filename> file.</para>
<para>The third section sets
<varname>USE_BUILTIN.<replaceable>pkg</replaceable></varname>
and is <emphasis>required</emphasis> in all
<filename>builtin.mk</filename> files. The code in this
section must make the determination whether the built-in
software is adequate to satisfy the dependencies listed in
<varname>BUILDLINK_API_DEPENDS.<replaceable>pkg</replaceable></varname>.
This is typically done by comparing
<varname>BUILTIN_PKG.<replaceable>pkg</replaceable></varname>
against each of the dependencies in
<varname>BUILDLINK_API_DEPENDS.<replaceable>pkg</replaceable></varname>.
<varname>USE_BUILTIN.<replaceable>pkg</replaceable></varname>
<emphasis>must</emphasis> be set to the correct value by the
end of the <filename>builtin.mk</filename> file. Note that
<varname>USE_BUILTIN.<replaceable>pkg</replaceable></varname>
may be <quote>yes</quote> even if
<varname>IS_BUILTIN.<replaceable>pkg</replaceable></varname>
is <quote>no</quote> because we may make the determination
that the built-in version of the software is similar enough to
be used as a replacement.</para>
<para>The last section is guarded by
<varname>CHECK_BUILTIN.<replaceable>pkg</replaceable></varname>,
and includes code that uses the value of
<varname>USE_BUILTIN.<replaceable>pkg</replaceable></varname>
set in the previous section. This typically includes, e.g.,
adding additional dependency restrictions and listing additional
files to symlink into <filename>${BUILDLINK_DIR}</filename> (via
<varname>BUILDLINK_FILES.<replaceable>pkg</replaceable></varname>).</para>
</sect2>
<sect2 id="native-or-pkgsrc-preference">
<title>Global preferences for native or pkgsrc software</title>
<para>When building packages, it's possible to choose whether to set
a global preference for using either the built-in (native)
version or the pkgsrc version of software to satisfy a
dependency. This is controlled by setting
<varname>PREFER_PKGSRC</varname> and
<varname>PREFER_NATIVE</varname>. These variables take values
of either <quote>yes</quote>, <quote>no</quote>, or a list of
packages. <varname>PREFER_PKGSRC</varname> tells pkgsrc to
use the pkgsrc versions of software, while
<varname>PREFER_NATIVE</varname> tells pkgsrc to use the
built-in versions. Preferences are determined by the most
specific instance of the package in either
<varname>PREFER_PKGSRC</varname> or
<varname>PREFER_NATIVE</varname>. If a package is specified
in neither or in both variables, then
<varname>PREFER_PKGSRC</varname> has precedence over
<varname>PREFER_NATIVE</varname>. For example, to require
using pkgsrc versions of software for all but the most basic
bits on a NetBSD system, you can set:</para>
<programlisting>
PREFER_PKGSRC= yes
PREFER_NATIVE= getopt skey tcp_wrappers
</programlisting>
<para>A package <emphasis>must</emphasis> have a
<filename>builtin.mk</filename>
file to be listed in <varname>PREFER_NATIVE</varname>,
otherwise it is simply ignored in that list.</para>
</sect2>
</sect1>
</chapter>