367 lines
14 KiB
XML
367 lines
14 KiB
XML
<!-- $NetBSD: makefile.xml,v 1.23 2007/06/01 11:07:25 rillig Exp $ -->
|
|
|
|
<chapter id="makefile"> <?dbhtml filename="makefile.html"?>
|
|
<title>Programming in <filename>Makefile</filename>s</title>
|
|
|
|
<para>Pkgsrc consists of many <filename>Makefile</filename> fragments,
|
|
each of which forms a well-defined part of the pkgsrc system. Using
|
|
the &man.make.1; system as a programming language for a big system
|
|
like pkgsrc requires some discipline to keep the code correct and
|
|
understandable.</para>
|
|
|
|
<para>The basic ingredients for <filename>Makefile</filename>
|
|
programming are variables (which are actually macros) and shell
|
|
commands. Among these shell commands may even be more complex ones
|
|
like &man.awk.1; programs. To make sure that every shell command runs
|
|
as intended it is necessary to quote all variables correctly when they
|
|
are used.</para>
|
|
|
|
<para>This chapter describes some patterns, that appear quite often in
|
|
<filename>Makefile</filename>s, including the pitfalls that come along
|
|
with them.</para>
|
|
|
|
<sect1 id="makefile.style">
|
|
<title>Caveats</title>
|
|
|
|
<itemizedlist><listitem><para>When you are creating a file as a
|
|
target of a rule, always write the data to a temporary file first
|
|
and finally rename that file. Otherwise there might occur an error
|
|
in the middle of generating the file, and when the user runs
|
|
&man.make.1; for the second time, the file exists and will not be
|
|
regenerated properly. Example:</para>
|
|
|
|
<programlisting>
|
|
wrong:
|
|
@echo "line 1" > ${.TARGET}
|
|
@echo "line 2" >> ${.TARGET}
|
|
@false
|
|
|
|
correct:
|
|
@echo "line 1" > ${.TARGET}.tmp
|
|
@echo "line 2" >> ${.TARGET}.tmp
|
|
@false
|
|
@mv ${.TARGET}.tmp ${.TARGET}
|
|
</programlisting>
|
|
|
|
<para>When you run <command>make wrong</command> twice, the file
|
|
<filename>wrong</filename> will exist, although there was an error
|
|
message in the first run. On the other hand, running <command>make
|
|
correct</command> gives an error message twice, as expected.</para>
|
|
|
|
<para>You might remember that &man.make.1; sometimes removes
|
|
<literal>${.TARGET}</literal> in case of error, but this only
|
|
happens when it is interrupted, for example by pressing
|
|
<literal>^C</literal>. This does <emphasis>not</emphasis> happen
|
|
when one of the commands fails (like &man.false.1; above).</para>
|
|
|
|
</listitem>
|
|
</itemizedlist>
|
|
</sect1>
|
|
|
|
<sect1 id="makefile.variables">
|
|
<title><filename>Makefile</filename> variables</title>
|
|
|
|
<para><filename>Makefile</filename> variables contain strings that
|
|
can be processed using the five operators ``='', ``+='', ``?='',
|
|
``:='', and ``!='', which are described in the &man.make.1; man
|
|
page.</para>
|
|
|
|
<para>When a variable's value is parsed from a
|
|
<filename>Makefile</filename>, the hash character ``#'' and the
|
|
backslash character ``\'' are handled specially. If a backslash is
|
|
followed by a newline, any whitespace immediately in front of the
|
|
backslash, the backslash, the newline, and any whitespace
|
|
immediately behind the newline are replaced with a single space. A
|
|
backslash character and an immediately following hash character are
|
|
replaced with a single hash character. Otherwise, the backslash is
|
|
passed as is. In a variable assignment, any hash character that is
|
|
not preceded by a backslash starts a comment that continues upto the
|
|
end of the logical line.</para>
|
|
|
|
<para><emphasis>Note:</emphasis> Because of this parsing algorithm
|
|
the only way to create a variable consisting of a single backslash
|
|
is using the ``!='' operator, for example: <!-- FIXME
|
|
--><varname>BACKSLASH!=echo "\\"</varname>.</para>
|
|
|
|
<para>So far for defining variables. The other thing you can do with
|
|
variables is evaluating them. A variable is evaluated when it is
|
|
part of the right side of the ``:='' or the ``!='' operator, or
|
|
directly before executing a shell command which the variable is part
|
|
of. In all other cases, &man.make.1; performs lazy evaluation, that
|
|
is, variables are not evaluated until there's no other way. The
|
|
``modifiers'' mentioned in the man page also evaluate the
|
|
variable.</para>
|
|
|
|
<para>Some of the modifiers split the string into words and then
|
|
operate on the words, others operate on the string as a whole. When
|
|
a string is split into words, it is split as you would expect
|
|
it from &man.sh.1;.</para>
|
|
|
|
<para>No rule without exception—the <command>.for</command>
|
|
loop does not follow the shell quoting rules but splits at sequences
|
|
of whitespace.</para>
|
|
|
|
<para>There are several types of variables that should be handled
|
|
differently. Strings and two types of lists.</para>
|
|
|
|
<itemizedlist>
|
|
|
|
<listitem><para><emphasis>Strings</emphasis> can contain arbitrary
|
|
characters. Nevertheless, you should restrict yourself to only
|
|
using printable characters. Examples are
|
|
<varname>PREFIX</varname> and
|
|
<varname>COMMENT</varname>.</para></listitem>
|
|
|
|
<listitem><para><emphasis>Internal lists</emphasis> are lists that
|
|
are never exported to any shell command. Their elements are
|
|
separated by whitespace. Therefore, the elements themselves cannot
|
|
have embedded whitespace. Any other characters are allowed.
|
|
Internal lists can be used in <command>.for</command> loops.
|
|
Examples are <varname>DEPENDS</varname> and
|
|
<varname>BUILD_DEPENDS</varname>.</para></listitem>
|
|
|
|
<listitem><para><emphasis>External lists</emphasis> are lists that
|
|
may be exported to a shell command. Their elements can contain any
|
|
characters, including whitespace. That's why they cannot be used
|
|
in <command>.for</command> loops. Examples are
|
|
<varname>DISTFILES</varname> and
|
|
<varname>MASTER_SITES</varname>.</para></listitem>
|
|
|
|
</itemizedlist>
|
|
|
|
<sect2 id="makefile.variables.names">
|
|
<title>Naming conventions</title>
|
|
|
|
<itemizedlist>
|
|
|
|
<listitem><para>All variable names starting with an underscore
|
|
are reserved for use by the pkgsrc infrastructure. They shall
|
|
not be used by package
|
|
<filename>Makefile</filename>s.</para></listitem>
|
|
|
|
<listitem><para>In <command>.for</command> loops you should use
|
|
lowercase variable names for the iteration
|
|
variables.</para></listitem>
|
|
|
|
<listitem><para>All list variables should have a ``plural''
|
|
name, e.g. <varname>PKG_OPTIONS</varname> or
|
|
<varname>DISTFILES</varname>.</para></listitem>
|
|
|
|
</itemizedlist>
|
|
</sect2>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="makefile.code">
|
|
<title>Code snippets</title>
|
|
|
|
<para>This section presents you with some code snippets you should
|
|
use in your own code. If you don't find anything appropriate here,
|
|
you should test your code and add it here.</para>
|
|
|
|
<sect2 id="adding-to-list">
|
|
<title>Adding things to a list</title>
|
|
|
|
<programlisting>
|
|
STRING= foo * bar `date`
|
|
INT_LIST= # empty
|
|
ANOTHER_INT_LIST= apache-[0-9]*:../../www/apache
|
|
EXT_LIST= # empty
|
|
ANOTHER_EXT_LIST= a=b c=d
|
|
|
|
INT_LIST+= ${STRING} # 1
|
|
INT_LIST+= ${ANOTHER_INT_LIST} # 2
|
|
EXT_LIST+= ${STRING:Q} # 3
|
|
EXT_LIST+= ${ANOTHER_EXT_LIST} # 4
|
|
</programlisting>
|
|
|
|
<para>When you add a string to an external list (example 3), it
|
|
must be quoted. In all other cases, you must not add a quoting
|
|
level. You must not merge internal and external lists, unless you
|
|
are sure that all entries are correctly interpreted in both
|
|
lists.</para>
|
|
|
|
</sect2>
|
|
|
|
<sect2 id="converting-internal-to-external">
|
|
<title>Converting an internal list into an external list</title>
|
|
|
|
<programlisting>
|
|
EXT_LIST= # empty
|
|
.for i in ${INT_LIST}
|
|
EXT_LIST+= ${i:Q}""
|
|
.endfor
|
|
</programlisting>
|
|
|
|
<para>This code converts the internal list
|
|
<varname>INT_LIST</varname> into the external list
|
|
<varname>EXT_LIST</varname>. As the elements of an internal list
|
|
are unquoted they must be quoted here. The reason for appending
|
|
<varname>""</varname> is explained below.</para>
|
|
|
|
</sect2>
|
|
|
|
<sect2 id="passing-variable-to-shell">
|
|
<title>Passing variables to a shell command</title>
|
|
|
|
<para>Sometimes you may want to print an arbitrary string. There
|
|
are many ways to get it wrong and only few that can handle every
|
|
nastiness.</para>
|
|
|
|
<programlisting>
|
|
STRING= foo bar < > * `date` $$HOME ' "
|
|
EXT_LIST= string=${STRING:Q} x=second\ item
|
|
|
|
all:
|
|
echo ${STRING} # 1
|
|
echo "${STRING}" # 2
|
|
echo "${STRING:Q}" # 3
|
|
echo ${STRING:Q} # 4
|
|
echo x${STRING:Q} | sed 1s,.,, # 5
|
|
printf "%s\\n" ${STRING:Q}"" # 6
|
|
env ${EXT_LIST} /bin/sh -c 'echo "$$string"; echo "$$x"'
|
|
</programlisting>
|
|
|
|
<para>Example 1 leads to a syntax error in the shell, as the
|
|
characters are just copied.</para>
|
|
|
|
<para>Example 2 leads to a syntax error too, and if you leave out
|
|
the last " character from <varname>${STRING}</varname>,
|
|
&man.date.1; will be executed. The <varname>$HOME</varname> shell
|
|
variable would be evaluated, too.</para>
|
|
|
|
<para>Example 3 outputs each space character preceded by a
|
|
backslash (or not), depending on the implementation of the
|
|
&man.echo.1; command.</para>
|
|
|
|
<para>Example 4 handles correctly every string that does not start
|
|
with a dash. In that case, the result depends on the
|
|
implementation of the &man.echo.1; command. As long as you can
|
|
guarantee that your input does not start with a dash, this form is
|
|
appropriate.</para>
|
|
|
|
<para>Example 5 handles even the case of a leading dash
|
|
correctly.</para>
|
|
|
|
<para>Example 6 also works with every string and is the
|
|
light-weight solution, since it does not involve a pipe, which has
|
|
its own problems.</para>
|
|
|
|
<para>The <varname>EXT_LIST</varname> does not need to be quoted
|
|
because the quoting has already been done when adding elements to
|
|
the list.</para>
|
|
|
|
<para>As internal lists shall not be passed to the shell, there is
|
|
no example for it.</para>
|
|
|
|
</sect2>
|
|
|
|
<sect2 id="quoting-guideline">
|
|
<title>Quoting guideline</title>
|
|
|
|
<para>There are many possible sources of wrongly quoted variables.
|
|
This section lists some of the commonly known ones.</para>
|
|
|
|
<itemizedlist>
|
|
|
|
<listitem><para>Whenever you use the value of a list, think
|
|
about what happens to leading or trailing whitespace. If the
|
|
list is a well-formed shell expression, you can apply the
|
|
<varname>:M*</varname> modifier to strip leading and trailing
|
|
whitespace from each word. The <varname>:M</varname> operator
|
|
first splits its argument according to the rules of the shell,
|
|
and then creates a new list consisting of all words that match
|
|
the shell glob expression <varname>*</varname>, that is: all.
|
|
One class of situations where this is needed is when adding a
|
|
variable like <varname>CPPFLAGS</varname> to
|
|
<varname>CONFIGURE_ARGS</varname>. If the configure script
|
|
invokes other configure scripts, it strips the leading and
|
|
trailing whitespace from the variable and then passes it to the
|
|
other configure scripts. But these configure scripts expect the
|
|
(child) <varname>CPPFLAGS</varname> variable to be the same as
|
|
the parent <varname>CPPFLAGS</varname>. That's why we better
|
|
pass the <varname>CPPFLAGS</varname> value properly trimmed. And
|
|
here is how we do it:</para>
|
|
|
|
<programlisting>
|
|
CPPFLAGS= # empty
|
|
CPPFLAGS+= -Wundef -DPREFIX=\"${PREFIX:Q}\"
|
|
CPPFLAGS+= ${MY_CPPFLAGS}
|
|
|
|
CONFIGURE_ARGS+= CPPFLAGS=${CPPFLAGS:M*:Q}
|
|
|
|
all:
|
|
echo x${CPPFLAGS:Q}x # leading and trailing whitespace
|
|
echo x${CONFIGURE_ARGS}x # properly trimmed
|
|
</programlisting></listitem>
|
|
|
|
<listitem><para>The example above contains one bug: The
|
|
<varname>${PREFIX}</varname> is a properly quoted shell
|
|
expression, but there is the C compiler after it, which also
|
|
expects a properly quoted string (this time in C syntax). The
|
|
version above is therefore only correct if
|
|
<varname>${PREFIX}</varname> does not have embedded backslashes
|
|
or double quotes. If you want to allow these, you have to add
|
|
another layer of quoting to each variable that is used as a C
|
|
string literal. You cannot use the <varname>:Q</varname>
|
|
operator for it, as this operator only works for the
|
|
shell.</para></listitem>
|
|
|
|
<listitem><para>Whenever a variable can be empty, the
|
|
<varname>:Q</varname> operator can have surprising results. Here
|
|
are two completely different cases which can be solved with the
|
|
same trick.</para>
|
|
|
|
<programlisting>
|
|
EMPTY= # empty
|
|
empty_test:
|
|
for i in a ${EMPTY:Q} c; do \
|
|
echo "$$i"; \
|
|
done
|
|
|
|
for_test:
|
|
.for i in a:\ a:\test.txt
|
|
echo ${i:Q}
|
|
echo "foo"
|
|
.endfor
|
|
</programlisting>
|
|
|
|
<para>The first example will only print two of the three lines
|
|
we might have expected. This is because
|
|
<varname>${EMPTY:Q}</varname> expands to the empty string, which
|
|
the shell cannot see. The workaround is to write
|
|
<varname>${EMPTY:Q}""</varname>. This pattern can be often found
|
|
as <varname>${TEST} -z ${VAR:Q}</varname> or as <varname>${TEST}
|
|
-f ${FNAME:Q}</varname> (both of these are wrong).</para>
|
|
|
|
<para>The second example will only print three lines instead of
|
|
four. The first line looks like <varname>a:\ echo foo</varname>.
|
|
This is because the backslash of the value
|
|
<varname>a:\</varname> is interpreted as a line-continuation by
|
|
&man.make.1;, which makes the second line the arguments of the
|
|
&man.echo.1; command from the first line. To avoid this, write
|
|
<varname>${i:Q}""</varname>.</para></listitem>
|
|
|
|
</itemizedlist>
|
|
</sect2>
|
|
|
|
<sect2 id="bsd-make-bug-workaround">
|
|
<title>Workaround for a bug in BSD Make</title>
|
|
|
|
<para>The pkgsrc bmake program does not handle the following
|
|
assignment correctly. In case <varname>_othervar_</varname>
|
|
contains a ``-'' character, one of the closing braces is included
|
|
in <varname>${VAR}</varname> after this code executes.</para>
|
|
|
|
<programlisting>
|
|
VAR:= ${VAR:N${_othervar_:C/-//}}
|
|
</programlisting>
|
|
|
|
<para>For a more complex code snippet and a workaround, see the
|
|
package <filename role="pkg">regress/make-quoting</filename>, testcase
|
|
<varname>bug1</varname>.</para>
|
|
|
|
</sect2>
|
|
|
|
</sect1>
|
|
</chapter>
|