Kbuild thin archives updates for v4.13

Thin archives migration by Nicholas Piggin.
 
 THIN_ARCHIVES has been available for a while as an optional feature
 only for PowerPC architecture, but we do not need two different
 intermediate-artifact schemes.
 
 Using thin archives instead of conventional incremental linking has
 various advantages:
  - save disk space for builds
  - speed-up building a little
  - fix some link issues (for example, allyesconfig on ARM) due to
    more flexibility for the final linking
  - work better with dead code elimination we are planning
 
 As discussed before, this migration has been done unconditionally
 so that any problems caused by this will show up with "git bisect".
 
 With testing with 0-day and linux-next, some architectures actually
 showed up problems, but they were trivial and all fixed now.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJZXsiSAAoJED2LAQed4NsGfqUQAIxbR4JcFCeGNNqgOV1q7Ban
 CaMzVZWPum0Mq+JWzknHrCJQzBE+4BPLbOtZH4Y0YhjXVfc2/M8QkzEzSWyEPm03
 FyaQ6WTq479mv7Ot2nAwaRSUYNSOuvlCx5KUOxITMJ/VmxwXXc9fCuT3ORu9opdK
 4iyh0P2D+IeABQlrS5k1Rj+y4u/BtpiGY9U5RDssn7u8sjEgBHWFXFfE2fQ0No+0
 1lzwa5EVyPHuq0XTBeZkPSDNxtou4iZzQC9QeNIYlyiod1G9deE4lzB55s+Qtkk0
 h6rN9WF+Rvy7/hjFUJy0TDPNx0io2kdJxMaMKp2HaES49w5fHv7NAgxuipFC91vE
 5UKs1sXxBe8dpPjfZWY7QSQ/JQv6NuG7NWcSGM29BWy3yFefSAXCggM+nn5IWzLH
 pSutfOBGeceJdyKMcdn3AgcHCj0wddFxX8AXst+ZebnqVoNxR/Nu6HGmyaucwyp3
 6fFTkbZ6DvOlu9MKbK0HSqrsT3DlAas2YWZKZ4Cc20wM99Z0OtFZlmpMCRIdiYtx
 hZBwze/ElheUbZu6igH6UX2lpOlat0V6nT5vKHGGeOJlwkxduKi3Kj6zVSkCHic5
 w3NLXr5FDWdkrMiC6/Z0Uae5mtAWOYyt6z1CwjgVmFrAkqlL8aWNagOcDCSFc1qR
 +3Cv7pZQSRWy2TaaLMzo
 =PAWi
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-thinar-v4.13' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild thin archives updates from Masahiro Yamada:
 "Thin archives migration by Nicholas Piggin.

  THIN_ARCHIVES has been available for a while as an optional feature
  only for PowerPC architecture, but we do not need two different
  intermediate-artifact schemes.

  Using thin archives instead of conventional incremental linking has
  various advantages:

   - save disk space for builds

   - speed-up building a little

   - fix some link issues (for example, allyesconfig on ARM) due to more
     flexibility for the final linking

   - work better with dead code elimination we are planning

  As discussed before, this migration has been done unconditionally so
  that any problems caused by this will show up with "git bisect".

  With testing with 0-day and linux-next, some architectures actually
  showed up problems, but they were trivial and all fixed now"

* tag 'kbuild-thinar-v4.13' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
  tile: remove unneeded extra-y in Makefile
  kbuild: thin archives make default for all archs
  x86/um: thin archives build fix
  tile: thin archives fix linking
  ia64: thin archives fix linking
  sh: thin archives fix linking
  kbuild: handle libs-y archives separately from built-in.o archives
  kbuild: thin archives use P option to ar
  kbuild: thin archives final link close --whole-archives option
  ia64: remove unneeded extra-y in Makefile.gate
  tile: fix dependency and .*.cmd inclusion for incremental build
  sparc64: Use indirect calls in hamming weight stubs
This commit is contained in:
Linus Torvalds 2017-07-07 15:11:12 -07:00
commit 98ced886dd
12 changed files with 93 additions and 72 deletions

View file

@ -236,5 +236,9 @@ Files specified with KBUILD_VMLINUX_INIT are linked first.
KBUILD_VMLINUX_MAIN
--------------------------------------------------
All object files for the main part of vmlinux.
KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN together specify
all the object files used to link vmlinux.
KBUILD_VMLINUX_LIBS
--------------------------------------------------
All .a "lib" files for vmlinux.
KBUILD_VMLINUX_INIT, KBUILD_VMLINUX_MAIN, and KBUILD_VMLINUX_LIBS together
specify all the object files used to link vmlinux.

View file

@ -31,7 +31,7 @@ you probably needn't concern yourself with isdn4k-utils.
====================== =============== ========================================
GNU C 3.2 gcc --version
GNU make 3.81 make --version
binutils 2.12 ld -v
binutils 2.20 ld -v
util-linux 2.10o fdformat --version
module-init-tools 0.9.10 depmod -V
e2fsprogs 1.41.4 e2fsck -V
@ -75,10 +75,9 @@ You will need GNU make 3.81 or later to build the kernel.
Binutils
--------
Linux on IA-32 has recently switched from using ``as86`` to using ``gas`` for
assembling the 16-bit boot code, removing the need for ``as86`` to compile
your kernel. This change does, however, mean that you need a recent
release of binutils.
The build system has, as of 4.13, switched to using thin archives (`ar T`)
rather than incremental linking (`ld -r`) for built-in.o intermediate steps.
This requires binutils 2.20 or newer.
Perl
----

View file

@ -944,19 +944,19 @@ core-y := $(patsubst %/, %/built-in.o, $(core-y))
drivers-y := $(patsubst %/, %/built-in.o, $(drivers-y))
net-y := $(patsubst %/, %/built-in.o, $(net-y))
libs-y1 := $(patsubst %/, %/lib.a, $(libs-y))
libs-y2 := $(patsubst %/, %/built-in.o, $(libs-y))
libs-y := $(libs-y1) $(libs-y2)
libs-y2 := $(filter-out %.a, $(patsubst %/, %/built-in.o, $(libs-y)))
virt-y := $(patsubst %/, %/built-in.o, $(virt-y))
# Externally visible symbols (used by link-vmlinux.sh)
export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y) $(virt-y)
export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y)
export KBUILD_VMLINUX_LIBS := $(libs-y1)
export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds
export LDFLAGS_vmlinux
# used by scripts/pacmage/Makefile
export KBUILD_ALLDIRS := $(sort $(filter-out arch/%,$(vmlinux-alldirs)) arch Documentation include samples scripts tools)
vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN)
vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS)
# Include targets which we want to execute sequentially if the rest of the
# kernel build went well. If CONFIG_TRIM_UNUSED_KSYMS is set, this might be

View file

@ -550,7 +550,7 @@ config CC_STACKPROTECTOR_STRONG
endchoice
config THIN_ARCHIVES
bool
def_bool y
help
Select this if the architecture wants to use thin archives
instead of ld -r to create the built-in.o files.

View file

@ -1,8 +1,8 @@
# The gate DSO image is built using a special linker script.
targets += gate.so gate-syms.o
targets += gate.so gate.lds gate.o gate-dummy.o
extra-y += gate.so gate-syms.o gate.lds gate.o
obj-y += gate-syms.o
CPPFLAGS_gate.lds := -P -C -U$(ARCH)
@ -14,13 +14,14 @@ GATECFLAGS_gate.so = -shared -s -Wl,-soname=linux-gate.so.1 \
$(obj)/gate.so: $(obj)/gate.lds $(obj)/gate.o FORCE
$(call if_changed,gate)
$(obj)/built-in.o: $(obj)/gate-syms.o
$(obj)/built-in.o: ld_flags += -R $(obj)/gate-syms.o
GATECFLAGS_gate-syms.o = -r
$(obj)/gate-syms.o: $(obj)/gate.lds $(obj)/gate.o FORCE
GATECFLAGS_gate-dummy.o = -r
$(obj)/gate-dummy.o: $(obj)/gate.lds $(obj)/gate.o FORCE
$(call if_changed,gate)
LDFLAGS_gate-syms.o := -r -R
$(obj)/gate-syms.o: $(obj)/gate-dummy.o FORCE
$(call if_changed,ld)
# gate-data.o contains the gate DSO image as data in section .data..gate.
# We must build gate.so before we can assemble it.
# Note: kbuild does not track this dependency due to usage of .incbin

View file

@ -463,14 +463,6 @@ config MPROFILE_KERNEL
depends on PPC64 && CPU_LITTLE_ENDIAN
def_bool !DISABLE_MPROFILE_KERNEL
config USE_THIN_ARCHIVES
bool "Build the kernel using thin archives"
default n
select THIN_ARCHIVES
help
Build the kernel using thin archives.
If you're unsure say N.
config IOMMU_HELPER
def_bool PPC64

View file

@ -1,11 +1,11 @@
obj-y += vsyscall.o vsyscall-syscall.o
obj-y += vsyscall.o vsyscall-syscall.o vsyscall-syms.o
$(obj)/vsyscall-syscall.o: \
$(foreach F,trapa,$(obj)/vsyscall-$F.so)
# Teach kbuild about targets
targets += $(foreach F,trapa,vsyscall-$F.o vsyscall-$F.so)
targets += vsyscall-note.o vsyscall.lds
targets += vsyscall-note.o vsyscall.lds vsyscall-dummy.o
# The DSO images are built using a special linker script
quiet_cmd_syscall = SYSCALL $@
@ -26,11 +26,11 @@ $(obj)/vsyscall-%.so: $(src)/vsyscall.lds $(obj)/vsyscall-%.o FORCE
# We also create a special relocatable object that should mirror the symbol
# table and layout of the linked DSO. With ld -R we can then refer to
# these symbols in the kernel code rather than hand-coded addresses.
extra-y += vsyscall-syms.o
$(obj)/built-in.o: $(obj)/vsyscall-syms.o
$(obj)/built-in.o: ld_flags += -R $(obj)/vsyscall-syms.o
SYSCFLAGS_vsyscall-syms.o = -r
$(obj)/vsyscall-syms.o: $(src)/vsyscall.lds \
SYSCFLAGS_vsyscall-dummy.o = -r
$(obj)/vsyscall-dummy.o: $(src)/vsyscall.lds \
$(obj)/vsyscall-trapa.o $(obj)/vsyscall-note.o FORCE
$(call if_changed,syscall)
LDFLAGS_vsyscall-syms.o := -r -R
$(obj)/vsyscall-syms.o: $(obj)/vsyscall-dummy.o FORCE
$(call if_changed,ld)

View file

@ -4,9 +4,9 @@
.text
.align 32
ENTRY(__arch_hweight8)
ba,pt %xcc, __sw_hweight8
sethi %hi(__sw_hweight8), %g1
jmpl %g1 + %lo(__sw_hweight8), %g0
nop
nop
ENDPROC(__arch_hweight8)
EXPORT_SYMBOL(__arch_hweight8)
.section .popc_3insn_patch, "ax"
@ -17,9 +17,9 @@ EXPORT_SYMBOL(__arch_hweight8)
.previous
ENTRY(__arch_hweight16)
ba,pt %xcc, __sw_hweight16
sethi %hi(__sw_hweight16), %g1
jmpl %g1 + %lo(__sw_hweight16), %g0
nop
nop
ENDPROC(__arch_hweight16)
EXPORT_SYMBOL(__arch_hweight16)
.section .popc_3insn_patch, "ax"
@ -30,9 +30,9 @@ EXPORT_SYMBOL(__arch_hweight16)
.previous
ENTRY(__arch_hweight32)
ba,pt %xcc, __sw_hweight32
sethi %hi(__sw_hweight32), %g1
jmpl %g1 + %lo(__sw_hweight32), %g0
nop
nop
ENDPROC(__arch_hweight32)
EXPORT_SYMBOL(__arch_hweight32)
.section .popc_3insn_patch, "ax"
@ -43,9 +43,9 @@ EXPORT_SYMBOL(__arch_hweight32)
.previous
ENTRY(__arch_hweight64)
ba,pt %xcc, __sw_hweight64
sethi %hi(__sw_hweight16), %g1
jmpl %g1 + %lo(__sw_hweight16), %g0
nop
nop
ENDPROC(__arch_hweight64)
EXPORT_SYMBOL(__arch_hweight64)
.section .popc_3insn_patch, "ax"

View file

@ -5,15 +5,14 @@ vdso-syms = rt_sigreturn gettimeofday
obj-vdso = $(patsubst %, v%.o, $(vdso-syms))
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.lds
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.lds vdso-dummy.o
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
# vdso32 is only for tilegx -m32 compat task.
VDSO32-$(CONFIG_COMPAT) := y
obj-y += vdso.o
obj-y += vdso.o vdso-syms.o
obj-$(VDSO32-y) += vdso32.o
extra-y += vdso.lds
CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
# vDSO code runs in userspace and -pg doesn't help with profiling anyway.
@ -42,23 +41,22 @@ $(obj)/vdso.o: $(obj)/vdso.so
# link rule for the .so file, .lds has to be first
SYSCFLAGS_vdso.so.dbg = $(c_flags)
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
# We also create a special relocatable object that should mirror the symbol
# table and layout of the linked DSO. With ld -R we can then refer to
# these symbols in the kernel code rather than hand-coded addresses.
extra-y += vdso-syms.o
$(obj)/built-in.o: $(obj)/vdso-syms.o
$(obj)/built-in.o: ld_flags += -R $(obj)/vdso-syms.o
SYSCFLAGS_vdso.so.dbg = -shared -s -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=both)
SYSCFLAGS_vdso_syms.o = -r
$(obj)/vdso-syms.o: $(src)/vdso.lds $(obj)/vrt_sigreturn.o FORCE
SYSCFLAGS_vdso_dummy.o = -r
$(obj)/vdso-dummy.o: $(src)/vdso.lds $(obj)/vrt_sigreturn.o FORCE
$(call if_changed,vdsold)
LDFLAGS_vdso-syms.o := -r -R
$(obj)/vdso-syms.o: $(obj)/vdso-dummy.o FORCE
$(call if_changed,ld)
# strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@ -96,17 +94,17 @@ KBUILD_CFLAGS_32 := $(filter-out -m64,$(KBUILD_CFLAGS))
KBUILD_CFLAGS_32 += -m32 -fPIC -shared
obj-vdso32 = $(patsubst %, v%32.o, $(vdso-syms))
obj-vdso32 := $(addprefix $(obj)/, $(obj-vdso32))
targets += $(obj-vdso32) vdso32.so vdso32.so.dbg
obj-vdso32 := $(addprefix $(obj)/, $(obj-vdso32))
$(obj-vdso32:%=%): KBUILD_AFLAGS = $(KBUILD_AFLAGS_32)
$(obj-vdso32:%=%): KBUILD_CFLAGS = $(KBUILD_CFLAGS_32)
$(obj)/vgettimeofday32.o: $(obj)/vgettimeofday.c
$(obj)/vgettimeofday32.o: $(obj)/vgettimeofday.c FORCE
$(call if_changed_rule,cc_o_c)
$(obj)/vrt_sigreturn32.o: $(obj)/vrt_sigreturn.S
$(obj)/vrt_sigreturn32.o: $(obj)/vrt_sigreturn.S FORCE
$(call if_changed,as_o_S)
# Force dependency
@ -114,5 +112,5 @@ $(obj)/vdso32.o: $(obj)/vdso32.so
SYSCFLAGS_vdso32.so.dbg = -m32 -shared -s -Wl,-soname=linux-vdso32.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=both)
$(obj)/vdso32.so.dbg: $(src)/vdso.lds $(obj-vdso32)
$(obj)/vdso32.so.dbg: $(src)/vdso.lds $(obj-vdso32) FORCE
$(call if_changed,vdsold)

View file

@ -53,7 +53,7 @@ CFLAGS_REMOVE_vdso-note.o = -pg -fprofile-arcs -ftest-coverage
CFLAGS_REMOVE_um_vdso.o = -pg -fprofile-arcs -ftest-coverage
targets += vdso-syms.lds
obj-$(VDSO64-y) += vdso-syms.lds
extra-$(VDSO64-y) += vdso-syms.lds
#
# Match symbols in the DSO that look like VDSO*; produce a file of constants.

View file

@ -437,8 +437,8 @@ $(sort $(subdir-obj-y)): $(subdir-ym) ;
ifdef builtin-target
ifdef CONFIG_THIN_ARCHIVES
cmd_make_builtin = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS)
cmd_make_empty_builtin = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS)
cmd_make_builtin = rm -f $@; $(AR) rcSTP$(KBUILD_ARFLAGS)
cmd_make_empty_builtin = rm -f $@; $(AR) rcSTP$(KBUILD_ARFLAGS)
quiet_cmd_link_o_target = AR $@
else
cmd_make_builtin = $(LD) $(ld_flags) -r -o
@ -478,7 +478,7 @@ ifdef lib-target
quiet_cmd_link_l_target = AR $@
ifdef CONFIG_THIN_ARCHIVES
cmd_link_l_target = rm -f $@; $(AR) rcsT$(KBUILD_ARFLAGS) $@ $(lib-y)
cmd_link_l_target = rm -f $@; $(AR) rcsTP$(KBUILD_ARFLAGS) $@ $(lib-y)
else
cmd_link_l_target = rm -f $@; $(AR) rcs$(KBUILD_ARFLAGS) $@ $(lib-y)
endif
@ -531,7 +531,7 @@ cmd_link_multi-link = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps) $(cmd_secana
ifdef CONFIG_THIN_ARCHIVES
quiet_cmd_link_multi-y = AR $@
cmd_link_multi-y = rm -f $@; $(AR) rcST$(KBUILD_ARFLAGS) $@ $(link_multi_deps)
cmd_link_multi-y = rm -f $@; $(AR) rcSTP$(KBUILD_ARFLAGS) $@ $(link_multi_deps)
else
quiet_cmd_link_multi-y = LD $@
cmd_link_multi-y = $(cmd_link_multi-link)

View file

@ -3,9 +3,12 @@
# link vmlinux
#
# vmlinux is linked from the objects selected by $(KBUILD_VMLINUX_INIT) and
# $(KBUILD_VMLINUX_MAIN). Most are built-in.o files from top-level directories
# in the kernel tree, others are specified in arch/$(ARCH)/Makefile.
# Ordering when linking is important, and $(KBUILD_VMLINUX_INIT) must be first.
# $(KBUILD_VMLINUX_MAIN) and $(KBUILD_VMLINUX_LIBS). Most are built-in.o files
# from top-level directories in the kernel tree, others are specified in
# arch/$(ARCH)/Makefile. Ordering when linking is important, and
# $(KBUILD_VMLINUX_INIT) must be first. $(KBUILD_VMLINUX_LIBS) are archives
# which are linked conditionally (not within --whole-archive), and do not
# require symbol indexes added.
#
# vmlinux
# ^
@ -16,6 +19,9 @@
# +--< $(KBUILD_VMLINUX_MAIN)
# | +--< drivers/built-in.o mm/built-in.o + more
# |
# +--< $(KBUILD_VMLINUX_LIBS)
# | +--< lib/lib.a + more
# |
# +-< ${kallsymso} (see description in KALLSYMS section)
#
# vmlinux version (uname -v) cannot be updated during normal
@ -37,9 +43,10 @@ info()
fi
}
# Thin archive build here makes a final archive with
# symbol table and indexes from vmlinux objects, which can be
# used as input to linker.
# Thin archive build here makes a final archive with symbol table and indexes
# from vmlinux objects INIT and MAIN, which can be used as input to linker.
# KBUILD_VMLINUX_LIBS archives should already have symbol table and indexes
# added.
#
# Traditional incremental style of link does not require this step
#
@ -50,7 +57,7 @@ archive_builtin()
if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
info AR built-in.o
rm -f built-in.o;
${AR} rcsT${KBUILD_ARFLAGS} built-in.o \
${AR} rcsTP${KBUILD_ARFLAGS} built-in.o \
${KBUILD_VMLINUX_INIT} \
${KBUILD_VMLINUX_MAIN}
fi
@ -63,11 +70,17 @@ modpost_link()
local objects
if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
objects="--whole-archive built-in.o"
objects="--whole-archive \
built-in.o \
--no-whole-archive \
--start-group \
${KBUILD_VMLINUX_LIBS} \
--end-group"
else
objects="${KBUILD_VMLINUX_INIT} \
--start-group \
${KBUILD_VMLINUX_MAIN} \
${KBUILD_VMLINUX_LIBS} \
--end-group"
fi
${LD} ${LDFLAGS} -r -o ${1} ${objects}
@ -83,11 +96,18 @@ vmlinux_link()
if [ "${SRCARCH}" != "um" ]; then
if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
objects="--whole-archive built-in.o ${1}"
objects="--whole-archive \
built-in.o \
--no-whole-archive \
--start-group \
${KBUILD_VMLINUX_LIBS} \
--end-group \
${1}"
else
objects="${KBUILD_VMLINUX_INIT} \
--start-group \
${KBUILD_VMLINUX_MAIN} \
${KBUILD_VMLINUX_LIBS} \
--end-group \
${1}"
fi
@ -96,11 +116,18 @@ vmlinux_link()
-T ${lds} ${objects}
else
if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
objects="-Wl,--whole-archive built-in.o ${1}"
objects="-Wl,--whole-archive \
built-in.o \
-Wl,--no-whole-archive \
-Wl,--start-group \
${KBUILD_VMLINUX_LIBS} \
-Wl,--end-group \
${1}"
else
objects="${KBUILD_VMLINUX_INIT} \
-Wl,--start-group \
${KBUILD_VMLINUX_MAIN} \
${KBUILD_VMLINUX_LIBS} \
-Wl,--end-group \
${1}"
fi