kpriv/linux-hardened
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

firmware: revamp firmware documentation

Browse source 
Understanding this code is getting out of control without any
notes. Give the firmware_class driver a much needed documentation love,
and while at it convert it to the new sphinx documentation format.

v2: typos and small fixes

Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This commit is contained in:
Luis R. Rodriguez 2016-12-16 03:10:36 -08:00 committed by Greg Kroah-Hartman
parent 880444e214
commit 113ccc3837
12 changed files with 474 additions and 128 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

38
Documentation/driver-api/firmware/built-in-fw.rst Normal file
View file

@ -0,0 +1,38 @@
=================
Built-in firmware
=================
Firmware can be built-in to the kernel, this means building the firmware
into vmlinux directly, to enable avoiding having to look for firmware from
the filesystem. Instead, firmware can be looked for inside the kernel
directly. You can enable built-in firmware using the kernel configuration
options:
* CONFIG_EXTRA_FIRMWARE
* CONFIG_EXTRA_FIRMWARE_DIR
This should not be confused with CONFIG_FIRMWARE_IN_KERNEL, this is for drivers
which enables firmware to be built as part of the kernel build process. This
option, CONFIG_FIRMWARE_IN_KERNEL, will build all firmware for all drivers
enabled which ship its firmware inside the Linux kernel source tree.
There are a few reasons why you might want to consider building your firmware
into the kernel with CONFIG_EXTRA_FIRMWARE though:
* Speed
* Firmware is needed for accessing the boot device, and the user doesn't
want to stuff the firmware into the boot initramfs.
Even if you have these needs there are a few reasons why you may not be
able to make use of built-in firmware:
* Legalese - firmware is non-GPL compatible
* Some firmware may be optional
* Firmware upgrades are possible, therefore a new firmware would implicate
a complete kernel rebuild.
* Some firmware files may be really large in size. The remote-proc subsystem
is an example subsystem which deals with these sorts of firmware
* The firmware may need to be scraped out from some device specific location
dynamically, an example is calibration data for for some WiFi chipsets. This
calibration data can be unique per sold device.

16
Documentation/driver-api/firmware/core.rst Normal file
View file

@ -0,0 +1,16 @@
==========================
Firmware API core features
==========================
The firmware API has a rich set of core features available. This section
documents these features.
.. toctree::
fw_search_path
built-in-fw
firmware_cache
direct-fs-lookup
fallback-mechanisms
lookup-order

30
Documentation/driver-api/firmware/direct-fs-lookup.rst Normal file
View file

@ -0,0 +1,30 @@
========================
Direct filesystem lookup
========================
Direct filesystem lookup is the most common form of firmware lookup performed
by the kernel. The kernel looks for the firmware directly on the root
filesystem in the paths documented in the section 'Firmware search paths'.
The filesystem lookup is implemented in fw_get_filesystem_firmware(), it
uses common core kernel file loader facility kernel_read_file_from_path().
The max path allowed is PATH_MAX -- currently this is 4096 characters.
It is recommended you keep /lib/firmware paths on your root filesystem,
avoid having a separate partition for them in order to avoid possible
races with lookups and avoid uses of the custom fallback mechanisms
documented below.
Firmware and initramfs
----------------------
Drivers which are built-in to the kernel should have the firmware integrated
also as part of the initramfs used to boot the kernel given that otherwise
a race is possible with loading the driver and the real rootfs not yet being
available. Stuffing the firmware into initramfs resolves this race issue,
however note that using initrd does not suffice to address the same race.
There are circumstances that justify not wanting to include firmware into
initramfs, such as dealing with large firmware firmware files for the
remote-proc subsystem. For such cases using a userspace fallback mechanism
is currently the only viable solution as only userspace can know for sure
when the real rootfs is ready and mounted.

195
Documentation/driver-api/firmware/fallback-mechanisms.rst Normal file
View file

@ -0,0 +1,195 @@
===================
Fallback mechanisms
===================
A fallback mechanism is supported to allow to overcome failures to do a direct
filesystem lookup on the root filesystem or when the firmware simply cannot be
installed for practical reasons on the root filesystem. The kernel
configuration options related to supporting the firmware fallback mechanism are:
* CONFIG_FW_LOADER_USER_HELPER: enables building the firmware fallback
mechanism. Most distributions enable this option today. If enabled but
CONFIG_FW_LOADER_USER_HELPER_FALLBACK is disabled, only the custom fallback
mechanism is available and for the request_firmware_nowait() call.
* CONFIG_FW_LOADER_USER_HELPER_FALLBACK: force enables each request to
enable the kobject uevent fallback mechanism on all firmware API calls
except request_firmware_direct(). Most distributions disable this option
today. The call request_firmware_nowait() allows for one alternative
fallback mechanism: if this kconfig option is enabled and your second
argument to request_firmware_nowait(), uevent, is set to false you are
informing the kernel that you have a custom fallback mechanism and it will
manually load the firmware. Read below for more details.
Note that this means when having this configuration:
CONFIG_FW_LOADER_USER_HELPER=y
CONFIG_FW_LOADER_USER_HELPER_FALLBACK=n
the kobject uevent fallback mechanism will never take effect even
for request_firmware_nowait() when uevent is set to true.
Justifying the firmware fallback mechanism
==========================================
Direct filesystem lookups may fail for a variety of reasons. Known reasons for
this are worth itemizing and documenting as it justifies the need for the
fallback mechanism:
* Race against access with the root filesystem upon bootup.
* Races upon resume from suspend. This is resolved by the firmware cache, but
the firmware cache is only supported if you use uevents, and its not
supported for request_firmware_into_buf().
* Firmware is not accessible through typical means:
* It cannot be installed into the root filesystem
* The firmware provides very unique device specific data tailored for
the unit gathered with local information. An example is calibration
data for WiFi chipsets for mobile devices. This calibration data is
not common to all units, but tailored per unit. Such information may
be installed on a separate flash partition other than where the root
filesystem is provided.
Types of fallback mechanisms
============================
There are really two fallback mechanisms available using one shared sysfs
interface as a loading facility:
* Kobject uevent fallback mechanism
* Custom fallback mechanism
First lets document the shared sysfs loading facility.
Firmware sysfs loading facility
===============================
In order to help device drivers upload firmware using a fallback mechanism
the firmware infrastructure creates a sysfs interface to enable userspace
to load and indicate when firmware is ready. The sysfs directory is created
via fw_create_instance(). This call creates a new struct device named after
the firmware requested, and establishes it in the device hierarchy by
associating the device used to make the request as the device's parent.
The sysfs directory's file attributes are defined and controlled through
the new device's class (firmare_class) and group (fw_dev_attr_groups).
This is actually where the original firmware_class.c file name comes from,
as originally the only firmware loading mechanism available was the
mechanism we now use as a fallback mechanism.
To load firmware using the sysfs interface we expose a loading indicator,
and a file upload firmware into:
* /sys/$DEVPATH/loading
* /sys/$DEVPATH/data
To upload firmware you will echo 1 onto the loading file to indicate
you are loading firmware. You then cat the firmware into the data file,
and you notify the kernel the firmware is ready by echo'ing 0 onto
the loading file.
The firmware device used to help load firmware using sysfs is only created if
direct firmware loading fails and if the fallback mechanism is enabled for your
firmware request, this is set up with fw_load_from_user_helper(). It is
important to re-iterate that no device is created if a direct filesystem lookup
succeeded.
Using::
echo 1 > /sys/$DEVPATH/loading
Will clean any previous partial load at once and make the firmware API
return an error. When loading firmware the firmware_class grows a buffer
for the firmware in PAGE_SIZE increments to hold the image as it comes in.
firmware_data_read() and firmware_loading_show() are just provided for the
test_firmware driver for testing, they are not called in normal use or
expected to be used regularly by userspace.
Firmware kobject uevent fallback mechanism
==========================================
Since a device is created for the sysfs interface to help load firmware as a
fallback mechanism userspace can be informed of the addition of the device by
relying on kobject uevents. The addition of the device into the device
hierarchy means the fallback mechanism for firmware loading has been initiated.
For details of implementation refer to _request_firmware_load(), in particular
on the use of dev_set_uevent_suppress() and kobject_uevent().
The kernel's kobject uevent mechanism is implemented in lib/kobject_uevent.c,
it issues uevents to userspace. As a supplement to kobject uevents Linux
distributions could also enable CONFIG_UEVENT_HELPER_PATH, which makes use of
core kernel's usermode helper (UMH) functionality to call out to a userspace
helper for kobject uevents. In practice though no standard distribution has
ever used the CONFIG_UEVENT_HELPER_PATH. If CONFIG_UEVENT_HELPER_PATH is
enabled this binary would be called each time kobject_uevent_env() gets called
in the kernel for each kobject uevent triggered.
Different implementations have been supported in userspace to take advantage of
this fallback mechanism. When firmware loading was only possible using the
sysfs mechanism the userspace component "hotplug" provided the functionality of
monitoring for kobject events. Historically this was superseded be systemd's
udev, however firmware loading support was removed from udev as of systemd
commit be2ea723b1d0 ("udev: remove userspace firmware loading support")
as of v217 on August, 2014. This means most Linux distributions today are
not using or taking advantage of the firmware fallback mechanism provided
by kobject uevents. This is specially exacerbated due to the fact that most
distributions today disable CONFIG_FW_LOADER_USER_HELPER_FALLBACK.
Refer to do_firmware_uevent() for details of the kobject event variables
setup. Variables passwdd with a kobject add event:
* FIRMWARE=firmware name
* TIMEOUT=timeout value
* ASYNC=whether or not the API request was asynchronous
By default DEVPATH is set by the internal kernel kobject infrastructure.
Below is an example simple kobject uevent script::
# Both $DEVPATH and $FIRMWARE are already provided in the environment.
MY_FW_DIR=/lib/firmware/
echo 1 > /sys/$DEVPATH/loading
cat $MY_FW_DIR/$FIRMWARE > /sys/$DEVPATH/data
echo 0 > /sys/$DEVPATH/loading
Firmware custom fallback mechanism
==================================
Users of the request_firmware_nowait() call have yet another option available
at their disposal: rely on the sysfs fallback mechanism but request that no
kobject uevents be issued to userspace. The original logic behind this
was that utilities other than udev might be required to lookup firmware
in non-traditional paths -- paths outside of the listing documented in the
section 'Direct filesystem lookup'. This option is not available to any of
the other API calls as uevents are always forced for them.
Since uevents are only meaningful if the fallback mechanism is enabled
in your kernel it would seem odd to enable uevents with kernels that do not
have the fallback mechanism enabled in their kernels. Unfortunately we also
rely on the uevent flag which can be disabled by request_firmware_nowait() to
also setup the firmware cache for firmware requests. As documented above,
the firmware cache is only set up if uevent is enabled for an API call.
Although this can disable the firmware cache for request_firmware_nowait()
calls, users of this API should not use it for the purposes of disabling
the cache as that was not the original purpose of the flag. Not setting
the uevent flag means you want to opt-in for the firmware fallback mechanism
but you want to suppress kobject uevents, as you have a custom solution which
will monitor for your device addition into the device hierarchy somehow and
load firmware for you through a custom path.
Firmware fallback timeout
=========================
The firmware fallback mechanism has a timeout. If firmware is not loaded
onto the sysfs interface by the timeout value an error is sent to the
driver. By default the timeout is set to 60 seconds if uevents are
desirable, otherwise MAX_JIFFY_OFFSET is used (max timeout possible).
The logic behind using MAX_JIFFY_OFFSET for non-uevents is that a custom
solution will have as much time as it needs to load firmware.
You can customize the firmware timeout by echo'ing your desired timeout into
the following file:
* /sys/class/firmware/timeout
If you echo 0 into it means MAX_JIFFY_OFFSET will be used. The data type
for the timeout is an int.

51
Documentation/driver-api/firmware/firmware_cache.rst Normal file
View file

@ -0,0 +1,51 @@
==============
Firmware cache
==============
When Linux resumes from suspend some device drivers require firmware lookups to
re-initialize devices. During resume there may be a period of time during which
firmware lookups are not possible, during this short period of time firmware
requests will fail. Time is of essence though, and delaying drivers to wait for
the root filesystem for firmware delays user experience with device
functionality. In order to support these requirements the firmware
infrastructure implements a firmware cache for device drivers for most API
calls, automatically behind the scenes.
The firmware cache makes using certain firmware API calls safe during a device
driver's suspend and resume callback. Users of these API calls needn't cache
the firmware by themselves for dealing with firmware loss during system resume.
The firmware cache works by requesting for firmware prior to suspend and
caching it in memory. Upon resume device drivers using the firmware API will
have access to the firmware immediately, without having to wait for the root
filesystem to mount or dealing with possible race issues with lookups as the
root filesystem mounts.
Some implementation details about the firmware cache setup:
* The firmware cache is setup by adding a devres entry for each device that
uses all synchronous call except :c:func:`request_firmware_into_buf`.
* If an asynchronous call is used the firmware cache is only set up for a
device if if the second argument (uevent) to request_firmware_nowait() is
true. When uevent is true it requests that a kobject uevent be sent to
userspace for the firmware request. For details refer to the Fackback
mechanism documented below.
* If the firmware cache is determined to be needed as per the above two
criteria the firmware cache is setup by adding a devres entry for the
device making the firmware request.
* The firmware devres entry is maintained throughout the lifetime of the
device. This means that even if you release_firmware() the firmware cache
will still be used on resume from suspend.
* The timeout for the fallback mechanism is temporarily reduced to 10 seconds
as the firmware cache is set up during suspend, the timeout is set back to
the old value you had configured after the cache is set up.
* Upon suspend any pending non-uevent firmware requests are killed to avoid
stalling the kernel, this is done with kill_requests_without_uevent(). Kernel
calls requiring the non-uevent therefore need to implement their own firmware
cache mechanism but must not use the firmware API on suspend.

26
Documentation/driver-api/firmware/fw_search_path.rst Normal file
View file

@ -0,0 +1,26 @@
=====================
Firmware search paths
=====================
The following search paths are used to look for firmware on your
root filesystem.
* fw_path_para - module parameter - default is empty so this is ignored
* /lib/firmware/updates/UTS_RELEASE/
* /lib/firmware/updates/
* /lib/firmware/UTS_RELEASE/
* /lib/firmware/
The module parameter ''path'' can be passed to the firmware_class module
to activate the first optional custom fw_path_para. The custom path can
only be up to 256 characters long. The kernel parameter passed would be:
* 'firmware_class.path=$CUSTOMIZED_PATH'
There is an alternative to customize the path at run time after bootup, you
can use the file:
* /sys/module/firmware_class/parameters/path
You would echo into it your custom path and firmware requested will be
searched for there first.

16
Documentation/driver-api/firmware/index.rst Normal file
View file

@ -0,0 +1,16 @@
==================
Linux Firmware API
==================
.. toctree::
introduction
core
request_firmware
.. only:: subproject and html
Indices
=======
* :ref:`genindex`

27
Documentation/driver-api/firmware/introduction.rst Normal file
View file

@ -0,0 +1,27 @@
============
Introduction
============
The firmware API enables kernel code to request files required
for functionality from userspace, the uses vary:
* Microcode for CPU errata
* Device driver firmware, required to be loaded onto device
microcontrollers
* Device driver information data (calibration data, EEPROM overrides),
some of which can be completely optional.
Types of firmware requests
==========================
There are two types of calls:
* Synchronous
* Asynchronous
Which one you use vary depending on your requirements, the rule of thumb
however is you should strive to use the asynchronous APIs unless you also
are already using asynchronous initialization mechanisms which will not
stall or delay boot. Even if loading firmware does not take a lot of time
processing firmware might, and this can still delay boot or initialization,
as such mechanisms such as asynchronous probe can help supplement drivers.

18
Documentation/driver-api/firmware/lookup-order.rst Normal file
View file

@ -0,0 +1,18 @@
=====================
Firmware lookup order
=====================
Different functionality is available to enable firmware to be found.
Below is chronological order of how firmware will be looked for once
a driver issues a firmware API call.
* The ''Built-in firmware'' is checked first, if the firmware is present we
return it immediately
* The ''Firmware cache'' is looked at next. If the firmware is found we
return it immediately
* The ''Direct filesystem lookup'' is performed next, if found we
return it immediately
* If no firmware has been found and the fallback mechanism was enabled
the sysfs interface is created. After this either a kobject uevent
is issued or the custom firmware loading is relied upon for firmware
loading up to the timeout value.

56
Documentation/driver-api/firmware/request_firmware.rst Normal file
View file

@ -0,0 +1,56 @@
====================
request_firmware API
====================
You would typically load firmware and then load it into your device somehow.
The typical firmware work flow is reflected below::
if(request_firmware(&fw_entry, $FIRMWARE, device) == 0)
copy_fw_to_device(fw_entry->data, fw_entry->size);
release_firmware(fw_entry);
Synchronous firmware requests
=============================
Synchronous firmware requests will wait until the firmware is found or until
an error is returned.
request_firmware
----------------
.. kernel-doc:: drivers/base/firmware_class.c
:functions: request_firmware
request_firmware_direct
-----------------------
.. kernel-doc:: drivers/base/firmware_class.c
:functions: request_firmware_direct
request_firmware_into_buf
-------------------------
.. kernel-doc:: drivers/base/firmware_class.c
:functions: request_firmware_into_buf
Asynchronous firmware requests
==============================
Asynchronous firmware requests allow driver code to not have to wait
until the firmware or an error is returned. Function callbacks are
provided so that when the firmware or an error is found the driver is
informed through the callback. request_firmware_nowait() cannot be called
in atomic contexts.
request_firmware_nowait
-----------------------
.. kernel-doc:: drivers/base/firmware_class.c
:functions: request_firmware_nowait
request firmware API expected driver use
========================================
Once an API call returns you process the firmware and then release the
firmware. For example if you used request_firmware() and it returns,
the driver has the firmware image accessible in fw_entry->{data,size}.
If something went wrong request_firmware() returns non-zero and fw_entry
is set to NULL. Once your driver is done with processing the firmware it
can call call release_firmware(fw_entry) to release the firmware image
and any related resource.

1
Documentation/driver-api/index.rst
View file

@ -30,6 +30,7 @@ available subsections can be seen below.
miscellaneous
vme
80211/index
firmware/index
.. only:: subproject and html

128
Documentation/firmware_class/README
View file

@ -1,128 +0,0 @@
request_firmware() hotplug interface:
------------------------------------
Copyright (C) 2003 Manuel Estrada Sainz
Why:
---
Today, the most extended way to use firmware in the Linux kernel is linking
it statically in a header file. Which has political and technical issues:
1) Some firmware is not legal to redistribute.
2) The firmware occupies memory permanently, even though it often is just
used once.
3) Some people, like the Debian crowd, don't consider some firmware free
enough and remove entire drivers (e.g.: keyspan).
High level behavior (mixed):
============================
1), kernel(driver):
- calls request_firmware(&fw_entry, $FIRMWARE, device)
- kernel searches the firmware image with name $FIRMWARE directly
in the below search path of root filesystem:
User customized search path by module parameter 'path'[1]
"/lib/firmware/updates/" UTS_RELEASE,
"/lib/firmware/updates",
"/lib/firmware/" UTS_RELEASE,
"/lib/firmware"
- If found, goto 7), else goto 2)
[1], the 'path' is a string parameter which length should be less
than 256, user should pass 'firmware_class.path=$CUSTOMIZED_PATH'
if firmware_class is built in kernel(the general situation)
2), userspace:
- /sys/class/firmware/xxx/{loading,data} appear.
- hotplug gets called with a firmware identifier in $FIRMWARE
and the usual hotplug environment.
- hotplug: echo 1 > /sys/class/firmware/xxx/loading
3), kernel: Discard any previous partial load.
4), userspace:
- hotplug: cat appropriate_firmware_image > \
/sys/class/firmware/xxx/data
5), kernel: grows a buffer in PAGE_SIZE increments to hold the image as it
comes in.
6), userspace:
- hotplug: echo 0 > /sys/class/firmware/xxx/loading
7), kernel: request_firmware() returns and the driver has the firmware
image in fw_entry->{data,size}. If something went wrong
request_firmware() returns non-zero and fw_entry is set to
NULL.
8), kernel(driver): Driver code calls release_firmware(fw_entry) releasing
the firmware image and any related resource.
High level behavior (driver code):
==================================
if(request_firmware(&fw_entry, $FIRMWARE, device) == 0)
copy_fw_to_device(fw_entry->data, fw_entry->size);
release_firmware(fw_entry);
Sample/simple hotplug script:
============================
# Both $DEVPATH and $FIRMWARE are already provided in the environment.
HOTPLUG_FW_DIR=/usr/lib/hotplug/firmware/
echo 1 > /sys/$DEVPATH/loading
cat $HOTPLUG_FW_DIR/$FIRMWARE > /sys/$DEVPATH/data
echo 0 > /sys/$DEVPATH/loading
Random notes:
============
- "echo -1 > /sys/class/firmware/xxx/loading" will cancel the load at
once and make request_firmware() return with error.
- firmware_data_read() and firmware_loading_show() are just provided
for testing and completeness, they are not called in normal use.
- There is also /sys/class/firmware/timeout which holds a timeout in
seconds for the whole load operation.
- request_firmware_nowait() is also provided for convenience in
user contexts to request firmware asynchronously, but can't be called
in atomic contexts.
about in-kernel persistence:
---------------------------
Under some circumstances, as explained below, it would be interesting to keep
firmware images in non-swappable kernel memory or even in the kernel image
(probably within initramfs).
Note that this functionality has not been implemented.
- Why OPTIONAL in-kernel persistence may be a good idea sometimes:
- If the device that needs the firmware is needed to access the
filesystem. When upon some error the device has to be reset and the
firmware reloaded, it won't be possible to get it from userspace.
e.g.:
- A diskless client with a network card that needs firmware.
- The filesystem is stored in a disk behind an scsi device
that needs firmware.
- Replacing buggy DSDT/SSDT ACPI tables on boot.
Note: this would require the persistent objects to be included
within the kernel image, probably within initramfs.
And the same device can be needed to access the filesystem or not depending
on the setup, so I think that the choice on what firmware to make
persistent should be left to userspace.
about firmware cache:
--------------------
After firmware cache mechanism is introduced during system sleep,
request_firmware can be called safely inside device's suspend and
resume callback, and callers needn't cache the firmware by
themselves any more for dealing with firmware loss during system
resume.