Synopsis

This is a simple guide for setting up PXE + NFS booting on an OpenWrt system.

How-to (filesystem)

Setting up GRUB2 (bootloader)

Let's say you are using a USB storage device to store your diskless bootloader, mounted at /mnt/tftp.

For setting up the GRUB bootloader files, you simply have to run:

grub-mknetdir --net-directory /mnt/tftp

You can alter your pre-loaded modules, locales, themes, etc. Just make sure, if you'd like to be able to BIOS-boot, EFI32 boot, or EFI64, to have your grub compiled with that support.

Setting up the NFS root (if this is what you want)

The NFS root does not not have to be in the tftp path.

The simplest way to do it, is to have a rootfs for each distro, in a directory, on a fs supporting uid, gid, and suid/guid. Ext4 is the most common, however you might prefer using F2FS or other filesystems.

The fs requirement is mandatory, but having it in a directory is not - you could as well keep it in a squashfs. But then you have to figure out how to serve it, and load it.

Kernel & Initramfs

Probably, your initramfs/initrd does not come with all the features you'd like, such as nfs. For that, you have to build your own initramfs. To aid with the process, I've written some examples (dracut and mkinitcpio) in ./pxe-root. You might not need all the modules, such as cryptsetup or nvme.

The way I did it, was to chroot in the new root - as the tool would probably install some binaries in the initial ramdisk. You can probably try to specify the root and/or sysroot argument. For cross-building, I don't know if the standalone tool would be able to deal with this. You can try chroot-ing with QEMU (you can find a nice example https://wiki.gentoo.org/wiki/Raspberry_Pi#Compiling_using_chroot][here).

grub.cfg

You can find a sample grub.cfg file as well. It's not guaranteed to work with all kernels, but it works for me.

How-to (server)

First, the server has to be configured.

Start by installing the packages in the ./openwrt-conf directory. The list assumes you already have a dhcp server installed. On top, a NFS server is added, support for the ext4 filesystem, and for USB devices.

For the next steps, sed patches are used - you'd have to replace the variables in the comments for it to work.

fstab

Patch it with fstab_patch.sed, and replace the device with your desired device (i.e: /dev/sda1 for example), and also the mountpoint (i.e: /mnt).

DHCP

Next, the configuration files have to be patched. The /etc/config/dhcp config can be patched by applying the sed script dhcp_patch.sed. You will have to customize ${PXE_BOOT_PATH}, to point to the parent of the grub directory, and the ${PXE_HOST} and ${PXE_ADDR} to point to your OpenWrt machine.

It adds the TFTP server and configures it to announce via DHCP that the client can use it to boot remotely.

You probably want to restart dnsmasq now.

NFS

In case you'd like an NFS filesystem, you can install the nfs server.

You'd also have to create/patch the /etc/exports file, to let nfsd know what and how you are serving. For that, exports_patch.sed can be used. The NFS root should be where your rootfs/images are located.

Hot-plugging

It's not guaranteed, and also not secure, to always serve the nfs server. You can modify the script to wait for a certain mount to become available, and stop it when it's unavailable.

So I wrote a little script to do this:

mkdir -p /usr/local/bin

cp /tmp/pxe_watcher.sh /usr/local/bin/
cp /tmp/pxe_watcher /etc/init.d/

/etc/init.d/pxe_watcher enable
/etc/init.d/pxe_watcher start

It would be nice to temporarily enable PXE boot and TFTP as well.