SD cards do fail … and often at inconvenient times. Rather than face the loss of saved games, I recently configured my RetroPie to boot off the network using PXE, TFTP, and a NFS root volume.
To start, read the existing documentation on this topic at https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/net_tutorial.md. Based on that, I did the “client” work of configuring the Pi for USB boot. As I had existing DHCP (MicroTik), as well as TFTP and NFS (FreeNAS), I only needed the file system from my Pi, so I rsync’d that over ssh to the location I wanted to use on the FreeNAS host. Since I am moving the existing root filesystem from the SD card to NFS, I did not remove the ssh host config. After copying the files, I did edit /etc/fstab on the network copy, removing the /dev/mmcblkp1 and p2 lines (only proc should be left) as covered in the tutorial. Finally, before shutting down the Pi and removing the SD Card, grab the serial number by looking at /proc/cpuinfo because we will use it when configuring the TFTP server.
Next, I configured NFS within Services on FreeNAS (number of servers=4, serve UDP NFS clients=checked, Bind IP Address=checked the one listed, allow non-root mount=unchecked, enabled NFSv4=unchecked, NFSv3 ownership model for NFSv4=unchecked, require kerberos for NFSv4=unchecked, mountd bind port=, rpc.statd bind port=, rpc.lockd bind port=, support >16 groups=unchecked, log mountd requests=unchecked, log rpc.statd and rpc.lockd=unchecked). Then in Services, click Start Now to start NFS, and select Start on boot.
For the NFS Share, within Sharing on FreeNAS, I pointed to the path I wanted to use (/mnt/storage/user/retropie/root), and configured mapall user=root, mapall group=wheel, taking defaults for the rest (delete=unchecked, authorized ips/networks=, all directories=unchecked, read only=unchecked, quiet=unchecked, maproot user=, maproot group=).
Now, I’d like to say that all I had to do was copy some files over to the TFTP server and it worked; however, that is not what happened. After a bit of tcpdump and other troubleshooting, I came to find that the Pi insisted on having the TFTP server address be the same as the DHCP server address. This insistence was true for both the MicroTik router as well as an Ubiquiti EdgeRouter 6P. Regardless, what I did instead to get this working was put the Pi’s TFTP files onto the router and use TFTP on the router (instead of the FreeNAS I already had setup).
I will cover the MicroTik config first, as that is what I originally got this working on, and the EdgeRouter next. Within MicroTik, I setup the special DHCP option for the Pi. In DHCP Server, added a new Option (name=piboot, code=43, value=’Raspberry Pi Boot ‘) – note, there are three extra spaces at the end of the value based on reading https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/net.md. Then, I edited the DHCP Network to add my piboot DHCP Option. Note, on my MicroTik I have the Boot File Name set to pxelinux.0 and Next Server as 192.168.88.217, which still work (the Pi does not use these).
Then, from the Pi’s /boot directory I added these files to the MicroTik under TFTP (all are allow=checked and read only=checked):
Request Filename Real Filename
--------------------------------------------------------------
bootcode.bin bootcode.bin
bf7072fe/bcm2710-rpi-3-b.dtb bcm2710-rpi-3-b.dtb
bf7072fe/cmdline.txt cmdline.txt
bf7072fe/config.txt config.txt
bf7072fe/fixup.dat fixup.dat
bf7072fe/kernel.img kernel.img
bf7072fe/kernel7.img kernel7.img
bf7072fe/start.elf start.elf
A couple notes on the above:
- bf7072fe is the serial number of my Pi (yours will be different, check /proc/cpuinfo)
- To figure out which files were needed, I used Tools -> Packet Sniffer on the MicroTik with Filter on ports 67, 68, 69 (click Apply, then Start, then Packets)
- There were several iterations of start packet sniffer filter, reboot Pi, watch Pi fail to boot, stop packet sniffer, view Packets, tweak, try again
- There were a couple files it was looking for (based on packet sniffing) that did not exist in the /boot directory to begin with … they are not on the TFTP server and it is booting just fine
To configure the EdgeRouter, I started with the DHCP option 43. There are a couple ways you can do this from CLI to GUI, and I did both. I will cover the GUI here since it may be easier for some folks. In EdgeOS, go to the Config Tree and drill down to service, dhcp-server. We will add a global-parameter (click Add):
option option-43 code 43 = string;
Next, drill down into shared-network-name, the LAN the Pi is on, subnet, the subnet address, and add a subnet-parameter:
option option-43 "Raspberry Pi Boot ";
Click Preview then Save. If you want another reference, there is a support article on Defining Custom DHCP Options.
You can confirm your changes (and the formatting result of " if you wish) by looking at /opt/vyatta/etc/dhcpd.conf (on the CLI, cat /opt/vyatta/etc/dhcpd.conf):
# ... only showing relative lines here ...
# The following 1 lines were added as global-parameters in the CLI and
# have not been validated
option option-43 code 43 = string;
shared-network LAN_1 {
not authoritative;
subnet 192.168.1.0 netmask 255.255.255.0 {
option domain-name-servers 192.168.88.81, 192.168.88.90;
# The following 1 lines were added as subnet-parameters in the CLI and have not been validated
option option-43 "Raspberry Pi Boot ";
option routers 192.168.1.1;
option bootfile-name "pxelinux.0";
filename "pxelinux.0";
next-server 192.168.88.217;
default-lease-time 86400;
max-lease-time 86400;
range 192.168.1.100 192.168.1.254;
}
}
You can see I still have the “normal” bootfile-name and bootfile-server configured on this subnet, because the Pi ignores them.
Next, I had to install TFTP on EdgeOS (because it is not one of the “built-in” services). There is a support article that covers Adding Debian Packages to EdgeOS:
configure
set system package repository wheezy components 'main contrib non-free'
set system package repository wheezy distribution wheezy
set system package repository wheezy url http://http.us.debian.org/debian
commit
save
sudo apt-get update
sudo apt-get install tftpd-hpa
To then configure TFTP on EdgeOS, I setup the directory structure:
sudo mkdir -p /config/user-data/tftp/bf7072fe
sudo cp bootcode.bin /config/user-data/tftp/bootcode.bin
sudo cp bcm2710-rpi-3-b.dtb /config/user-data/tftp/bf7072fe/bcm2710-rpi-3-b.dtb
sudo cp cmdline.txt /config/user-data/tftp/bf7072fe/cmdline.txt
sudo cp config.txt /config/user-data/tftp/bf7072fe/config.txt
sudo cp fixup.dat /config/user-data/tftp/bf7072fe/fixup.dat
sudo cp kernel.img /config/user-data/tftp/bf7072fe/kernel.img
sudo cp kernel7.img /config/user-data/tftp/bf7072fe/kernel7.img
sudo cp start.elf /config/user-data/tftp/bf7072fe/start.elf
sudo chown -R root:vyattacfg /config/user-data/tftp
… and update the configuration file /etc/default/tftpd-hpa:
TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/config/user-data/tftp"
TFTP_ADDRESS="192.168.1.1:69"
TFTP_OPTIONS="--secure"
You will see I modified the default address from 0.0.0.0 to a specific internal only address on the LAN where the Pi was because I did not want the TFTP server running on the Internet. Finally, a quick restart was all that was needed to get TFTP running (sudo /etc/init.d/tftpd-hpa stop ; /etc/init.d/tftpd-hpa start).
Once you have all the files the Pi needs on the TFTP server, and the NFS volume working, it should network boot as expected.