Setting up a Multi-Boot Linux USB Stick

My friend Daniel has recently issued a new version of his "grubs" tool for building a multi-boot USB stick. I immediately checked out his repo and made myself a USB stick. For some ISOs this works very well, but for others, the process of trying to get them to boot verges on nightmarish. I've just spent about 15 hours on this ...

The usual way of making installation media is this (Linux command line):

# dd bs=4M if=/path/to/bootable.iso of=/dev/sdx status=progress && sync

(Example from the ever-excellent ArchWiki ... but the "status=progress" option isn't universally available, check your man page.) This you would only do after confirming:

  • /dev/sdx is a valid USB device
  • checking about three times you're okay with wiping /dev/sdx

This is an extremely reliable way of creating a bootable USB stick: so long as you can convince the target computer to boot from USB, you're in business.

But - what if you want to be able to install Debian Stable, Fedora, and Debian Testing, and also boot Knoppix, Tiny Core Linux, and the GParted Live CD? You need to keep wiping your USB stick, or you need six USB sticks (and who can keep track of which is which anyway?). GRUB2 (actually I think it was GRUB 1.98, but that's getting pretty technical) introduced a mechanism to boot install media from an ISO image. This can be used to boot an ISO from the same hard drive as your operating system, or to create a USB stick that can boot multiple ISOs - and have the leftover space available as regular USB storage.

Unfortunately, there are a whole slew of catches:

  • installing GRUB2 to USB is somewhat non-obvious (Daniel's grubs.sh command helps immensely with this)
  • GRUB2 can chainload an OS on another partition (such as Windows or one of the BSDs), but can't chainload an ISO syslinux menu: this means you can boot a kernel on the ISO, but you can't see the same menu as if you'd dd-ed that ISO onto a USB stick (more frustrating: grub4dos appears to be able to do this)
  • the invocation for every ISO is different, and some involve significant hoop-jumping
  • some ISOs have defied my attempts to boot them from multiboot USB for three days: in a practical sense, I now consider them "impossible" to boot (probably not true, but "tricky" seems inadequate)
  • you'll require a good chunk of space for ISO files - twice - both on the USB stick and in the boot/iso/ subfolder of Daniel's 'grubs' repository
  • because vfat is the preferred filesystem of the multi-bootable USB, there's a nasty catch for large ISOs (I found this out with the current Knoppix DVD and ended up using the last of the older Knoppix CD images instead): if the image is larger than 4 GB, it can't be copied to a vfat partition. I've considered messing about with creating a separate partition to bypass the problem, but I doubt GRUB can access an exfat partition so ext4 is probably called for and that presents other problems.
  • booting behaviour appears to be different between different machines, notably when going from 32-bit systems to 64-bit systems: while 64-bit ISOs won't boot on 32-bit systems, 32-bit ISOs should boot on 64-bit systems ... but I've seen some fail even though they worked correctly on 32-bit systems

The Process

  • clone Daniel's "grubs" github repository
  • download ISOs you want to boot into the grubs/boot/iso/ folder
  • copy grubs/boot/grub/grub.cfg.sample to grubs/boot/grub/grub.cfg and edit it - HEAVILY
  • the primary unit you'll be working with is a GRUB2 'menuentry' similar to this:
menuentry 'SystemRescueCD 5.0.0 32-bit' {
    set isofile='/boot/iso/systemrescuecd-x86-5.0.0.iso'
    loopback loop $isofile
    linux (loop)/isolinux/rescue32 isoloop=$isofile setkbmap=us
    initrd (loop)/isolinux/initram.igz
}
  • this should work for that exact ISO, but changing it to work with another ISO requires a lot more than just changing the location of isofile
  • every single ISO has the initrd in a different place, with a different name
  • every single ISO has the kernel (the line starting with "linux") in a different place, with a different name
  • every single kernel needs to be invoked with a different set of parameters - and I don't think which parameters are available are even remotely consistent from one kernel to the next
  • when an ISO changes from one version to the next, there's a fair chance that both the initrd and the kernel will move (because the maintainers aren't thinking about this model for booting their ISOs), and an even better chance that the boot parameters will change
  • if you're attempting to use Debian's installer disks, you have even more hoops to jump through because the 'initrd' doesn't have 'iso-scan' so it can't even find the damn ISO it booted from (see The Debian Fix)
  • to figure out the location of the initrd and kernel, and find what parameters are needed to make the kernel happy:
    • loop mount the ISO: mount -o loop /home/giles/Downloads/slacko-6.3.2-uefi.iso /mnt/tmp
    • figure out where the initrd and kernel are by exploring the mounted filesystem
    • look at the isolinux.cfg file to see what added parameters are needed: cat /mnt/tmp/isolinux.cfg (or maybe it's syslinux.cfg)
    • the above are examples: the files are usually somewhere else and you have to explore to find them
    • don't forget to umount the loopback filesystem when you're done
  • create a menuentry for every ISO you want to boot
  • run ./grubs.sh sdx1 when you're ready to construct your USB stick (again, making damn sure you got the value of sdx1 correct)
  • boot the stick in another computer, and write extensive notes about how each menuentry fails
  • get on the web and research the failure, making more notes
  • edit grub.cfg again, updating from your notes
  • run ./grubs.sh sdx1 again, this time choosing the "UPDATE" option rather than the "INSTALL" option from the menu
  • rinse and repeat, many, many times

The Debian Fix

I don't fully understand what causes the problem with the Debian-based installer ISOs, but if you use one of them you're likely to find that they boot fine ... but when you get to the point in the installation where Debian tries to find its own CD image, it will fail. This seems very odd because it just booted from it, but there it is. I'm told this is because the Debian initrd doesn't include iso-scan. The way to fix this is to download an initrd image from the matching hd-media/ folder for the distro. This is detailed both at Daniel's site and at the site he got it from. It's ugly, but it worked not only for the Debian Jessie 32-bit installer, but also for the Debian-derived 'Bunsen Labs' disc (and I used the Debian initrd, not their own). However, it did NOT work for the Debian Stretch 32-bit installer.