OpenBSD Journal

OpenBSD adds boot(8) support for keydisk-based softraid crypto volumes

Contributed by jcr on from the shake-your-key-disk dept.

On i386 and amd64 boot(8) support has been added for keydisk-based softraid crypto volumes. Undeadly editor Sean Cody (sean<at>tinfoilhat<dot>ca) did some testing and wrote in to tell us how to use this feature.

CVSROOT:	/cvs
Module name:	src
Changes by:	stsp@cvs.openbsd.org	2013/10/20 07:25:21

Modified files:
	sys/arch/amd64/stand/boot: conf.c 
	sys/arch/amd64/stand/libsa: softraid.c 
	sys/arch/i386/stand/boot: conf.c 
	sys/arch/i386/stand/cdboot: conf.c 
	sys/arch/i386/stand/libsa: softraid.c 
	sys/arch/i386/stand/pxeboot: conf.c 

Log message:
Add i386/amd64 boot(8) support for keydisk-based softraid crypto volumes.
So far, only passphrase-based crypto volumes were bootable. Full disk
encryption with keydisks required a non-crypto partition to load the
kernel.

The bootloader now scans all BIOS-visible disks for RAID partitions and
automatically associates keydisk partitions with their crypto volume.
Attempting to boot from a volume without its keydisk currently results
in a passphrase prompt (this might be changed in the future).

There is no need to re-create existing volumes. Moving the root
partition onto the crypto disk and running installboot(8) is all that's
needed.

help & ok jsing

Sean wrote in with the following:

Using keydisk based crypto volumes is very easy to setup and fun. Adding boot support makes using keydisks less of a pain to setup since your entire root volume can now be encrypted and deciphered by the key on an external volume/disk (like a USB or SD card). Prior to this patch the way to get this to work was to setup a root volume containing only the kernel that was bootable and then do the rest of the install on the encrypted volume.

There have been a few articles on how to do this in the past such as a blog post by one Ryan Kavanagh and another blog post by Bryan Vyhmeister. One thing slightly amiss with both articles is how they give some slightly awkward advice on how to handle swap slices. OpenBSD has encrypted the swap partition by default since 2005, but not putting the swap slice in the crypto volume is well, inadvisable. The result of putting swap on the crypto volume is a performance penalty caused by encrypting twice. While I understand and respect the idea of keeping swap outside of the crypto volume, doing this can be problematic for how the kernel finds and saves crash core dumps on warm reboots with savecore(8).

By default, the kernel expects crash core dumps to be written to swap on the 'b' slice of the boot volume, and if found, savecore(8) is called from rc(8) to write the core dump to a file in /var/crash on reboot. This isn't a savecore(8) specific option, but instead, it's a kernel option, so changing the dump device requires modifying the kernel options and rebuilding. Since running GENERIC is always advised, running a custom kernel for a single feature seems a bit excessive. If you are using the GENERIC kernel and mistakenly use the 'b' slice of the boot volume for something else like /usr or /home, you might be ok since it is not defined as a swap partition in /etc/fstab, but you have lost the ability of the kernel to record crash state and therefore lost the ability to debug a kernel panics. My recommendation is to keep a swap volume on the host disk AND one on the crypto volume, then in /etc/rc.local remove the crypto volume's swap slice from the active swap set. This gives you the best of both worlds at the cost of a bit of disk space.

$ grep '^swapctl' /etc/rc.local
swapctl -d /dev/sd0b

Since /etc/rc runs before /etc/rc.local, the 'b' slice of the boot volume exits as a swap device when it's needed during warm reboots for saving crash dumps to /var/crash with savecore(8). Since you're disabling the 'b' slice on the crypto volume with swapctl(8) afterwards, you no longer suffer the double encryption performance penalty. None the less, the 'b' slice still exists if you need to write an initial crash dump with:

ddb> boot crash

Though you can use the 'dumps on' directive in the configuration file used with config(8) to change the device where crash(8) dumps are saved, the result would be you're running an unsupported, custom kernel. The default (below) uses the 'b' slice as the target device for the 'dumps on' directive:

$ grep '^config' /usr/src/sys/arch/i386/conf/GENERIC
config          bsd     swap generic

Another thing to consider is how you are going to manage your keydisk. Having only one copy of the key is, well, ill advised. If the USB stick fails, your machine is no longer bootable and your data (reasonably) unrecoverable. USB sticks are only slightly more reliable than floppies, and the reliability of floppies is horrible.

A safe way to manage keydisks is to create and test multiple keydisks and also keep a disk image of the key volume in a safe place. This is pretty easy to do if you plan out how you are going to manage your keydisk. For instance I would prefer to have one keydisk drive for all my systems, so I don't have to fumble with a stack of flash drives. I'll create a custom label and use /dev/sd0a for system A, and /dev/sd0d for system B etc. I'll also have another MSDOS partition on the USB stick to store copies of the slices and make it easy to transfer to a trusted backup host. Disk space doesn't matter much for the keydisk slices. You can make them 1MB with no issues, and go as small as 64 sectors (i.e. 16KB). Smaller may be possible but I have not tried personally. From looking at the softraid(8) source for a bit, (specifically softraid.h,softraid_crypto.c), it appears the minimum space is roughly 1KB since you need at least 32 keys times 32 bytes per key (256bits for AES-XTS-256). This is a bit opaque to those like myself unfamiliar with the softraid code base. The point is to not be too skimpy. I would suggest sticking with 1MB as it isn't really that big relative to the available flash media today and it's easier to handle. For example, a keydisk I'm using to play with this feature is a 16GB stick which is laid out like:

$ sudo disklabel -p m sd1
# /dev/rsd1c:
type: SCSI
disk: SCSI disk
label:
duid:
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 1946
total sectors: 31277056 # total bytes: 15272.0M
boundstart: 0
boundend: 31277056
drivedata: 0

16 partitions:
#                size           offset  fstype [fsize bsize  cpg]
  a:             1.0M         29296882    RAID
  b:             1.0M         29298930    RAID
  c:         15272.0M                0  unused
  i:         14305.1M                2   MSDOS

A clean and safe way to backup and restore the keydisk softraid metadata is to use dd(1) and seek in 8192 bytes so as to avoid the disklabel on the drive.

Backup:

dd bs=8192 skip=1 if=/dev/rsd1a of=backup-keydisk.img
Restore:
dd bs=8192 seek=1 if=backup-keydisk.img of=/dev/rsd1a

To visually check or inspect the keydisk the handy hexdump(1) can be used.

$ sudo hexdump -C /dev/sd1a | head -15  
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002000  6d 61 72 63 43 52 41 4d  05 00 00 00 00 00 00 00  |marcCRAM........|
00002010  b9 18 57 73 b4 30 47 eb  b2 71 2e ba b9 05 34 d2  |..Ws.0G..q....4.|
00002020  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
00002030  fe ff ff ff fe ff ff ff  00 00 00 00 00 00 00 00  |................|
00002040  4f 50 45 4e 42 53 44 00  53 52 20 4b 45 59 44 49  |OPENBSD.SR KEYDI|
00002050  53 4b 00 00 00 00 00 00  30 30 35 00 00 00 00 00  |SK......005.....|
00002060  c6 41 01 52 4c dc 9e 39  fd 2b 7a 89 70 dd 31 e9  |.A.RL..9.+z.p.1.|
00002070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002090  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
000020a0  00 00 00 00 00 00 00 00  43 00 00 00 00 00 00 00  |........C.......|
000020b0  73 64 30 61 00 00 00 00  00 00 00 00 00 00 00 00  |sd1a............|
000020c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
$ 

Note: All softraid volumes start at 0x2000 (8192) bytes from the beginning of the volume and start with the string 'marcCRAM'. Also note the device ID at 0x20b0, while the softraid driver should manage the devices moving around (ie. detected out of order). Just keep it in mind in case you are having issues (ie. keep the slice arrangement consistent on your keydisks... may not be necessary but I had some issues when playing with it myself.

The final proviso is that your keydisk needs to be discoverable from the BIOS/EFI. If the machine you want to play with can boot from USB then you should be ok.

If you wonder why using a keydisk would be useful, then consider how the keydisk is required to boot the system and can be removed immediately after the encrypted volume is mounted. From a physical security perspective, this is a pretty handy feature assuming you have a UPS and don't mind traveling to the machine location to boot the system. Another neat use is for laptops, as this can give you full disk encryption without having to resort to TrueCrypt or similar. You only need to have the keydisk (USB, SD card etc.) plugged in to boot, so if you are disciplined about ejecting the keydisk, then the laptop data is somewhat more secured if stolen.

Like most use of RAID, a keydisk setup is only as reliable as your ability to recover from it, so make sure to backup what you care about, and of course, practice and validate your procedures and tools.

(Comments are closed)


Comments
  1. By Anonymous Coward (209.132.186.34) on

    I hope next step would be dual factor - keydisk and passphrase :)

    Comments
    1. By sean (64.42.217.69) sean <at> tinfoilhat <dot> ca on I don't work here.

      > I hope next step would be dual factor - keydisk and passphrase :)

      That would not be trivial as it would involve messing with and adding to the softraid metadata structure.

      Definitely interesting idea but not sure how much 'support' there would be to push this. If I ever find myself bored out of my tree I may consider messing with this but there are other features I want more (like dhcpd updating pf tables with leases and expiry (especially if your dhcpd daemon is hosted separate from your firewall) and ability to add more of the dhcp options like 82 for wpad).

      Comments
      1. By phessler (phessler) on why in god's name am I wearing pants?

        > there are other
        >features I want more (like dhcpd updating pf tables with leases and expiry (especially if your
        >dhcpd daemon is hosted separate from your firewall)

        man dhcpd. Look closely at the command-line options (yes, this probably should go into the conf file).

        > and ability to add more of the dhcp >options like 82 for wpad).


        dhcp options are pretty simple to add. You can either use the generic method, or write a little bit of code.

        Comments
        1. By sthen (2001:8b0:648e:cc01:f2de:f1ff:fef9:a752) on

          > > there are other
          > >features I want more (like dhcpd updating pf tables with leases and expiry (especially if your
          > >dhcpd daemon is hosted separate from your firewall)
          >
          > man dhcpd. Look closely at the command-line options (yes, this probably should go into the conf file).

          I don't think we have any way to sync the firewall tables (even running dhcp's sync protocol doesn't cause the IPs to be added to the tables).

          > > and ability to add more of the dhcp >options like 82 for wpad).
          >
          >
          > dhcp options are pretty simple to add. You can either use the generic method, or write a little bit of code.
          >

          WPAD is option 252, and already available as "option autoproxy-script", see dhcp-options(5). Option 82 is used by dhcp relay agents like dhcrelay(8) (though OpenBSD dhcpd can't hand out different settings to clients based on the option-82 setting).

      2. By mcbride (116.214.126.4) on

        > > I hope next step would be dual factor - keydisk and passphrase :)
        >
        > That would not be trivial as it would involve messing with and adding to the softraid metadata structure.

        Nah, I have a diff that supports this which I have been using on my laptops since the general hackathon this summer and it works really nicely for me. No change to the metadata, all is handled in dev/softraid.c. Unfortunately it's a bit of a hack so it might never make it into the tree, but this is how it works:

        Basically, create a softraid crypto volume on your USB, with password authentication. Set up a small 'a' partition with ffs to hold your boot kernel, boot.conf, biosboot, and your keyfile RAID partitions in other slices of the RAID softraid crypto. Build a kernel with something like the following, put the kernel on the USB RAID's /a partition and invoke the correct magic with installboot. (put bsd.rd here too, just in case)

        config bsd root on sd3a

        To use, plug in the USB before powering on and boot from it (with the added bonus that perhaps you can better secure the USB and gain some assurance that the bootloader has not been tampered with). Enter the password to unlock the USB, and then the kernel loads it loops through to find the key partitions inside the USB softraid (this is the part my diff handles). So the devices come up something like this.

        sd0 (Laptop Drive)
        a - RAID

        sd1 (USB) <- boot from this device
        a - RAID

        sd2 (Softraid Crypto on sd1a)
        a - ffs /w kernel and bootloader
        d - RAID (keydisk)
        e - RAID (keydisk)
        ...

        sd3 (Softraid Crypto on sd0a) <- OpenBSD installed here
        a - / <- but no kernel needed here.
        b - swap
        d - /var
        ...

        Setting it up is a bit fiddly in the shell before running the installer, but it works quite well and I can boot multiple laptops with the same USB. Another benefit is this can make 'dual boot' much less painful. You can let the other unfortunate OS do all it's own booting without touching the bootloader.




Credits

Copyright © - Daniel Hartmeier. All rights reserved. Articles and comments are copyright their respective authors, submission implies license to publish on this web site. Contents of the archive prior to as well as images and HTML templates were copied from the fabulous original deadly.org with Jose's and Jim's kind permission. This journal runs as CGI with httpd(8) on OpenBSD, the source code is BSD licensed. undeadly \Un*dead"ly\, a. Not subject to death; immortal. [Obs.]