Creating a Full Disk Encryption system with FreeBSD 6. 2

These are the actual steps I took to create a full disk encryption system with FreeBSD 6.2. As it was a great personal exercise in patience and sanity, I hope that this guide will help others save a few more hair strings.

In this guide I will be creating a system that uses a keyfile filled with random data, that also prompts for a passphrase during boot. The /boot partition needs to remain unencrypted so it will be placed on either a usb pendrive or a bootable cd-rom.

The target audience for this guide is a beginner(ie myself) to intermediate user of FreeBSD. But I think even a seasoned veteran may find something useful in here.

WARNING!! BACKUP ANY IMPORTANT DATA BEFORE YOU PROCEED!!

MY SETUP
========

Two computers
- comp_A : which will have its hard drive (/dev/ad0) sanitised then encrypted, which has two physical cd-rom drives attached. One drive used to boot the FreeSBIE cd, the other to copy the 6.2-RELEASE files.
-comp_B : with an existing FreeBSD system.

- live cd (freesbie.org), comp_A
- disc1 of the FreeBSD 6.2 RELEASE installation CDs, comp_A
- USB pendrive

- A working internet connection to obtain all the sources, or a copy of an up to date /usr/src tree

METHOD
=======

Preparing the system
Boot with the freesbie cd, then su to root.

# su root

Sanitise hard drive (/dev/ad0) that we'll later encrypt.

# dd if=/dev/random of=/dev/ad0

Create a keyfile (ad0.key) filled with 64bytes of random data. Then initialise the disk with geli(8) with a keylength of 256 bits (default is AES, 128bit).
-b; prompts us for the passphrase at boot
-l ; sets the key length of the cryptographic algorithm
-s ; sets the sector size to 4096
-K; uppercase K, path to save the keyfile data

# dd if=/dev/random of=/tmp/ad0.key0 bs=64 count=1
# geli init -b -l 256 -s 4096 -K /tmp/ad0.key0 /dev/ad0
> Enter new passphrase:
> Reenter new passphrase:


!!COPY THIS KEYFILE STRAIGHT AWAY TO SOME PLACE SAFE!! eg a backup usb pendrive. If you lose the keyfile, you will be unable to decrypt your geli drive.

Attach the encrypted device. Notice the lowercase k

#geli attach -k /path-to/ad0.key0 /dev/ado
> Enter passphrase:
> GEOM_ELI: Device ad0.eli created.
> GEOM_ELI: Encryption: AES-CBC 256
> GEOM_ELI: Crypto: software

From now on we will be working with /dev/ad0.eli, not /dev/ad0.
Write and Edit the disk label for our encrypted device.

Note: The default editor is set to vi. If you're not familiar with vi a simple Google Search will bring up some basic tutorials. If you'd rather use another editor, such as ee;

# setenv EDITOR ee

will bring up ee whenever a command brings up something for you to edit.

# bsdlabel -w /dev/ad0.eli
# bsdlabel -e /dev/ad0.eli

You will then be presented with something similar to:

> #/dev/ad0:
> 8 partitions:
> #     size   offset   fstype   [fsize bsize bps/cpg]
> a: 1001936        2    unused       0    0     0
> c: 1001952        0    unused       0    0     0 # "raw" part, don't edit

In the a: entry, used for our root partition, change "unused" to "4.2BSD"

> #     size   offset   fstype   [fsize bsize bps/cpg]
> a: 1001936        2    4.2BSD       0    0     0

In my setup I also created swap, /home, /tmp, /var, and /usr partitions.

> #/dev/ad0:
> 8 partitions:
> #     size   offset   fstype   [fsize bsize bps/cpg]
> a: 1001936        2    4.2BSD       0    0     0
> b:   1024M        *      swap
> c: 1001952        0    unused       0    0     0 # "raw" part, don't edit
> d:    500M        *    4.2BSD       0    0     0 # /home
> e:    500M        *    4.2BSD       0    0     0 # /tmp
> f:   2046M        *    4.2BSD       0    0     0 # /var
> g:       *        *    4.2BSD       0    0     0 # /usr


In all honesty, you can leave [fsize bsize bps/cpg] blank. newfs(8) and bsdlabel(8) will figure out the correct values for you.

Each line takes 6-7 values.
- #; the partition entry, a-g in my case. DO NOT touch the c: partition entry as that applies to the whole disk.
- size; number followed by K, M or G for Kilo, Mega or Gigabytes.
- offset; the asterisk * lets newfs(8) figure out the offset automagically, otherwise it is the sector value of the end of the previous partition, plus one (ignoring c:)
- fstype; file system type, in our case 4.2BSD with "swap" for the swap partition.
- [fsize]; fragment size, set to zero to let newfs(8) sort it out.
- [bsize]; block size, set to zero to let newfs(8) sort it out
- [bps/cpg]; beats the hell out of me... set to zero to let newfs(8) sort it out.

Check the bsdlabel(8) and newfs(8) manpages if you want more info. The asterisk under the size column in the g: partition entry tells newfs to set the size of the partition to whatever space is left over.

Then save and quit.

Now we are going to create the file system on our partition. Repeat for other partitions created.

# newfs /dev/ad0.elia
# newfs /dev/ad0.elid
etc..

Note .eli[a,d-g] as we are creating filesystems on those partitions. No filesystem will be created for the swap partiton.

Now mount the 6.2-RELEASE.disc1 CD, from which the base installation and manpages will be copied. The reason I didn't boot with 6.2-RELEASE.disc1 CD is because GELI wouldn't allow me to attach a device.

# mkdir /tmp/cdrom
# mount_cd9660 /dev/iso9660/FreeBSD_Install /tmp/cdrom

Now we are going to work with the encrypted drive. The root partition will be mounted under /tmp/fixed

# mkdir /tmp/fixed
# mount /dev/ad0.elia /tmp/fixed

We will also mount the other partitions, so we need to create directories for their mount points too.

# cd /tmp/fixed
# mkdir home tmp var usr
# mount /dev/ad0.elid home
# mount /dev/ad0.elie tmp
# mount /dev/ad0.elif var
# mount /dev/ad0.elig usr

Installing the base system

Now we are going to install the base system. I also like to install the manpages and catpages.

# sentenv DESTDIR /tmp/fixed
# cd /tmp/cdrom/6.2-RELEASE/base && ./install.sh
# cd ../manpages && ./install.sh
# cd ../catpages && ./install.sh

Repeat for other distributions you choose to install.

Alternatively, you can drop to an sh(8) shell and run a for-loop to do the same

# /bin/sh
# export DESTDIR=/tmp/fixed
# cd /tmp/cdrom/6.2-RELEASE
# for i in base manpages catpages
# do
# cd $i; ./install.sh; cd..;
# done

Installing an up to date source tree

We are now going to create a usr/src tree.

Note: The following requires an internet connection, so hopefully you're behind a nicely configured firewall. Though if you aren't, the only service listening [#sockstat -46l] on the FreeSBIE live-cd is syslog so in all honesty you should be fine.

In my setup I used csup but it may be possible to use another copy of an up to date src tree if available.

[It may be possible to do the following in a chroot, though I haven't tried it]

Copy the stable-supfile to /root on the encrypted drive.

# cp /usr/share/examples/cvsup/stable-supfile /tmp/fixed/root/

Edit the stable-supfile and change a few lines to reflect the following

> *default host=CVSUP-SERVER-NEAREST-TO-YOU.FreeBSD.org
> *default base=/tmp/fixed/var/db
> *default prefix=/tmp/fixed/usr
> *default tag=RELENG_6_2

Now before you can run csup you need to make sure that a sup directory exists in var/db/sup

# mkdir /tmp/fixed/var/db/sup
# csup -L2 stable-supfile

Your usr/src directory should now be populated with an up to date source tree. Note: if you plan to use that supfile again after successfully setting your system up, remove the /tmp/fixed prefix from the base= and prefix= lines.

[I would like to know if the following works or not:
#cd /path-to/6.2-RELEASE/src && cat s*.?? | tar -zxvf - -C /usr/src/
I honestly don't know..]

Configuring, Building and Installing a Kernel

The /dev filesystem needs to be mounted as that will be needed to compile the kernel.

# mount_devfs devfs /tmp/fixed/devfs

From now on we will work in a chroot as that will be alot easier.

# chroot /tmp/fixed

Create a copy of the GENERIC kernel. Of course you can replace CUSTOM with whatever you decide to name your new kernel. What the following does is create a copy of GENERIC called CUSTOM placed in /root/kernels/, then creates a symbolic link from /usr/src/sys/i386/conf/CUSTOM -> /root/kernels/CUSTOM

# cd /usr/src/sys/i386/conf
# mkdir /root/kernels
# cp GENERIC /root/kernels/CUSTOM
# ln -s /root/kernels/CUSTOM

If you've never installed a custom kernel before, I suggest reading the FreeBSD handbook, especially the chapters on Kernel Configuration and Building and Installing a Custom Kernel.

If you know the system that you are installing to well enough, then you can edit your own customisations here also.

# vi CUSTOM

Two things to note:
- Change the ident value to the name of your new kernel
- Remove the "device kbdmux" line

Set up a couple of environment variables, then build and install the kernel.

# setenv MAKEOBJDIRPREFIX /usr/obj2 && setenv DESTDIR /
# cd /usr/src
# make buildkernel KERNCONF=CUSTOM
# make installkernel KERNCONF=CUSTOM

Check to see if (/tmp/fixed)/boot/kernel is populated. If all goes well, you've just installed yourself a functional FreeBSD 6.2 system, albeit with no way of booting into it yet as the drive is encrypted.
NOTE
From the handbook: "By default, when you build a custom kernel, all kernel modules will be rebuilt as well. If you want to update a kernel faster or to build only custom modules, you should edit /etc/make.conf before starting to build the kernel."
[This is why we populated the usr/src tree]

Preparing the boot media

An unencrypted /boot partition is necessary to load the geli module during startup so that it can decrypt the encrypted geli drive. If you haven't done so by now, exit out of the chroot(8). We will be placing /boot on either a bootable usb pendrive or a bootable cd-rom drive, either works fine.

First we will go through the steps to create a bootable usb pendrive. Following silimar steps done with the encrypted drive, we will create slice table and edit the partition table on the usb pendrive (/dev/da0).

!! BE CAREFUL NOT TO CONFUSE /dev/ad0 WITH /dev/da0 AS I HAVE DONE A COUPLE OF TIMES. YOU HAVE BEEN WARNED !!


# fdisk -BI /dev/da0
# bsdlabel -B -w /dev/da0s1
# bsdlabel -e /dev/da0s1

Similar to the encrypted drive, you will be presented with something similar to the following:

> #/dev/da0s1:
> 8 partitions:
> #     size   offset   fstype   [fsize bsize bps/cpg]
> a: 1001936        2    unused       0    0     0
> c: 1001952        0    unused       0    0     0 # "raw" part, don't edit

Change "unused" to "4.2BSD" in the a: partition entry.

> #     size   offset   fstype   [fsize bsize bps/cpg]
> a: 1001936        2    4.2BSD       0    0     0

Create a new file system on the drive.

# newfs /dev/da0s1

Mount the usb pendrive.

# mkdir /tmp/usb
# mount /dev/da0s1a /tmp/usb

Copy the encrypted /boot partition to the usb pendrive

# cp -Rpv /tmp/fixed/boot /tmp/usb

Copy the keyfile we created earlier (/tmp/ad0.key) to the usb pendrive.

# mkdir /tmp/usb/boot/keys
# cp /tmp/ad0.key0 /tmp/usb/boot/keys/

If you compiled GEOM_ELI support into the kernel, you needn't enter the geom_eli_load="YES" line. Otherwise, the following is what should be in boot/loader.conf so that the GEOM_ELI module and the keyfile are loaded during boot.

> geom_eli_load="YES"
> geli_ad0_keyfile0_load="YES"
> geli_ad0_keyfile0_type="da0:geli_keyfile0"
> geli_ad0_keyfile0_name="/boot/keys/ad0.key0"

We will now create a etc/fstab file so that geli knows where to mount the root partition.

# cd /tmp/usb
# mkdir etc && cd etc
# echo "/dev/ad0.elia / ufs rw 1 1" >> fstab

We will also need to create a etc/fstab on the encrypted drive so that the other partitions are mounted at boot also. I like to do it this way as it leaves less information unencrypted.

# vi /tmp/fixed/etc/fstab
> # Device        Mountpoint FStype Options Dump Pass#
> /dev/ad0.elib   none       swap   sw      0    0
> /dev/ad0.elid   /home      ufs    rw      2    2
> /dev/ad0.elie   /tmp       ufs    rw      2    2
> /dev/ad0.elif   /var       ufs    rw      2    2
> /dev/ad0.elig   /usr       ufs    rw      2    2

If your system supports booting from USB pendrives, then you should be able reboot and test it out.

IMPORTANT - BEFORE YOU REBOOT:
- unmount all the partitions and filesystems mounted
# cd /tmp/fixed && umount home var tmp usr devfs
# cd .. && umount /tmp/fixed
# umount /tmp/usb
# geli detach /dev/ad0
- Remove both the FreeSBIE cd and the 6.2-RELEASE discs
- Check to see in the BIOS that the usb pendrive is the first boot device

If all goes well.. CONGRATULATIONS!

Steps to creating a bootable CD

If your system doesn't support booting from a usb pendrive, then you will need access to another FreeBSD system in which you can mount the usb pendrive.

I used mkisofs(8) to create an iso image containing the contents of the usb pendrive.

# mount /dev/da0s1a /mnt && cd /mnt
# mkisofs -R -no-emul-boot -b boot/cdboot -o bootable.iso .

- R; use the Rock Ridge protocol to enable extended file names
- no-emul-boot; necessary as image is not of a floppy image
- b; boot image to use. [Afaik, this refers to /boot/cdboot]
- o; /path-to/output_file.iso
- .; last argument refers to directory to make iso image of

The full path to the new iso image should be /mnt/bootable.iso

I used cdrecord(8), part of the cdrtools package to burn the iso to a cd.

# cdrecord dev=X,X,X /tmp/usb/bootable.iso

To find the values of X,X,X run `cdrecord -scanbus` and scroll down the list to find the values of your CD recorder/burner.

Alternatively, you may be able to take the iso and burn it under another OS if that is easier for your situation.

And that... hopefully, should be it. Dont forget to unmount and detach your geli drive before you restart, otherwise you'll get errors about a dirty dismount.

Additional:
===========
To check if your iso image turned out correctly, mount and check contents:

# mdconfig -a -t vnode -f bootable.iso
> md%
# mount_cd9660 /dev/md% /cdrom

In Closing
==========
Personally, I ended up burning and throwing away about 7 cds before I got my setup right. Hopefully you won't have as many problems as I did, and on that note, I'll leave the rest of the system configuration up to you.

Criticisms, corrections (I'm sure there's plenty) and/or comments are welcome.
t41pad [at] gmail [dot] com

References
==========

"Complete Hard Disk Encryption with FreeBSD" - Marc Schiesser
22nd Chaos Communication Congress, 29 December 2005

FreeBSD-Geom Mailing List Archives
http://lists.freebsd.org/pipermail/freebsd-geom/

bootable FreeBSD on USB-Flash-Drive [Solution]
http://lists.freebsd.org/pipermail/freebsd-questions/2006-March/117738.html

FreeBSD Handbook, Chapter 18.6 - Creating and Using Optical Media(CDs)
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/creating-cds.html

FreeBSD Handbook, Chapter 8.3 - Building and Installing a Custom Kernel
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-building.html

Marc Fritsche, marc [at] proportion [dot] ch
http://www.proportion.ch/index.php?page=31

And a few people on #freebsd on irc.freenode.net. Sorry, can't remember your names...