RAID-1 with GPT on FreeBSD

In this handbook we will see how you can create a software RAID-1 with GPT on a FreeBSD host.

This setup has been tested and is working properly under FreeBSD 8.0.

In the following handbook we will see how to setup a freshly installed FreeBSD 8.0 system with RAID-1 support and GPT.

You can also use this guide in order to migrate your MBR to GPT.

One more thing to add about this documentation is that we are going to create mirror of the partitions instead of the whole disks.

NOTE: In this documentation we will see how to create a RAID-1 on a FreeBSD host. During the implementation of the RAID-1 we will be creating mirror for the partitions, instead of the whole disks.

Some of you may be wondering what is the benefit of this approach, so here is a short list of pros and cons when using this approach.

Pros for creating mirror of the partitions, instead of the whole disks:

  • Able to mirror different in size disks
  • Can still create mirror for identical disks
  • Able to dual-boot between different OS

Cons for creating mirror of the partitions, instead of the whole disks:

  • In case of a disk failure you need to manually create all the partitions on the newly inserted disk(s)

NOTE: Starting from FreeBSD 8.2 gpart(8) now supports two additional options - backup and restore, which makes it very easy to recover your partitions when needed.

So to summarize this again - in this handbook we will show you a different approach of creating a RAID-1 on a FreeBSD host for partitions only and also how you can migrate from MBR to GPT.


  • root access
  • 2 or more physical disks in order to create a RAID-1

Preparing the system for GPT

There are a lot of different approaches you can use in order to get FreeBSD booting from a GPT partition table. Currently sysinstall does not have support to create GPT scheme on your disks, so we will have to do this manually.

With the above being said, lets go into the real stuff..

Make a fresh FreeBSD install

The first method demonstrates how we can make a fresh FreeBSD install with RAID-1 and GPT.

NOTE: During the installation we will refer to the first disk as ad4 and the second disk as ad6.

Install FreeBSD 8.0 or later. Make partitions only on the first disk.

When you reach the Disklabel Editor menu, create the following partitions on the first slice of the first disk. We won’t be touching the second disk at all during the installation of the system.

Create the following partitions - root, swap, usr, tmp and var. Each of these partitions is of the following sizes: root - 1GB, swap - 1GB, tmp - 1G, usr - 4GB and var - 1G.

NOTE: The reason that I’ve chosen these sizes for the partitions is that these partitions are actually going to be temporary and used only during the installation. Later in this document we will be re-creating the disk partition scheme.

So after I’ve created my partitions on the first disk, the layout is as follows:

Part      Mount     Size    Newfs
-----     -----     ----    -----
ad4s1a    /       1024MB    UFS2
ad4s1b    swap    1024MB    SWAP
ad4s1d    /var    1024MB    UFS2+S
ad4s1e    /tmp    1024MB    UFS2+S
ad4s1f    /usr    4096MB    UFS2+S

Perform a Minimal install and go throught the post-install configuration steps.

Creating backups

Now that you have installed FreeBSD, boot into your freshly installed system and create backups of the partitions you have. We will restore their contents later when we create the GPT disk scheme.

You may want to use an external USB disk to store the backups or copy the backups on some other machine once you are done with them. In the following example I’m using an external USB disk to store the backups

  • da0.

So let’s go ahead and create those backups.

Create a filesystem on the USB disk, if you haven’t done this already.

newfs -O2 -U /dev/da0

Now mount the USB disk:

# mkdir -p /mnt/usb
# mount /dev/da0 /mnt/usb

Now make a backup of each partition you have - root, var, tmp, usr.

# dump -0aL -f /mnt/usb/root.dump /
# dump -0aL -f /mnt/usb/tmp.dump /tmp
# dump -0aL -f /mnt/usb/var.dump /var
# dump -0aL -f /mnt/usb/usr.dump /usr

If you do not have a USB disk, you may want to copy over the backups to another machine using OpenSSH. To do this just create the backups this way:

# dump -0aL -f - / | ssh user@host dd of=/path/to/dumps/root.dump
# dump -0aL -f - /tmp | ssh user@host dd of=/path/to/dumps/tmp.dump
# dump -0aL -f - /var | ssh user@host dd of=/path/to/dumps/var.dump
# dump -0aL -f - /usr | ssh user@host dd of=/path/to/dumps/usr.dump

Once ready with the backups, unmount the USB disk if you have used one:

# umount /mnt/usb

Now we can continue to the next step - creating the GPT disk scheme.

Booting from a Fixit media

For this step we are going to boot into Fixit mode - for the purpose, you can use the livefs image that can be found on the FreeBSD FTP Server.

Once you’ve booted from a Fixit media, go to Fixit -> CDROM/DVD or** Fixit -> USB** whether you are booting from CD or USB.

Now you can see the partitions we have created during the installation of the system if you execute the following command:

Fixit# gpart show ad4
=>      63  488397101    ad4  MBR  (233G)
        63  488397038      1  freebsd  [active]  (233G)

=>       0  488397038  ad4s1  BSD  (233G)
         0    2097152      1  freebsd-ufs  (1.0G)
   2097152    2097152      2  freebsd-swap (1.0G)
   4194304    2097152      4  freebsd-ufs  (1.0G)
   6291456    2097152      5  freebsd-ufs  (1.0G)
   8388608    8388608      6  freebsd-ufs  (4.0G)
  16777216  471619822         - free -     (225G)

Now we are going to delete these partitions, and create GPT ones. So execute the following gpart(8) commands

Fixit# gpart delete -i 1 ad4
ad4s1 deleted
Fixit# gpart show ad4
=>      63  488397101    ad4  MBR  (233G)
        63  488397101         - free - (233G)
Fixit# gpart destroy ad4
ad4 destroyed

You can now check that we have successfully removed the partitions that were created during the installation of our FreeBSD system. Just execute the following gpart command, so you are sure that we have removed those partitions.

Fixit# gpart show

Creating GPT partition tables

Now we can create the GPT tables on our disk. In this step we will create our partitions with the sizes we want.

You may want to choose even bigger sizes for the partitions since once we create them, we cannot resize them, so choosing bigger partition sizes is something you should consider doing.

In the commands below I’m creating the following partitions with these sizes: root - 2G, swap - 2G, var - 20G, tmp - 2G, usr - 20G and for home - the rest of the disk. So let’s create these partitions.

Create the GPT table on the disk:

Fixit# gpart create -s gpt ad4
ad4 created

And now create the GPT partitions, including the boot one:

Fixit# gpart add -b 34 -s 128 -t freebsd-boot ad4   # GPT boot partition
Fixit# gpart add -s 2G -t freebsd-ufs ad4           # root partition
Fixit# gpart add -s 2G -t freebsd-swap ad4          # swap partition
Fixit# gpart add -s 20G -t freebsd-ufs ad4          # var partition
Fixit# gpart add -s 2G -t freebsd-ufs ad4           # tmp partition
Fixit# gpart add -s 20G -t freebsd-ufs ad4          # usr partition
Fixit# gpart add -t freebsd-ufs ad4                 # home partition

We also need to install the bootcode, so FreeBSD is able to boot from our GPT partition:

Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptboot -i 1 ad4

Now verify the partitions are created using the gpart command:

Fixit# gpart show ad4
=>       34  488397101  ad4  GPT  (233G)
         34        128    1  freebsd-boot (64K)
        162    4194304    2  freebsd-ufs  (2.0G)
    4194466    4194304    3  freebsd-swap (2.0G)
    8388770   41943040    4  freebsd-ufs  (20G)
   50331810    4194304    5  freebsd-ufs  (2.0G)
   54526114   41943040    6  freebsd-ufs  (20G)
   96469154  391927981    7  freebsd-ufs  (187G)

Now we can create filesystems on the partitions - no soft-updates for the root partition:

Fixit# newfs -O2 ad4p2       # root partition
Fixit# newfs -U -O2 ad4p4    # var partition
Fixit# newfs -U -O2 ad4p5    # tmp partition
Fixit# newfs -U -O2 ad4p6    # usr partition
Fixit# newfs -U -O2 ad4p7    # home partition

If you are running FreeBSD 8.2 or later you may consider using the backup option to gpart(8), which will dump the partition table of the given provider. This way later on you can restore the partition tables on another provider easily just by using the restore option of gpart(8).

Now that we have created the GPT scheme and partitions we can restore the data of our system on them.

Restoring data from backups

Now we can restore the data we previously made a backup of.

Mount the USB disk, or manually configure the network device if you have copied the backups on a remote system in order to get them back.

You might want to restore the tmp partition first and restore it’s data, and then set the TMPDIR environment variable first, so we can take advantage of it’s space during the restore.

In the commands below /mnt/usb is the location we mount the USB disk on, /mnt/tmp-partition is the tmp partition’s mount point - ad4p5, and /mnt/partition we will use as a mount point for the rest of the partitions - root, var, usr and home.

So let’s restore tmp first:

Fixit# mkdir -p /mnt/usb
Fixit# mkdir -p /mnt/tmp-partition
Fixit# mkdir -p /mnt/partition
Fixit# mount /dev/da0 /mnt/usb
Fixit# mount /dev/ad4p5 /mnt/tmp-partition
Fixit# export TMPDIR=/mnt/tmp-partition
Fixit# cd /mnt/tmp-partition
Fixit# restore -rf /mnt/usb/tmp.dump

Now we can restore the rest of the partitions. In the commands below replace partition with your actual partition - ad4p2, ad4p4 and ad4p6, ad4p7 and also replace dumpfile-name with the actual filename of the dump you restore from - root.dump, var.dump and usr.dump.

Fixit# mount /dev/<partition> /mnt/partition
Fixit# cd /mnt/partition && restore -rf /mnt/usb/<dumpfile-name>
Fixit# cd /mnt
Fixit# umount /mnt/partition

Repeat the commands above until you restore all of the partitions.

One last thing we need to setup is to edit fstab file. So mount the root partition again and open the file for editing. Once you’ve done editing your fstab file it would look like something like this:

# Device                Mountpoint      FStype  Options         Dump Pass#
/dev/ad4p3              none            swap    sw              0    0
/dev/ad4p2              /               ufs     rw              1    1
/dev/ad4p5              /tmp            ufs     rw              2    2
/dev/ad4p6              /usr            ufs     rw              2    2
/dev/ad4p4              /var            ufs     rw              2    2
/dev/ad4p7              /usr/home       ufs     rw              2    2
/dev/acd0               /cdrom          cd9660  ro,noauto       0    0

And that’s it! You are now ready to boot into your FreeBSD system! Just remove the Fixit media and reboot.

Creating software RAID-1

Now it is time to create the GPT partition tables on the second disk.

Please refer to the Creating GPT partition tables chapter, for information on how to do this.

Again, if you are running FreeBSD 8.2 or later, consider using the backup and restore options of gpart(8) for easy recovering of partition tables on a different provider.

Now boot your FreeBSD system into single-user mode. From the loader prompt execute the following command:

OK boot -s

You are now in single-user mode. The first thing that we are going to do is to edit the fstab file, so next time we boot into FreeBSD we’ll be using the mirrored devices. Now mount the root partition in read-write mode, and update the fstab file:

# mount -u /
# cat << __EOF__ > /etc/fstab
> # Device                Mountpoint      FStype  Options         Dump    Pass#
> /dev/ad4p3              none            swap    sw              0       0
> /dev/ad6p3              none            swap    sw              0       0
> /dev/mirror/gm0-root    /               ufs     rw              1       1
> /dev/mirror/gm0-tmp     /tmp            ufs     rw              2       2
> /dev/mirror/gm0-usr     /usr            ufs     rw              2       2
> /dev/mirror/gm0-var     /var            ufs     rw              2       2
> /dev/mirror/gm0-home    /usr/home       ufs     rw              2       2
> /dev/acd0               /cdrom          cd9660  ro,noauto       0       0
> __EOF__

Do not worry about all those /dev/mirror entries if it’s something unknown for you. You will see how we create these devices in a few steps later.

We also need to load the geom_mirror kernel module during boot-time, so execute the following command to do so:

# echo 'geom_mirror_load="YES"' >> /boot/loader.conf

Now we can create the mirrors.

NOTE: Please note that the first disk, that FreeBSD is currently booting from needs to be the first component we add to the mirror! Otherwise you will end up with empty partitions if you add the second disk as first component!

Creating a mirror for the root partition:

# gmirror label -vb round-robin gm0-root /dev/ad4p2
# gmirror insert gm0-root /dev/ad6p2

Creating a mirror for the tmp partition:

# gmirror label -vb round-robin gm0-tmp /dev/ad4p5
# gmirror insert gm0-tmp /dev/ad6p5

Creating a mirror for the var partition:

# gmirror label -vb round-robin gm0-var /dev/ad4p4
# gmirror insert gm0-var /dev/ad6p4

Creating a mirror for the usr partition:

# gmirror label -vb round-robin gm0-usr /dev/ad4p6
# gmirror insert gm0-usr /dev/ad6p6

Creating a mirror for the home partition:

# gmirror label -vb round-robin gm0-home /dev/ad4p7
# gmirror insert gm0-home /dev/ad6p7

While the mirror is rebuilding you may monitor it’s status with the following command:

# gmirror status

Once the mirrors have been rebuilt you should see something similar:

# gmirror status
           Name    Status  Components
mirror/gm0-root  COMPLETE  ad0p2
 mirror/gm0-var  COMPLETE  ad0p4
 mirror/gm0-tmp  COMPLETE  ad0p5
 mirror/gm0-usr  COMPLETE  ad0p6
mirror/gm0-home  COMPLETE  ad0p7

And that was it! Now you can boot your FreeBSD system from the mirrored devices we have created.

Written on January 12, 2011