[afnog] FreeBSD mirroring

[Phil Regnauld]

On Thu, Apr 27, 2006 at 05:32:49PM +0200, Antonio Godinho wrote:
> Hi All, 
> 
> I would like to mirror an already operational FreeBSD server running FreeBSD 
> 6.0. I have read the instructions at 
> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom-mirror.html 
> 
> but I am not quite sure of what I will do specially when I have more than 
> the two partitions they reccommend. I have /, swap, /usr and /var. 

    Mirror the whole disk!  It doesn't matter how many partitions you have.

> Has anyone done this before? I don't want to ruin the already operational 
> installation and would like to have some additional pointers. 

    Plenty :)  Got 10-15 boxes running it (plus many others running ATAPI
    RAID/ar, ccd, vinum, raidframe -- I've tried everything :)

> The system is running on a HP server which now has two hotswap scsi disks 
> and the FreeBSD 6.0 is running in one of them (id 0). 

    Which HP server is it ?  Is it a Proliant ?  If so, they all have
    hardware RAID...

> I don't want to find that after going through these steps the system will 
> fail to boot. 

    It's easy to do -- what I recommend is doing it via the live
    FS CD.  Alternatively, you can do it by:

    - installing a minimal FreeBSD on the disk that will be the target
      (blank disk)
    - boot on it
    - create the gmirror label on the source disk (the one where all your
      data is) -- the label is 512 bytes written at the end of the
      disk / slice / partition you use:

    - change fstab on the source disk's fstab to use /dev/mirror/* devices
    - add geom_mirror_load="YES" to the source disk's /boot/loader.conf

    - reboot from the source disk, watch it magically discover the gmirror
      volume
    - check everything is running, then insert the spare disk into the
      raid with gmirror insert

    In practice, if you have disks da0 and da1, and we have:

    da0 = source, master, good disk with data
    da1 = destination, victim, will be overwritten by da0

    1. install FreeBSD minimal on da1 - MAKE SURE YOU DON'T ERASE the other
       disk!

    2. boot from da1 (by setting the boot options or moving it to the
       first slot, whatever your machine is)

       WARNING: at this point, booting from da1 might mean that it
       becomes da0, BE CAREFUL!

    3. check that da0 and da1 are both visible -- camcontrol devlist

    4. # gmirror label -v mymirror0 /dev/daX   (your source disk --
       notice I don't say da0 and da1 anymore: YOU must figure out which
       is which)

    5. # mount /dev/daXs1a /mnt
       # vi /mnt/etc/fstab

       Replace /dev/daXs1a by /dev/mirror/mymirror0s1a where relevant
       # vi /mnt/boot/loader.conf
       add geom_mirror_load="YES"

       # umount /mnt

    6. Reconfigure the box so that the source disk is the boot disk

    7. boot single user, you should see near the end of the boot that
       it has found the volume (if geom_mirror.ko loaded successfully)

    8. try mount -a, see if everything is ok

    9. insert the target disk (the one you booted from before) into
       the raid:

       # gmirror insert mymirror0 da1

    10. That's it!  Check with gmirror status that it's rebuilding in the
        background.

    Note: the underlying disk is ALWAYS available -- just remove
    geom_mirror_load from the loader.conf, reboot, boot with -a (ask for
    root device) and you can without problem access /dev/da0s1a for example,
    but then the raid is broken...

    Hope this is clear.