For a machine that has fully-mirrored file systems using
	vinum, it is desirable to also
	mirror the root file system.  Setting up such a configuration
	is less trivial than mirroring an arbitrary file system
	because:
The root file system must be available very early
	    during the boot process, so the
	    vinum infrastructure must
	    already be available at this time.
The volume containing the root file system also
	    contains the system bootstrap and the kernel.  These must
	    be read using the host system's native utilities, such as
	    the BIOS, which often cannot be taught about the details
	    of vinum.
In the following sections, the term “root
	  volume” is generally used to describe the
	vinum volume that contains the root
	file system.
vinum must be available early
	  in the system boot as loader(8) must be able to load
	  the vinum kernel module before starting the kernel.  This
	  can be accomplished by putting this line in
	  /boot/loader.conf:
geom_vinum_load="YES"
The current FreeBSD bootstrap is only 7.5 KB of code and
	does not understand the internal
	vinum structures.  This means that it
	cannot parse the vinum configuration
	data or figure out the elements of a boot volume.  Thus, some
	workarounds are necessary to provide the bootstrap code with
	the illusion of a standard a partition
	that contains the root file system.
For this to be possible, the following requirements must be met for the root volume:
The root volume must not be a stripe or RAID-5.
The root volume must not contain more than one concatenated subdisk per plex.
Note that it is desirable and possible to use multiple
	plexes, each containing one replica of the root file system.
	The bootstrap process will only use one replica for finding
	the bootstrap and all boot files, until the kernel mounts the
	root file system.  Each single subdisk within these plexes
	needs its own a partition illusion, for
	the respective device to be bootable.  It is not strictly
	needed that each of these faked a
	partitions is located at the same offset within its device,
	compared with other devices containing plexes of the root
	volume.  However, it is probably a good idea to create the
	vinum volumes that way so the
	resulting mirrored devices are symmetric, to avoid
	confusion.
In order to set up these a
	partitions for each device containing part of the root
	volume, the following is required:
The location, offset from the beginning of the device, and size of this device's subdisk that is part of the root volume needs to be examined, using the command:
#gvinum l -rv root
vinum offsets and sizes are
	    measured in bytes.  They must be divided by 512 in order
	    to obtain the block numbers that are to be used by
	    bsdlabel.
Run this command for each device that participates in the root volume:
#bsdlabel -edevname
devname must be either the
	    name of the disk, like da0 for
	    disks without a slice table, or the name of the
	    slice, like ad0s1.
If there is already an a
	    partition on the device from a
	    pre-vinum root file system, it
	    should be renamed to something else so that it remains
	    accessible (just in case), but will no longer be used by
	    default to bootstrap the system.  A currently mounted root
	    file system cannot be renamed, so this must be executed
	    either when being booted from a “Fixit”
	    media, or in a two-step process where, in a mirror, the
	    disk that is not been currently booted is manipulated
	    first.
The offset of the vinum
	    partition on this device (if any) must be added to the
	    offset of the respective root volume subdisk on this
	    device.  The resulting value will become the
	    offset value for the new
	    a partition.  The
	    size value for this partition can be
	    taken verbatim from the calculation above.  The
	    fstype should be
	    4.2BSD.  The
	    fsize, bsize,
	    and cpg values should be chosen
	    to match the actual file system, though they are fairly
	    unimportant within this context.
That way, a new a partition will
	    be established that overlaps the
	    vinum partition on this device.
	    bsdlabel will only allow for this
	    overlap if the vinum partition
	    has properly been marked using the
	    vinum fstype.
A faked a partition now exists
	    on each device that has one replica of the root volume.
	    It is highly recommendable to verify the result using a
	    command like:
#fsck -n /dev/devnamea
It should be remembered that all files containing control
	information must be relative to the root file system in the
	vinum volume which, when setting up
	a new vinum root volume, might not
	match the root file system that is currently active.  So in
	particular, /etc/fstab and
	/boot/loader.conf need to be taken care
	of.
At next reboot, the bootstrap should figure out the
	appropriate control information from the new
	vinum-based root file system, and act
	accordingly.  At the end of the kernel initialization process,
	after all devices have been announced, the prominent notice
	that shows the success of this setup is a message like:
Mounting root from ufs:/dev/gvinum/root
After the vinum root volume has
	been set up, the output of gvinum l -rv
	  root could look like:
... Subdisk root.p0.s0: Size: 125829120 bytes (120 MB) State: up Plex root.p0 at offset 0 (0 B) Drive disk0 (/dev/da0h) at offset 135680 (132 kB) Subdisk root.p1.s0: Size: 125829120 bytes (120 MB) State: up Plex root.p1 at offset 0 (0 B) Drive disk1 (/dev/da1h) at offset 135680 (132 kB)
The values to note are 135680 for the
	offset, relative to partition
	/dev/da0h.  This
	translates to 265 512-byte disk blocks in
	bsdlabel's terms.  Likewise, the size of
	this root volume is 245760 512-byte blocks.  /dev/da1h, containing the
	second replica of this root volume, has a symmetric
	setup.
The bsdlabel for these devices might look like:
... 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 245760 281 4.2BSD 2048 16384 0 # (Cyl. 0*- 15*) c: 71771688 0 unused 0 0 # (Cyl. 0 - 4467*) h: 71771672 16 vinum # (Cyl. 0*- 4467*)
It can be observed that the size
	parameter for the faked a partition
	matches the value outlined above, while the
	offset parameter is the sum of the offset
	within the vinum partition
	h, and the offset of this partition
	within the device or slice.  This is a typical setup that is
	necessary to avoid the problem described in Section 8.4.3, “Nothing Boots, the Bootstrap
	  Panics”.  The entire
	a partition is completely within the
	h partition containing all the
	vinum data for this device.
In the above example, the entire device is dedicated to
	vinum and there is no leftover
	pre-vinum root partition.
The following list contains a few known pitfalls and solutions.
If for any reason the system does not continue to boot,
	  the bootstrap can be interrupted by pressing
	  space at the 10-seconds warning.  The
	  loader variable vinum.autostart can be
	  examined by typing show and manipulated
	  using set or
	  unset.
If the vinum kernel module was
	  not yet in the list of modules to load automatically, type
	  load geom_vinum.
When ready, the boot process can be continued by typing
	  boot -as which
	  -as requests the kernel to ask for the
	  root file system to mount (-a) and make the
	  boot process stop in single-user mode (-s),
	  where the root file system is mounted read-only.  That way,
	  even if only one plex of a multi-plex volume has been
	  mounted, no data inconsistency between plexes is being
	  risked.
At the prompt asking for a root file system to mount,
	  any device that contains a valid root file system can be
	  entered.  If /etc/fstab is set up
	  correctly, the default should be something like
	  ufs:/dev/gvinum/root.  A typical
	  alternate choice would be something like
	  ufs:da0d which could be a
	  hypothetical partition containing the
	  pre-vinum root file system.  Care
	  should be taken if one of the alias
	  a partitions is entered here, that it
	  actually references the subdisks of the
	  vinum root device, because in a
	  mirrored setup, this would only mount one piece of a
	  mirrored root device.  If this file system is to be mounted
	  read-write later on, it is necessary to remove the other
	  plex(es) of the vinum root volume
	  since these plexes would otherwise carry inconsistent
	  data.
If /boot/loader fails to load, but
	  the primary bootstrap still loads (visible by a single dash
	  in the left column of the screen right after the boot
	  process starts), an attempt can be made to interrupt the
	  primary bootstrap by pressing
	  space.  This will make the bootstrap stop
	  in stage two.  An attempt
	  can be made here to boot off an alternate partition, like
	  the partition containing the previous root file system that
	  has been moved away from a.
This situation will happen if the bootstrap had been
	  destroyed by the vinum
	  installation.  Unfortunately, vinum
	  accidentally leaves only 4 KB at the beginning of its
	  partition free before starting to write its
	  vinum header information.  However,
	  the stage one and two bootstraps plus the bsdlabel require 8
	  KB.  So if a vinum partition was
	  started at offset 0 within a slice or disk that was meant to
	  be bootable, the vinum setup will
	  trash the bootstrap.
Similarly, if the above situation has been recovered,
	  by booting from a “Fixit” media, and the
	  bootstrap has been re-installed using
	  bsdlabel -B as described in ../../../../doc/en_US.ISO8859-1/books/handbook/boot.html#boot-boot1, the bootstrap will trash the
	  vinum header, and
	  vinum will no longer find its
	  disk(s).  Though no actual vinum
	  configuration data or data in vinum
	  volumes will be trashed, and it would be possible to recover
	  all the data by entering exactly the same
	  vinum configuration data again, the
	  situation is hard to fix.  It is necessary to move the
	  entire vinum partition by at least
	  4 KB, in order to have the vinum
	  header and the system bootstrap no longer collide.
All FreeBSD documents are available for download at https://download.freebsd.org/ftp/doc/
Questions that are not answered by the
    documentation may be
    sent to <freebsd-questions@FreeBSD.org>.
    Send questions about this document to <freebsd-doc@FreeBSD.org>.