Difference between revisions of "Building a Xen Virtual Guest Filesystem using Logical Volume Management (LVM)"

Latest revision as of 18:52, 29 May 2016

eBookFrenzy.com

Purchase and download the full PDF and ePub versions of this Xen eBook for only $8.99

Logical Volume Management (LVM) provides a flexible and high level approach to managing disk space. Instead of each disk drive being split into partitions of fixed sizes onto which fixed size file systems are in turn created, LVM provides a way to group together disk space into logical volumes which can be easily resized and moved. In addition, LVM allows administrators to carefully control disk space assigned to different groups of users by allocating distinct volume groups or logical volumes to those users. When the space initially allocated to the volume is exhausted the administrator can simply add more space without having to move the user files to a different file system.

When a Xen guest domain (domU) system is installed in a logical volume, that volume can be increased in size at any time simply by allocating additional disk space to the volume and extending the guest domain's filesystem to utilize the space.

The Key Components of Logical Volume Management

LVM consists of the following components:

Volume Group (VG)

The Volume Group is the high level container which holds one or more logical volumes and physical volumes.

Physical Volume (PV)

A physical volume represents a storage device such as a disk drive or other storage media.

Logical Volume (LV)

A logical volume is the equivalent to a disk partition and, as with a disk partition, can contain a file system.

Physical Extent (PE)

Each physical volume (PV) is divided into equal size blocks known as physical extents.

Logical Extent (LE)

Each logical volume (LV) is divided into equal size blocks called logical extents.

Let's suppose we are creating a new volume group called VolGroup001. This volume group needs physical disk space in order to function so we allocate three disk partitions /dev/sda1, /dev/sdb1 and /dev/sdb2. These become physical volumes in VolGroup001. We would then create a logical volume called LogVol001 within the volume group made up of the three physical volumes.

If we run out of space in LogVol001 we simply add more disk partitions as physical volumes and assign them to the volume group and logical volume.

Preparing for an LVM based Xen Guest Domain

In this chapter we will be working with a scenario in which we have a system containing four physical disk drives. The first disk drive, referenced as /dev/sda contains the host operating system. The second and third disks (/dev/sdc and /dev/sdd) will be converted into physical volumes and used to create a logical volume to hold the root filesystem for our Xen guest operating system. We will then use a partition on the fourth disk drive (/dev/sdb2) for the guest's swap space. It is important to note that LVM is highly flexible and this is just one of a potentially infinite number of different possible scenarios. For more details about Logical Volume Management read the Logical Volume Management Reference Guide.

Converting Physical Disks into Physical Volumes

Physical storage devices and partitions must first be converted into physical volumes before they can be added to volume groups and in turn assigned to logical volumes. As previously outlined, in this scenario we plan to convert two physical disk drives into physical volumes and then use them to construct a logical volume which will contain the root filesystem of a Xen guest domain.

Physical devices and partitions are converted into physical volumes using the pvcreate command-line tool. Using this tool we can, therefore, turn our two new disk drives (/dev/sdc and /dev/sdd) into physical volumes:

# pvcreate /dev/sdc
  Physical volume "/dev/sdc" successfully created

# pvcreate /dev/sdd
  Physical volume "/dev/sdd" successfully created

Information about physical volumes can be obtained by running the pvdisplay tool:

# pvdisplay /dev/sdc
  --- NEW Physical volume ---
  PV Name               /dev/sdc
  VG Name               
  PV Size               2.00 GB
  Allocatable           NO
  PE Size (KByte)       0
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               8PuJyh-gvgr-mSVh-2bTu-NScw-qrMq-VMFT1m

Now that we have two physical volumes the next step is to create a volume group to contain them.

Creating a Volume Group

A volume group is a container into which physical volumes are assigned and in which logical volumes are constructed. To add physical volumes to a volume group the vgcreate command is used. The following command creates a volume group called XenVolG and assigns the /dev/sdc physical volume to it:

# vgcreate XenVolG /dev/sdc
  Volume group "XenVolG" successfully created

Next we need to add the second physical volume to XenVolG, again using the vgextend command:

# vgextend XenVolG /dev/sdd
  Volume group "XenVolG" successfully extended

Use the vgdisplay command to list information about the new volume group:

# vgdisplay XenVolG
  --- Volume group ---
  VG Name               XenVolG
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               3.99 GB
  PE Size               4.00 MB
  Total PE              1022
  Alloc PE / Size       0 / 0   
  Free  PE / Size       1022 / 3.99 GB
  VG UUID               4py3dJ-kc8T-6bt5-RgvD-gwpj-Y3TX-l1TCp2

Now that we have a volume group containing physical volumes the next step is to create a logical volume comprising the two physical volumes in our volume group to use for the root filesystem of our Xen guest domain (domU).

Creating a Logical Volume for the Xen Guest System

Logical volumes are created using the lvcreate command which takes a number of command line arguments. Firstly, the -L flag is used to specify the size of the volume. Secondly, the -n flag is used to specify a name for the logical volume. Finally the name of the Volume Group to which the logical volume is to belong (and from which the space is to be acquired) is specified. Therefore, the following command will create a 3.99Gb logical volume called XenLogVol as part of the XenVolG volume group:

# lvcreate -L 3.99G -n XenLogVol XenVolG
  Rounding up size to full physical extent 3.99 GB
  Logical volume "XenLogVol" created

Information about the logical volume may be accessed using the lvdisplay command:

# lvdisplay
  --- Logical volume ---
  LV Name                /dev/XenVolG/XenLogVol
  VG Name                XenVolG
  LV UUID                i2ZQbV-TwVN-BxW6-h1m5-BDmf-9rax-0WiEoI
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                3.99 GB
  Current LE             1022
  Segments               2
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:2

As we can see from the above output we now have a 3.00GB logical volume which is accessed via the /dev/XenVolG/XenLogVol device.

Creating a Filesystem on the Logical Volume

The next step in the configuration process is to create a filesystem on the logical volume suitable as the root filesystem for our Xen guest domain. This is achieved using the mkfs tool referencing the device file of the XenLogVol logical volume as reported by lvdisplay:

# mkfs -t ext3 /dev/XenVolG/XenLogVol
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
523264 inodes, 1046528 blocks
52326 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1073741824
32 block groups
32768 blocks per group, 32768 fragments per group
16352 inodes per group
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376, 294912, 819200, 884736

Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 24 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

The root filesystem is now ready to be mounted and populated with system files.

Configuring the Swap Partition for the Xen Guest System

In addition to a root filesystem the second prerequisite for a Xen guest (domU) system is swap space. When we partitioned our physical disk drive previously we did so with the intention using the second partition (dev/sdb2) for swap space. As outlined previously, the objective is to use a partition on a different disk drive for swap space. In this scenario, the device file for the partition is /dev/sdb2. The mkswap command is used to configure this partition as swap space:

# mkswap /dev/sdb2
Setting up swapspace version 1, size = 1357164 kB

Mounting the Root Filesystem

Now that we have created an empty root filesystem on the XenLogVol logical volume, the next step is populate it with the system files and directories necessary to create a functioning guest operating system. Before this can be achieved, however, the partition needs to be mounted. For this we need to use the mount command in conjunction with a pre-existing directory to act as a mount point. The following commands create a directory named /xen and then mount the filesystem located on /dev/XenVolG/XenLogVol at this location:

# mkdir /xen
# mount /dev/XenVolG/XenLogVol /xen
# cd /xen
# ls
lost+found

As we can see, the guest root filesystem contains a single item, the lost+found directory created during the mkfs process. The next step is to clone the system files from the host operating system onto our guest root filesystem.

Cloning the Host OS on the Guest Root Partition

One of the easiest ways to install the necessary files on the guest OS root filesystem is to simply copy the files from the host operating system. Before doing so, however, it is important to keep in mind a few points.

Firstly, the following steps will copy a considerable amount of files from the host operating system to the guest root filesystem. This may well be much more than is actually necessary. For example, you may need a minimal Linux installation for your guest OS. If the host system has graphical desktop environments, office productivity applications and other packages not needed by the guest OS then cloning the host environment may not be the best approach. If, on the other hand, you are confident that everything on your host operating system will be of use in the guest system, or are unconcerned about disk space then this approach will work well for you.

The following steps assume that the root filesystem for the Xen guest has been mounted in /xen as outlined in the previous section.

Having created the mount point and mounted our root filesystem image the next step is to copy the relevant files from the host operating system to guest filesystem image. Execute the following command to copy the required directories and files to the root filesystem (be very careful to substitute the correct destination path if you mounted the filesystem somewhere other than /xen):

cp -ax /{bin,dev,etc,lib,root,sbin,usr,var} /xen

The amount of time the copy process takes to complete will depend to a large extent on the speed of physical disk drive and the number of additional applications and packages installed in the root partition of the host operating system.

Once the copy process is completed a number of directories need to be created (once again the following example assumes the image has been mounted at /xen):

mkdir /xen/{home,proc,opt,sys,tmp}

The tmp directory created above must have read/write access for all users. To ensure this the chmod command needs to be run:

chmod 777 /xen/tmp

Creating a Xen Configuration File

Now that we have physical disk partitions for the root filesystem and the swap space, and have populated the root filesystem with the necessary system files the next step is to create a Xen configuration file. In this file we need to specify the kernel and initial RAM Disk image files together with directives to map the disk partitions to devices within the guest OS.

For the purposes of this example we will load the initial RAM disk and kernel images from the host operating system. These are typically located in the /boot directory. The kernel file typically begins with the name vmlinuz and the initial RAM disk begins with initrd. These two images are defined in the Xen configuration file using the kernel = and ramdisk = directives. For example:

kernel = "/boot/vmlinuz-2.6.18-53.1.14.el5xen"
ramdisk = "/boot/initrd-2.6.18-53.1.14.el5xen.img"

Note that using the default host system initial RAM disk can cause problems on some Linux distributions. If the system fails to boot it may be necessary to generate an initial RAM disk without SCSI modules. Details of this and solutions to other Xen problems can be found at Linuxtopia.org

Next, the amount of memory to be allocated to the guest system is defined using the memory = directive. The following assigns 512Mb to the guest:

memory = 512

Amount to allocate will depend on both the available memory on the host machine and the amount of space required by the guest system.

Each guest domain needs to have a name by which it can be referenced when using the Xen management tools. This is assigned using name = :

name = "XenGuest1"

Now we specify network configuration options. The following lines indicate to Xen that the IP address of the system will be assigned using DHCP and the empty vif line instructs Xen to use the default values for the network interface configuration:

vif = [ '' ]
dhcp = "dhcp"

Finally the disk = directive is used to map the XenLogVol logical volume we have created with a guest operating system device. The following lines map the root logical volume (reported by lvdisplay as being located at /dev/XenVolG/XenLogVol) and swap partition to /dev/sdb1 and /dev/sdb2 respectively and define the root filesystem as /dev/sda1):

disk = ['phy:XenVolG/XenLogVol,sdb1,w', 'phy:sdb2,sdb2,w']
root = "/dev/sdb1 ro"

Note that these device mappings will be needed when the fstab file is configured later in this chapter.

Bringing all these configuration lines together should result in the following configuration file which we will name XenGuest1.cfg:

kernel = "/boot/vmlinuz-2.6.18-53.1.14.el5xen"
ramdisk = "/boot/initrd-2.6.18-53.1.14.el5xen.img"
memory = 512
name = "XenGuest1"
vif = [ '' ]
dhcp = "dhcp"
disk = ['phy:XenVolG/XenLogVol,sdb1,w', 'phy:sdb2,sdb2,w']
root = "/dev/sdb1 ro"

Configuring System Files for the Guest Operating System

It is important to keep in mind that at this point in the process we have copied all of the system files from our host operating system onto the disk image for our guest operating system. It is vital at this point, therefore, that we change any system configuration files to meet the requirements of our guest domain. Typical settings that will need to be changed are:

• /etc/fstab - This file contains the mappings between physical and network devices and filesystems. This will need to be changed to reflect the disk configuration used by the guest system.

• /etc/passwd - Contains password information for all user and application accounts on the host system. It is important to remove the entries for any accounts which will not be required on the guest system.

• /etc/group - This file contains information about user and application groups and, as with the passwd file, should be modified to remove any groups not required in the guest system.

• /etc/hosts - Contains information about the local host and other hosts on the network. Be sure to change the name of the local host to match the name to be used by the guest operating system.

• /etc/sysconfig/network or /etc/network/interfaces - Depending on which Linux distribution you are using, one of these files will contain networking information such as the host name and possibly a static IP address. Be sure to modify this file to remove any conflicts between the host and guest operating systems.

• /etc/exports - Contains information about any filesystems local to the host operating system which are exported for NFS access to remote systems. This should be modified to reflect any exports needed for the guest OS.

Modifying /etc/fstab for the Guest System

In the preceding section we talked about the system configuration files which may need to be modified in the root filesystem of the guest operating system. One file which needs particular attention is the /etc/fstab file. The file currently present in the guest root filesystem is a direct replica of the file used by the host operating system. It is almost certain that this file will not be configured to work with the disk represented by our root filesystem and swap disk partitions. The following shows a typical file from a CentOS, Red Hat Enterprise Linux or Fedora host operating system:

/dev/VolGroup00/LogVol00 /                       ext3    defaults        1 1
LABEL=/boot             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/VolGroup00/LogVol01 swap                    swap    defaults        0 0

Clearly, this doesn't match the disk configuration we have set up using disk partitions for the root filesystem and the swap device. The guest OS copy of the fstab file needs to be modified to reflect the device mappings in the Xen configuration file created above (recall that the root filesystem was mapped to /dev/sdb1 and swap to /dev/sdb2). For example:

/dev/sdb1               /                       ext3    defaults 1 1
/dev/sdb2               none                    swap    sw       0 0
none                    /dev/pts                devpts  gid=5,mode=620 0 0
none                    /dev/shm                tmpfs   defaults 0 0
none                    /proc                   proc    defaults 0 0
none                    /sys                    sysfs   defaults 0 0

Booting the Guest OS

Now all that remains is to boot the guest operating system. Before issuing the command to do so it is important to note that the system may not boot for a number of reasons.

If the system fails to boot be sure to read the Xen Guest Boot Problems section at the end of this chapter.

Before booting the guest system it is important to first unmount the guest root filesystem:

umount /xen

With the set up and configuration complete we are now ready to start up the guest OS. This is achieved using the xm create command:

xm create XenGuest1.cfg -c

The -c flag above instructs Xen to attach a console to the guest system so that we see output as the system boots. The following output displays the initial phases of a successful boot:

Using config file "./XenGuest1.cfg".
Started domain centos.5-1
Linux version 2.6.18-53.1.14.el5xen ([email protected]) (gcc version 4.1.2 20070626 (Red Hat 4.1.2-14)) #1 SMP Wed Mar 5 12:39:19 EST 2008
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 0000000020800000 (usable)
0MB HIGHMEM available.
520MB LOWMEM available.
NX (Execute Disable) protection: active
ACPI in unprivileged domain disabled
Built 1 zonelists.  Total pages: 133120
Kernel command line:  ip=:1.2.3.4::::eth0:dhcp root=/dev/sdb1 ro
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
CPU 0 irqstacks, hard=c0727000 soft=c0707000
PID hash table entries: 4096 (order: 12, 16384 bytes)
Xen reported: 2992.640 MHz processor.
Console: colour dummy device 80x25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Software IO TLB disabled

Xen Guest Boot Problems

A number of different problems can occur during the boot process of the Xen guest system.

Solutions to some common problems are listed below:

• Ubuntu Xen Guest (DomU) Hangs after EXT3-fs: mounted filesystem with ordered data mode message
• Ubuntu Xen System Boot Hangs After Setting System Clock Message
• Xen domainU fails to boot with a "Xen Guest OS Fails to Boot with Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)" error message
• A Xen Guest OS fails to boot with a "switchroot: mount failed: No such file or directory error message" error message.
• Xen CentOS/Fedora/Red Hat Guest OS Hangs During Boot
• Xen domainU Boot Fails with Invalid kernel / ERROR: Not a Xen-ELF image Message

eBookFrenzy.com

Purchase and download the full PDF and ePub versions of this Xen eBook for only $8.99