In this article today, we are going to show you how to install XEN dom0 on Almalinux from source. This guide will also work for CentOS 8. XEN is a widely used hypervisor and has been around for many years. It has evolved to be a popular choice for server virtualisation and lowering the total cost of ownership.
XEN is a great choice if you are looking to virtualise your infrastructure, or you are looking to simply setup a home lab.
Whilst you can install XEN from Almalinux packages, often the case these packages are behind in versions. To get the latest releases and keep up to date we will focus on compiling XEN from source.
The only impossible journey is the one you never begin.
- A server setup with Almalinux (fresh install recommended)
- Access to the internet
- You will need root access to the server
- The server should be a physical host, you can order one from Sebae
- SSH enabled on the server
Keeping your server up to date with releases and patches is highly recommended. As technology changes, so does the requirements to keep those changes secure. Running a virtualised environment puts the main hypervisor in the spotlight for attacks. We strongly advise to keep the hypervisor from being accessible from the internet and limit any access to virtual machines by firewalls.
You can follow our tutorial on how to configure automatic updates on CentOS and Almalinux.
Before You Begin
Login to your server as root and get the OS up to date.
let the process run, once finished Debian should be ready.
For our partition layout, we are using the following
/boot Ext4 - 500MB / Ext4 - 200GB /swap - 2GB LVM Volume for guests - 500GB
IMPORTANT – you do not need to follow our partition sizes, however when you install Debian and you choose to place everything inside an LVM volume, make sure you statically (*) set the dev mapper support instead of module (M) when compiling the new kernel, otherwise your system will not boot!
The following is an example of compiling LVM (dev mapper) support statically (*)
Step 1: Install Required Packages
Firstly, we need to enable the power tools repo by running the following command:
dnf config-manager --set-enabled powertools
In order to successfully compile the kernel and XEN, we will need to install the build packages. From the shell, run the following command:
dnf install wget glibc-devel python3 python3-devel gcc gettext patch glibc-devel glib2-devel acpica-tools flex bison uuid libuuid-devel ncurses-devel pixman-devel pixman yajl yajl-devel ninja-build openssl openssl-devel bc dwarves
During the install process, dnf will prompt to continue, if you are happy, please enter ‘Y‘.
We also need to run the following command:
dnf group install "Development Tools"
You wiill get another prompt to continue, if you are happy please enter ‘Y‘.
Step 2: Download And Compile XEN
Now we have all the packages installed, we can download the latest build of XEN and compile it.
Let change into the source build directory by running the following command:
Now download XEN by running the following command:
Now extract the tar file:
tar xvf xen-4.16.2.tar.gz
Change into the new XEN directory:
Now let’s configure the source files by running the following command:
./configure -enable-githttp -libdir=/usr/lib
Once complete, we can now build the source, this may take some time.
make -j 5 xen && make -j 5 tools && make -j 5 stubdom
Let’s install the newly compiled binaries, firstly change to the install directory with the following command:
Now we can copy the binaries into the appropriate directories.
Note ignore the error ‘cannot overwrite non-directory ‘/var/run’ with directory ‘var/run’
cp -R * /
Edit the following file with your favorite editor, we will be using vi
Find and comment out the following line then save the changes.
Xen is now compiled and installed.
Step 3: System Tweaks
The following tweaks should make XEN start when the system boots up. These are optional but recommended. These tweaks will also add more scheduling weight to dom0 and disable power save on the console (useful if you get system crashes).
Note, the following entries that contain rc.5 could be rc.3 depending on your runlevel. We are running runlevel 5 so the following will reflect that. To find out what runlevel you are in, simply enter runlevel at the prompt.
root@almatraining:/usr/src/xen-4.16.2/dist/install# runlevel N 3
Now enter the following commands:
ln -s /etc/init.d/xendomains /etc/rc0.d/S10xendomains ln -s /etc/init.d/xendomains /etc/rc6.d/S10xendomains ln -s /etc/init.d/xencommons /etc/rc3.d/S98xencommons ln -s /etc/init.d/xendomains /etc/rc3.d/S98xendomains ln -s /etc/xendom0caps /etc/rc3.d/S98xendom0caps echo "xl sched-credit -d Domain-0 -w 512" > /etc/xendom0caps chmod +x /etc/xendom0caps echo "/usr/bin/setterm -powersave off -blank 0" > /etc/rc.local
Finally lets also disable ballooning, this can save RAM but with a performance impact. If you have a fast CPU then you may choose to ignore this:
now change the following line from auto to 0 and uncomment:
Step 4: Configure Network Interfaces
Newer releases of XEN can now use the system bridge rather than in previously releases where XEN provided their own script. In order to do this, we need to use nmcli to edit the network configuration.
Caution is advised in the next step as this could lead to no network connectivity if the steps are not correctly followed.
Firstly, let’s find out what our interface is called, we can simply issue the following command to list the current configuration:
nmcli conn show --active
The output should look similar to the following
[root@almatraining ~]# nmcli conn show --active NAME UUID TYPE DEVICE enp9s0 2b502aca-07a4-413b-8ae5-02f29efd836f ethernet enp9s0
We can see here that enp9s0 is the main interface name on this server.
Now let’s create our new network bridge by running the following command:
nmcli conn add type bridge con-name xenbr0 ifname xenbr0
Now add your IP to the new bridge, make sure you use your existing IP address. We are using 192.168.0.199/24 so we will add that to the config by running the following command:
nmcli conn modify xenbr0 ipv4.addresses 192.168.0.199/24
Next add the gateway address to the bridge:
nmcli conn modify xenbr0 ipv4.gateway 192.168.0.1
Now set the IP method to manual on the bridge:
nmcli conn modify xenbr0 ipv4.method manual
We now need to add the ethernet interface to the bridge by running the following command:
nmcli conn add type ethernet slave-type bridge con-name bridge-xenbr0 ifname enp9s0 master xenbr0
Time to activate the new bridge, run the following command:
nmcli conn up xenbr0
Now we need to deactivate the existing Ethernet or wired connection by running the following command:
nmcli conn down enp9s0
The bridge setup is now complete, we can verify the active network connections by running the following:
nmcli conn show --active
The output should look like the following:
NAME UUID TYPE DEVICE xenbr0 2b502aca-07a4-413b-8ae5-02f29efd836f bridge xenbr0 bridge-xenbr0 1b06f0f2-c7d0-49bd-9251-141adb62814a ethernet enp9s0
Note check your DNS resolver config is correct as we found ours was sometimes empty after creating a new bridge.
At this point if you need a drink, now is the time 🙂
Step 5: Compile The XEN Kernel
Now we have completed the XEN steps, we need to install a XEN enabled kernel. The XEN aware kernel contains the necessary drivers and options we need to start the hypervisor. This will also allow us to run more up to date kernels than the standard images shipped with Almalinux.
Let’s change back to the source directory;
Now let’s get the latest kernel release
Extract the tar file:
tar xvf linux-6.0.6.tar.gz
Now let’s change into the kernel directory:
Now we will choose what to compile in the Linux kernel menu. Enter the following command:
Once loaded, you should be presented with a text-based menu.
Go into the Processor type and features menu:
Then choose Linux guest support – all the options related to XEN should have an asterisk * next to them already, if not change it so they all have an asterisk. You can enable the debug options if you wish to.
Now go back to the main menu by selecting exit, then exit again.
Now scroll down to Device Drivers and press enter
Next go down to XEN driver support and press enter
Most options should be either set to static (*) or module (M) if not you can change them all to static. Make sure the Backend driver support is enabled.
Now press Exit to get back to the Device Drivers menu.
Now go into the block devices menu
Make sure the 2 XEN options are set like below
Now go back to the Devices Menu by pressing Exit.
Go down to the Network device support menu and press enter
ensure the 2 XEN options are set statically (*) or set as a module (M)
Thats it, we can now exit the kernel menu by selecting Exit 3 times.
Make sure to press yes to saving the changes
Note run the following command after you have saved the config to avoid a certificate error when compiling
sed -ri '/CONFIG_SYSTEM_TRUSTED_KEYS/s/=.+/=""/g' .config
We can verify our saved options by running the following command:
cat /usr/src/linux-6.0.6/.config | grep XEN
The output should look like the following;
CONFIG_XEN=y CONFIG_XEN_PV=y CONFIG_XEN_512GB=y CONFIG_XEN_PV_SMP=y CONFIG_XEN_PV_DOM0=y CONFIG_XEN_PVHVM=y CONFIG_XEN_PVHVM_SMP=y CONFIG_XEN_PVHVM_GUEST=y CONFIG_XEN_SAVE_RESTORE=y # CONFIG_XEN_DEBUG_FS is not set # CONFIG_XEN_PVH is not set CONFIG_XEN_DOM0=y CONFIG_PCI_XEN=y # CONFIG_KVM_XEN is not set # CONFIG_XEN_PCIDEV_FRONTEND is not set CONFIG_XEN_BLKDEV_FRONTEND=y CONFIG_XEN_BLKDEV_BACKEND=y # CONFIG_XEN_SCSI_FRONTEND is not set CONFIG_NETXEN_NIC=m CONFIG_XEN_NETDEV_FRONTEND=y CONFIG_XEN_NETDEV_BACKEND=y CONFIG_INPUT_XEN_KBDDEV_FRONTEND=m CONFIG_HVC_XEN=y CONFIG_HVC_XEN_FRONTEND=y # CONFIG_TCG_XEN is not set CONFIG_XEN_WDT=m # CONFIG_DRM_XEN_FRONTEND is not set # CONFIG_XEN_FBDEV_FRONTEND is not set CONFIG_SND_XEN_FRONTEND=m # CONFIG_USB_XEN_HCD is not set # CONFIG_MMC_SDHCI_XENON is not set CONFIG_XEN_BALLOON=y CONFIG_XEN_BALLOON_MEMORY_HOTPLUG=y CONFIG_XEN_MEMORY_HOTPLUG_LIMIT=512 CONFIG_XEN_SCRUB_PAGES_DEFAULT=y CONFIG_XEN_DEV_EVTCHN=y CONFIG_XEN_BACKEND=y CONFIG_XENFS=y CONFIG_XEN_COMPAT_XENFS=y CONFIG_XEN_SYS_HYPERVISOR=y CONFIG_XEN_XENBUS_FRONTEND=y CONFIG_XEN_GNTDEV=m CONFIG_XEN_GRANT_DEV_ALLOC=m # CONFIG_XEN_GRANT_DMA_ALLOC is not set CONFIG_SWIOTLB_XEN=y CONFIG_XEN_PCI_STUB=y CONFIG_XEN_PCIDEV_BACKEND=m CONFIG_XEN_PVCALLS_FRONTEND=y CONFIG_XEN_PVCALLS_BACKEND=y # CONFIG_XEN_SCSI_BACKEND is not set CONFIG_XEN_PRIVCMD=y CONFIG_XEN_ACPI_PROCESSOR=m # CONFIG_XEN_MCE_LOG is not set CONFIG_XEN_HAVE_PVMMU=y CONFIG_XEN_EFI=y CONFIG_XEN_AUTO_XLATE=y CONFIG_XEN_ACPI=y CONFIG_XEN_SYMS=y CONFIG_XEN_HAVE_VPMU=y CONFIG_XEN_FRONT_PGDIR_SHBUF=m CONFIG_XEN_UNPOPULATED_ALLOC=y
Now we can start compiling the kernel with the XEN options included.
Run the following command to start compiling:
make -j 5 bzImage && make -j 5 modules && make -j 5 INSTALL_MOD_STRIP=1 modules_install
Once complete, we can then copy the kernel files into the correct directories by running the following commands;
cp -a arch/x86/boot/bzImage /boot/vmlinuz-6.0.6 cp -a System.map /boot/System.map-6.0.6 cp -a .config /boot/config-6.0.6 depmod -a mkinitrd -f -v /boot/initrd.img-6.0.6 6.0.6
Step 6: Configure Grub to boot XEN
We have reached the last part of the build. We now need to tell grub to boot in XEN mode and here we can also specify the RAM allocation to dom0.
change into the grub directory:
We now need to move the normal Linux kernel to a lower priority so Grub boots XEN, run the following command:
mv 10_linux 50_linux
Next edit the grub file
Now we need to add the following lines under the GRUB_CMDLINE_Linux section.
You can also edit the RAM allocation, here we have it set to 1GB.
The lines should look similar to this
GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="crashkernel=auto resume=/dev/mapper/almalinux-swap rd.lvm.lv=almalinux/root rd.lvm.lv=almalinux/swap" GRUB_CMDLINE_XEN_DEFAULT=dom0_mem=1024M,max:1024M GRUB_CMDLINE_XEN= GRUB_DISABLE_RECOVERY="true" GRUB_ENABLE_BLSCFG=false
save and exit.
Next issue the following command:
grub2-mkconfig -o /boot/grub2/grub.cfg
We have found that the environment variable LD_LIBRARY_PATH needs to also be set in order for XEN to locate the required libraries. In order to do this, edit the following file:
Now add the following line to the file
It will look similar to this:
include ld.so.conf.d/*.conf /usr/lib
Now save the file and exit.
Issue the following command which will set the environment variable:
Now reboot and you should be able to issue the following command once you have logged back in;
You should get the following output:
Name ID Mem VCPUs State Time(s) Domain-0 0 1024 2 r----- 16.7 root@almatraining:~#
You can now create your guest VM disks and config files, please refer to the XEN Project Wiki which contains useful information on settings these up.
Now you have learned how to install XEN dom0 on Almalinux from source.
We hope you found this useful, for any further help or interest in our products, you can contact us via our website.