A quick note, there are actually 3 modes, not two when it comes to the drivers in use:
- HVM: unmodified kernel and drivers using software emulated devices
- PV-HVM: unmodified kernel with paravirtualized (Xen specific) disk and network drivers
- PV: modified kernel and drivers
For a Xen guest/DomU you can do a very basic
uname
andlsmod
with a grep to list the modules in use:uname -a lsmod | grep xen
If
uname -a
lists a kernel with the string "xen" in it, then you have a modified kernel and it's likely a PV guest, and you will see output from thelsmod
command to confirm it. If you have output from the grep onlsmod
but no sign of a modified kernel then you are PV-HVM. Without any sign of either, it's a straight HVM.Note: Generally you can do more with VMs that have the PV tools installed, so that can be quite an obvious pointer, however you can fake the presence of the PV tools to allow suspend/resume etc. so you cannot rely on that in general.
...
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
References (network)
- Network Configuration Examples (Xen 4.1+): http://wiki.xenproject.org/wiki/Network_Configuration_Examples_%28Xen_4.1%2B%29
- https://wiki.debian.org/BridgeNetworkConnections
- Old, Ethernet Bridge + netfilter Howto (2005 but has a boot script example): http://www.tldp.org/HOWTO/Ethernet-Bridge-netfilter-HOWTO.html
Toolstacks (a.k.a toolkits and toolboxes)
There are several toolstacks available for the personal or scripted administration of Xen resources.
...
- Network Configuration Examples (Xen WIKI): http://wiki.xen.org/wiki/
...
- Host_Configuration/Networking
- Bridging Network Connections (Debian WIKI): https://wiki.debian.org/BridgeNetworkConnections
Toolstacks (a.k.a toolkits and toolboxes)
There are several toolstacks available for the personal or scripted administration of Xen resources.
Primary reference: http://wiki.xen.org/wiki/Choice_of_Toolstacks
Apparently, once a toolstack has been chosen, it is not easy to change. From Debian 7 Wheezy's /etc/default/xen (where the chosen toolstack is configured):
...
- The shell runs /usr/sbin/xm which is a symlink to /usr/lib/xen-common/bin/xen-toolstack-wrapper
- /usr/lib/xen-common/bin/xen-toolstack-wrapper:
- Sources /usr/lib/xen-common/bin/xen-toolstack to set TOOLSTACK to /usr/lib/xen-4.1/bin/xl
- Sources /usr/lib/xen-common/bin/xen-dir to set dir to /usr/lib/xen-4.1
- Sources /usr/lib/xen-common/bin/xen-version to set VERSION to 4.1
- /usr/lib/xen-common/bin/xen-version gets Xen version info from /sys/hypervisor/version/*
- Sets PATH to /usr/lib/xen-4.1/bin
- Sources /etc/default/xen to set TOOLSTACK to xl (would fall back to hardcoded default otherwise)
- Sets TOOLSTACK to /usr/lib/xen-4.1/bin/xl
- Generates message "ERROR: A different toolstack (xl) have been selected!"
Creating DomUs
xen-create-image
xen-create-image is part of the xen-tools package. TODO: does it build the image from Dom0's OS? If so we might prefer not to use it because Dom0 was installed from the Hetzner Debian image and an uncustomised Debian may be preferable.
Using Debian Installer
Allows installation of an uncustomised Debian.
Reference: http://wiki.xenproject.org/wiki/Debian_Guest_Installation_Using_Debian_Installer
Troubleshooting
TODO: integrate the next para.
A quick note, there are actually 3 modes, not two when it comes to the drivers in use:
...
Maintaining Debian DomUs
This is listed before "Creating Debian DomUs" because a newly created Debian should be updated during installation.
Installing apt-cacher on Dom0 will save bandwidth and speed maintenance (and require significant time cleaning broken cache files).
Creating Debian DomUs
debootstrap
TODO: determine if it creates a new installation based on the existing installation (that would not be desirable if we want to have all our Debians as standard and the Dom0 is a Hetzner customised Debian).
TODO: determine whether debootstrap's single repo limitation would be significant for us.
References:
- Debian WIKI: https://wiki.debian.org/Debootstrap
Installing new Debian systems with debootstrap (2006): http://www.debian-administration.org/article/426/Installing_new_Debian_systems_with_debootstrap
Installing Debian GNU/Linux from a Unix/Linux System: http://www.debian.org/releases/stable/amd64/apds03.html.en
Debian installer
Is the standard Debian installation method; it supports installation of standard Debian as a Xen PV.
Note: If we use either of the Debian installer methods, pre-seeding would be advantageous both for creating installations that differ only as intended and – more importantly – to accelerate re-installation during disaster recovery. This might be very useful for all Debian and ubuntu installations, not just DomUs. Reference: http://www.debian.org/releases/stable/amd64/apb.html.en. Actually FAI (http://fai-project.org/ and BLUE-1325) is the way to go but it will take too long to familiarise with it.
References:
- http://wiki.xenproject.org/wiki/Debian_Guest_Installation_Using_Debian_Installer
- Debian GNU/Linux Installation Guide (Wheezy 64-bit version): http://www.debian.org/releases/stable/amd64/
Netboot
http://www.debian.org/releases/stable/amd64/ch04s05.html.en
.iso image
xen-create-image (from xen-tools)
xen-create-image is part of the xen-tools package. TODO: does it build the image from Dom0's OS? If so we might prefer not to use it because Dom0 was installed from the Hetzner Debian image and an uncustomised Debian may be preferable.
Troubleshooting
TODO: integrate the next para.
...
- Domain a Xen VM.
- dom0 The Xen domain/VM used a) to control the hypervisor b) as the primary interface to the hardware.
- domu* Unprivileged Xen domains/VMs
- Enlightened guest An OS designed/modified for use in a domain/VM.
- Full virtualisation A domain/VM providing full emulation of physical hardware. The OS can be the same as runs on real hardware.
- Guest domain Same as domu*.
- Hardware Virtual Machine (HVM) Same as full virtualisation.
- Hypervisor a software system that allows the execution of multiple virtual guest operating systems simultaneously on a single physical machine. Xen's hypervisor is Type 1 or “bare-metal”, meaning it runs on the physical machine as opposed to within an operating system.
- HVM Hardware Virtual Machine.
- Para-virtualizationvirtualisation uses modified guest operating systems a.k.a. enlightened guests. They don’t require virtual hardware devices, instead they make special calls to the hypervisor that allow them to access CPUs, storage and network resources. Better performance that full virtualisation.
- PV para-virtualisation.
- Virtual Machine Monitor (VMM) Same as hypervisor.
- Xen Store A database of information shared by the hypervisor, the kernels, the drivers and the xen daemon. The memory for Xen Store is provided by Dom0. In Dom0 (and in enlightened guests?) it can be accessed via /proc/xen/xenbus/
...
- /etc/default/xend
- /etc/default/xendomains
- /etc/init.d/xen
- /etc/init.d/xendomains
- /etc/xen/scripts/*
- /etc/xen/xend-config.sxp
- /etc/xen/xend-pci-permissive.sxp
- /etc/xen/xend-pci-quirks.sxp
- /etc/xen/xl.conf
- /lib/udev/rules.d/xen-backend.rules
- /lib/udev/rules.d/xend.rules
- /usr/lib/xen-common/bin/*
- /usr/sbin/cpuperf-perfcntr
- /usr/sbin/cpuperf-xen
- /usr/sbin/xe
- /usr/sbin/xenperf
- /usr/sbin/xenpm
- /usr/sbin/xentop
- /usr/sbin/xentrace
- /usr/sbin/xentrace_format
- /usr/sbin/xentrace_setmask
- /usr/sbin/xentrace_setsize
- /usr/sbin/xl
- /usr/sbin/xentopxm
- /usr/sbin/xentrace
- /usr/sbin/xentrace_format
- /usr/sbin/xentrace_setmask
- /usr/sbin/xentrace_setsize
- /usr/sbin/xl
- /usr/sbin/xm
- /usr/share/xen-utils-common/default.xen
xenstore-utils
Provides (all in /usr/sbin):
- xenstore-ls
- xenstore-write
- xenstore-exists
- xenstore-rm
- xenstore-chmod
- xenstore-watch
- xenstore-list
- xenstore-read
References
Documentation
- Wikipedia: http://en.wikipedia.org/wiki/Xen
- Debian Wiki Xen page (in draft): https://wiki.debian.org/Xen
- Xen Project Beginners Guideshare/xen-utils-common/default.xen
xenstore-utils
Provides (all in /usr/sbin):
- xenstore-ls
- xenstore-write
- xenstore-exists
- xenstore-rm
- xenstore-chmod
- xenstore-watch
- xenstore-list
- xenstore-read
References
General references are listed here. Specific references are listed in the sections they apply to above.
Introductory
- Wikipedia: http://en.wikipedia.org/wiki/Xen
- Debian Wiki Xen page (in draft): https://wiki.debian.org/Xen
- Xen Project Beginners Guide: http://wiki.xenproject.org/wiki/Xen_Beginners_Guide#Setup_Linux_Bridge_for_guest_networking
- XPSU13 VIDEO: Xen for Beginners (49 min): http://www.xenproject.org/help/presentations-and-videos/video/latest/xpus13-beginners.html
Entry points
These are entry points to references.
- Xen WIKI: http://wiki.xenproject.org/wiki/Main_Page
- Xen man pages: http://wiki.xenproject.org/wiki/Xen_Beginners_Guide#Setup_Linux_Bridge_for_guest_networkingMan_Pages
Setup logs
- Xen On Debian Wheezy With LVM (HowtoForge. Has bridge example with fixed IP): http://www.howtoforge.com/xen-on-debian-wheezy-with-lvm
- Using Xen With LVM-Based VMs Instead Of Image-Based VMs (HowtoForge. Etch): http://www.howtoforge.com/using-xen-with-lvm-based-vms-instead-of-image-based-vms-debian-etch
- Debian Wheezy -> Jessie and set up Xen (HowtoForge): http://www.howtoforge.com/how-to-set-up-xen-4.3-on-debian-wheezy-7.0.2-and-then-upgrade-to-jessie
- Setting up XEN on a Hetzner Dedicated Server (Squeeze, xen-create-image, xm): https://gist.github.com/meskyanichi/3354956
- Routed XEN VM based on LVM (step-by-step for Hetzner DS3000. Lenny): http://wiki.summercode.com/routed_xen_vm_based_lvm_step_by_step_guide_for_hetzner_ds3000
- XPSU13 VIDEO: Xen for Beginners (49 min): http://www.xenproject.org/help/presentations-and-videos/video/latest/xpus13-beginners.htmlXen on 4 app servers (xl, no public IPs): http://wiki.xenproject.org/wiki/Xen_on_4_app_serversDebian Guest Installation Using Debian Installer: , no public IPs): http://wiki.xenxenproject.org/wiki/DebianXen_Gueston_Installation4_Usingapp_Debian_Installerservers
- The Debian Administrator's Handbook, Virtualisation chapter (Nice. xen-tools, xm, bridged): http://debian-handbook.info/browse/stable/sect.virtualization.html
- How to install Xen in Debian (Armen's Wiki. Has scripts for both public and private IPs for virtual machines): http://vardump.org/wiki.php/HowToInstallXenInDebianGNULinux.
- Network Configuration Examples (Xen WIKI): http://wiki.xen.org/wiki/Host_Configuration/Networking
- Bridging Network Connections (Debian WIKI): https://wiki.debian.org/BridgeNetworkConnections
...