Running Multiple Linux Servers as KVM Virtual MachinesKVM is virtual machine software for Linux that allows you to run multiple guest operating systems on top of the real host system. Unlike, the basic versions of VMWare and VirtualBox, KVM is designed for running operating systems that do not require a GUI, e.g. servers. In the past at work we had two separate physical internal servers: one for students and one for staff. So when we did a hardware upgrade I setup KVM so that both servers could run as separate virtual guests on the single physical host. Most of the instructions I followed came from the Ubuntu KVM Guide, as well as the man pages for virt-inst.
Installing Ubuntu on the Host MachineIn this case I installed Ubuntu Server 10.04 LTS (Lucid) on a PC with: Intel i7-2600 3.4GHz; Asus P8H67-M motherboard; 4GB RAM; 2 x Western Digital Green 1.5TB hard drives. I followed a standard install using the following options:
- Hostname: ictweb
- Partitioning: /boot plus LVM including /, /tmp, /var and /home, as well as two logical volumes for the guest virtual machines.
- Selected OpenSSH server and Virtual Machine Host as install package options.
Preparing Host Machine for Virtual Machine GuestsBy selecting the Virtual Machine Host during installation, most of the necessary packages (e.g. libvirt) were automatically installed. But some additional packages need to be manually installed:
$ sudo apt-get install qemu libcap2-bin virtinstThe X window system is not installed on the host, and so to initially access an installed VM guest I used VNC/SSH from my laptop. The access requires root password on the host, and so that must be set:
$ sudo su root@ictweb:/home/sgordon# passwdOptionally, change the SSH server port to 11111 (or any other unused other port number) by editing
Networking for Host and GuestsThe intended network topology for the host and VM guests is shown below: By selecting the Virtual Machine Host during installation, bridging/routing is correctly setup so that once the VM guests are installed, they can be accessed on their own LAN. By default the host obtained an IP for the Ethernet interface (eth0) from the network DHCP server, and a virtual bridge was created (virbr0) with IP 192.168.122.1. I edited
auto lo iface lo inet loopback auto eth0 iface eth0 inet static address 192.168.1.2 network 192.168.1.0 netmask 255.255.255.0 broadcast 192.168.1.255 gateway 192.168.1.1 auto eth0:0 iface eth0:0 inet static address 192.168.1.10 network 192.168.1.0 netmask 255.255.255.0 broadcast 192.168.1.255 gateway 192.168.1.1 auto eth0:1 iface eth0:1 inet static address 192.168.1.11 network 192.168.1.0 netmask 255.255.255.0 broadcast 192.168.1.255 gateway 192.168.1.1eth0:0 is a virtual interface and will be used to make one of the VM guests accessible via 192.168.1.10 (similar for eth0:1). Restart networking for the changes to have effect:
$ sudo /etc/init.d/networking stop $ sudo /etc/init.d/networking startHere is the output of ifconfig on the host machine (with many unnecessary details removed):
sgordon@ictweb:~$ ifconfig eth0 Link encap:Ethernet HWaddr f4:6d:... inet addr:192.168.1.2 Bcast:192.168.1.255 Mask:255.255.255.0 ... eth0:0 Link encap:Ethernet HWaddr f4:6d:... inet addr:192.168.1.10 Bcast:192.168.1.255 Mask:255.255.255.0 ... eth0:1 Link encap:Ethernet HWaddr f4:6d: inet addr:192.168.1.11 Bcast:192.168.1.255 Mask:255.255.255.0 ... lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 ... virbr0 Link encap:Ethernet HWaddr fe:54:... inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0 ... vnet0 Link encap:Ethernet HWaddr fe:54:... ... vnet1 Link encap:Ethernet HWaddr fe:54:... ...
Configuring the Firewall/NAT on HostThe firewall/NAT must be configured so that traffic coming into the host can be forwarded to the appropriate VM guest. In particular, data to/from the IP aliases addresses (e.g. 192.168.1.10) must be forwarded to the real VM guest address (192.168.122.10). The NAT table of iptables is used for this. In addition, rules are needed to allow the router to forward packets to the SSH, Web and Email servers on the guests. The rules I applied were:
$ sudo iptables -t nat -I PREROUTING -d 192.168.1.10 -j DNAT --to-destination 192.168.122.10 $ sudo iptables -t nat -I POSTROUTING -s 192.168.122.10 -j SNAT --to-source 192.168.1.10 $ sudo iptables -t nat -I PREROUTING -d 192.168.1.11 -j DNAT --to-destination 192.168.122.11 $ sudo iptables -t nat -I POSTROUTING -s 192.168.122.11 -j SNAT --to-source 192.168.1.11 $ sudo iptables -I FORWARD -p tcp -d 192.168.122.10 --dport 22 -j ACCEPT $ sudo iptables -I FORWARD -p tcp -d 192.168.122.10 --dport 25 -j ACCEPT $ sudo iptables -I FORWARD -p tcp -d 192.168.122.10 --dport 80 -j ACCEPT $ sudo iptables -I FORWARD -p tcp -d 192.168.122.11 --dport 22 -j ACCEPT $ sudo iptables -I FORWARD -p tcp -d 192.168.122.11 --dport 25 -j ACCEPT $ sudo iptables -I FORWARD -p tcp -d 192.168.122.11 --dport 80 -j ACCEPTTo automatically create the firewall table upon reboot I saved the current set of rules:
$ sudo iptables-save -c > ~/iptables.rules $ sudo mv ~/iptables.rules /etc/ $ sudo chown root.root /etc/iptables.rules $ sudo chmod go-rwx /etc/iptables.rulesAnd then added the following line to
iptables-restore < /etc/iptables.rulesOther rules can be added to this file as needed for your firewall. (Update: I needed to add a sleep 60 command before the restoration of the rules in the
Installing the Virtual Machine GuestsI had the ISO image for Ubuntu on a USB disk. The first task is to mount the USB disk (mine uses ext4 file system; the -t option below may need to be changed depending on the filesystem of your USB. Also, the device needs to be correct):
$ cd ~ $ mkdir usb $ sudo mount -t ext4 /dev/sdg1 ~/usbSet the capabilities/permissions for the QEMU executable, and also if necessary add your username to the cap_net_admin field in the file
$ sudo setcap cap_net_admin=ei /usr/bin/qemu-system-x86_64Now use virt-install to start the install of the Ubuntu VM guest. I am creating a guest with name vmict, 1.5GB of RAM and installing on onoe of the available Logical Volumes setup previously. That is, the guest will run from a raw partition, not from a disk file.
$ sudo virt-install --connect qemu:///system \ --name vmict \ --ram 1500 \ --os-type=linux \ --os-variant=ubuntukarmic \ --accelerate \ --cdrom /home/sgordon/usb/lucid-server-amd64.iso \ --disk path=/dev/ictweb/vmict \ --vnc \ --noautoconsole \ --network network=defaultAssuming no errors, the VM guest should have started. To check that it is running use virsh:
$ virsh -c qemu:///system list Id Name State ---------------------------------- 1 vmict runningNow to see the graphical output of the guest (and proceed with the Ubuntu installation) we need to connect to it with VNC. On my laptop, which is running Ubuntu, is on the 192.168.1.0 network and has virt-viewer installed, I connected to the running guest:
$ virt-viewer --connect qemu+ssh://email@example.com/system vmictThe root password of the host machine will be asked for (twice). Then you should see the output of the VM in a window, allowing you to start the Ubuntu installation. In this case if you press F4 you can choose a Minimial virtual machine install. After installing Ubuntu on the guest, you should again connect and configure networking. In particular, edit the
auto lo iface lo inet loopback auto eth0 iface eth0 inet static address 192.168.122.11 network 192.168.122.0 netmask 255.255.255.0 broadcast 192.168.122.255 gateway 192.168.122.1Also make sure the SSH server is running (it should be if you installed openssh-server or selected the SSH Server option in the Ubuntu install). Now you should be able to access the VM guest at its IP alias via SSH. From the host or some other computer:
$ ssh 192.168.1.11You should also test access the web server and email server, e.g. using telnet you should be able to connect:
$ telnet 192.168.1.11 80 Trying 192.168.1.11... Connected to 192.168.1.11. Escape character is '^]'. ^C Connection closed by foreign host.
Console Access to Virtual MachineOnce logged into a guest via VNC or SSH, configure console access. This allows access to the guest from the host without using VNC or SSH. This is useful if you make a mistake on the guest and disable SSH or network access; you can obtain direct access to the guest via the host. Create and edit the file
# ttyS0 - getty # # This service maintains a getty on ttyS0 from the point the system is # started until it is shut down again. start on stopped rc RUNLEVEL= stop on runlevel [!2345] respawn exec /sbin/getty -L 115200 ttyS0 xtermThe service will start whenever the guest VM starts. However, you need to manually start it now (or reboot the guest):
$ sudo start ttyS0 ttyS0 start/running, process 1646Now on the host machine you can use the console command in virsh to connect, although you must do it as sudo:
$ sudo virsh -c qemu:///session Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # list Id Name State ---------------------------------- 2 vmict running 4 vmit running virsh # console vmit Connected to domain vmit Escape character is ^] Ubuntu 10.04.2 LTS it ttyS0 it login:Note that after running the console command and the initial Escape character message is printed, you may have to press Enter to get the login. And once you log out, to exit the console use Ctrl-].
Starting VM Guests on BootTo make sure the VM guests start automatically when the host boots, use virsh:
$ virsh -c qemu:///system Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # autostart vmict Domain vmict marked as autostarted virsh # autostart vmit Domain vmit marked as autostarted
Created on Sun, 31 Jul 2011, 5:52pm
Last changed on Wed, 16 May 2012, 8:19pm