Building your own
Desktop Cloud Environment
Gautham Pai
July 9, 2011
Building your own
The problems we face as developers
- Our development environments accumulate lots of applications and libraries over a period of months.
- We fear that we may screw up our development environment and that means unproductive man-hours.
- We are usually in the habit of installing everything in one machine.
- We forget that a multi-machine deployment is different from a single machine deployment.
Solution?
How about Desktop Virtualization?
Why desktop virtualization?
- Bringing our development environment as close to the production environment as possible.
- Replicable deployment - verify everything before deploying to staging or production.
- Containerising functionality.
- "Don't panic" environment for experimentation
- Ease of provisioning for new employees.
Virtualization or cloud?
- What I am talking about is closer to virtualization than cloud computing.
- For example, we are not dealing with transparent scaling, spike load handling etc.
Why QEMU-KVM and not Virtualbox?
- KVM is part of the Linux Kernel
- Virtualbox is from Oracle
- KVM provides better performance
What we want to try out
- Create a development environment with functionalities partitioned over separate VMs.
- Ensure that setting up this environment is not more than a few hours job.
- Ensure that bringing up the environment does not take more than a few minutes if not seconds.
- Ensure that the environment can run in a laptop with standard configuration.
System Requirements
-
A laptop/desktop with
- a processor that supports virtulization technology to use KVM
- a Debian Linux environment
- atleast 2GB of free RAM
- atleast 10G of free disk space
Demo
The Approach
- In this demo, I will take you through the steps to create a multi-VM development environment
- In addition to our dev environment (dev-server), we will have a separate code-server, a web-server and a db-server setup.
- We then run it end-to-end – write some code, check it in, pull it into the web server, deploy the code and then test the new version of the app.
First, a look at a few terminologies
- QEMU is a generic and open source machine emulator and virtualizer.
- KVM (for Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 hardware.
- The Virtual Machine Manager application (virt-manager for short package name) is a desktop user interface for managing virtual machines.
Demo
The Approach
- We first create a base VM and install the OS in it, using the standard Ubuntu 10.04.2 Server ISO image
- We then install a SSH server and setup passwordless SSH access
- We then clone this system to as many systems as we need and setup various servers (code/caching/web/etc) on them
Installing Pre-requisites
$ sudo apt-get install virt-manager qemu-kvm
Starting Virt Manager
$ sudo virt-manager
Networking requirements
- Every VM should have its own IP address.
- Every VM should be accessible from every other VM and that includes host.
- Every VM must be able to connect to the Internet.
- Solution? Bridged Networking.
Setting up the Bridge Network
Name your virtual network
Choose an IPv4 Address Space
Select DHCP Range
Setup NAT forwarding
Review and finish the wizard
Create the Base VM
Choose the Distribution to Install
Set RAM and CPUs
Set the Disk Space
Finalize the VM creation
Install the OS in the Base VM
Select Hostname
Select Disk
Confirm Disk Format
Installation Progress
Installation Progress
Create User
Set Username
Configure APT
Set Automatic Upgrades
Installing Additional Software
Installing Pre-defined Collections
Installation Progress
Install GRUB
Install GRUB
Installation Complete
Installation Complete
Verifying the Base VM installation
- Login to the Base VM
- Install the latest updates
- Make sure you can connect to the host from guest and vice-versa
- Setup password-less SSH access
- (Update interface identifier)
Cloning the Base VM
- Add the DNS server that we use for guest IP allocation to resolv.conf, so that you can address VMs using their hostnames
- Use virt-clone to clone the base VM
- Start the new VM
- Change the hostname of the new VM
- Reboot the new VM
Starting and stopping the VMs
$ #Start the domains
$ virsh start webserver && virsh start dbserver && virsh start codeserver
$ #Destroy the domains
$ virsh destroy webserver && virsh destroy dbserver && virsh destroy codeserver
Where to go from here?
- Snapshot creation on the fly
- You can mimic things like S3, EBS etc
Resources
Courtesy
- Slide software from HTML5 Slides on Google Code
- Server image from OpenClipArt
- CD image from pr0gr4mm3r.com
- Thanks to Pradeep Kumar, Ananth M, Kiran S, Ananda M, Madhukar Pai and Shrinivas S Kulkarni for reviewing the slides and providing valuable feedback