Building your own virtual cloud environment

NOTE: This document is a companion to the slides on building virtual cloud environments in the desktop. I intend to speak about this at the PHP cloud event as part of my talk. I am making it available here, so that I can attract comments from anyone willing to try it before the event. These are just additional notes that you may be interested in.


Virtualization during development?

  • How often has it happened to you that some code ran fine in your development box, but when you deployed it, it didn’t work because you had a load balancer logic which caused your app to break and you didn’t have a similar setup in your development environment?
  • How often has it happened to you that you got something to work, but it took you a while to replicate the same thing on a production server because you don’t remember the exact steps?
  • How often have you screwed up your laptop because you tried to install something?
  • How nice would it be if I could have a virtual environment to test the app – an environment that does not have ‘my’ identity? An identity, that mimics the production setup?
  • How nice would it be if new employees can get started with development without requiring to spend a lot of time setting up a dev environment?
  • Examples of situations where a multi-machine deployment is different from a single machine deployment:
    • my.cnf requires change to the bind-address to allow remote hosts to connect
    • We need to GRANT PRIVILEGES to the remote host and user

Let us look at a sample usecase – where I have a codeserver, a webserver and a db server on 3 different boxes and I run a deployment end to end. We will create a setup that mimics a production environment.

After creating the base VM image

  • Login to the Base VM.
  • Verify that the network is configured fine.
ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/ ubuntu@base.
ssh -t ubuntu@base. sudo apt-get update && sudo apt-get upgrade -y
ssh -t ubuntu@base. sudo apt-get install openssh-server -y
ssh ubuntu@base. ping -c 3
ssh ubuntu@base. "cat /etc/network/interfaces.old | sed 's/eth0/eth1/g' > /tmp/interfaces"
ssh -t ubuntu@base. sudo mv /etc/network/interfaces /etc/network/interfaces.old
ssh -t ubuntu@base. sudo mv /tmp/interfaces /etc/network/interfaces

Cloning the base VM

  • We now clone this base image to multiple machines
$ #Add the DNS server to the resolv.conf
$ cat /etc/resolv.conf
$ cat
sudo virt-clone -o basevm -n $domain_name -f /var/lib/libvirt/images/$domain_name.img
virsh start $domain_name
echo "Sleeping 10 seconds for the OS to boot"
sleep 10
ssh -t ubuntu@base. "echo $domain_name | sudo tee /etc/hostname"
ssh -t ubuntu@base. "sudo init 6"
echo "Sleeping 10 seconds for the OS to reboot"
sleep 10
$ ./ <server-name>

Try logging in from host to guest and guest to host. Both should work.

Code server creation

Setup the GIT server over SSH

$ ./ codeserver
$ ssh -t ubuntu@codeserver. sudo apt-get install git-core -y
$ ssh -t ubuntu@codeserver. git config --global "Gautham Pai"
$ ssh -t ubuntu@codeserver. git config --global "<your-email-here>"
$ ssh -t ubuntu@codeserver. mkdir git-data
$ ssh -t ubuntu@codeserver. "cd git-data;git init;git config --bool core.bare true"

Initializing the base repository

$ git clone ubuntu@codeserver.:~/git-data
$ cd git-data/
$ echo “Test” > test-file
$ git add test-file
$ git commit -m "Test file" -a
$ git push origin master

Web server creation

Setting up Apache

$ ./ webserver
$ ssh -t ubuntu@webserver. sudo apt-get install apache2 php5-mysql libapache2-mod-php5 git-core -y
$ ssh -t ubuntu@webserver. sudo /etc/init.d/apache2 restart

DB server creation

Setting up MySQL

$ ./ dbserver
$ ssh -t ubuntu@dbserver. sudo apt-get install mysql-server -y
$ #Allowing external connections
$ ssh -t ubuntu@dbserver. "cat /etc/mysql/my.cnf | sed 's/^bind-address.*/bind-address\t\t=' > /tmp/my.cnf"
$ ssh -t ubuntu@dbserver. "sudo mv /etc/mysql/my.cnf /etc/mysql/my.cnf.bk"
$ ssh -t ubuntu@dbserver. "sudo mv /tmp/my.cnf /etc/mysql/my.cnf"
$ ssh -t ubuntu@dbserver. "sudo service mysql restart"

Connecting various servers

Connecting web server with db server

$ #Allowing web server to connect to MySQL
$ webserver_ip=`host webserver | grep "has address" | awk '{print $NF}'`
$ ssh -t ubuntu@dbserver. mysql -uroot -e "\"GRANT ALL PRIVILEGES ON *.* TO 'ubuntu'@$webserver_ip IDENTIFIED BY 'ubuntu'\""

Checking out and deploying code

$ ssh -t ubuntu@webserver. git-data/deploy/

Bringing up the system for testing

$ #Start the domains
$ virsh start webserver && virsh start dbserver && virsh start codeserver
$ #Destroy the domains
$ virsh destroy webserver && virsh destroy dbserver && virsh destroy codeserver

Key locations

  • /etc/libvirt/qemu/ - This is where the configuration of each of the Vms is stored.
  • /var/lib/libvirt/images - This is where the actual machines reside.