CS 401 – Building a Small Cluster with Eucalyptus
I consider myself an amateur system administrator. I know my way around a Linux machine. I know how to use basic command line utilities. I know how to use Vim. I work on servers at my internship. I maintain about 4. Now, here I am in CS 401… with 8 machines that I need to not only install an OS on and configure, but make them talk to each other as a full Eucalyptus cluster. It’s taking far longer to set this up than expected, but I feel accomplished because it took a lot of work to get to where I am today.
I came into this project knowing literally nothing about cloud computing. But from reading the Eucalyptus documentation, I have a vague understanding of what the various components do.
The first component is the Cloud Controller (CLC). The CLC is the gateway for access to a cluster. It basically runs the show. Note that I said clusters, plural. The CLC make high-level decisions and talks to the cluster controllers. Next, is Walrus, a persistent storage system for the users virtual machine images. The data is organized into buckets (get it?). Walrus can be installed on the same server as the CLC. The Cluster Controller (CC) is in charge of managing several Node Controllers (NCs). The CC also handles access to the Storage Controller (SC) which interfaces with different storage methods. The NCs are servers that run a virtual machine hypervisor (such as KVM or Xen). The hypervisor manages all the virtual machines (VMs) that are running on the server. Put it all together and you have yourself a cloud!
My class has 8 machines running CentOS 6.1. One server will be the CLC, running Walrus as well. One server will be the CC/SC. The remaining 6 will be our nodes. If combining tasks on the servers becomes a problem, it shouldn’t be too difficult to move them to other servers, the only downside being that we will have less NCs.
I was confused about which VM hypervisor was going to be installed on the machines. All of the Eucalyptus 2.0 guides were giving instructions to install Xen. However, when I tried to install the Xen packages, they were not found. After some googling, I learned that with CentOS 6, support for Xen was dropped in favor of KVM.
Compiling from Source
I grabbed the latest and greatest code from the Eucalyptus Launchpad page. I found 3 helpful documents that allowed me to successfully build Eucalyptus. one two three. I’ve been documenting the process as I go on the CS 401 Wiki. It took A LOT of reading and some help from the #eucalyptus IRC channel, but I did it! CS 401 is now in possession of some fresh 3.1 binaries.
Now that I have Eucalyptus 3.1 built, I need to install it on all of our machines. I have been using rsync to copy the binaries to the other servers. The CLC, CC, SC, Walrus, and NC require different Eucalyptus services to be running in order for them to function. As of this writing I am currently having some trouble initializing the databases on the NCs.
I’ve made more progress this past week than any other week so far. Hopefully we can have a functional system in a week’s time.