In this post, we will take the next step and that is uploading (to Walrus) and registering a machine image (EMI) with Eucalyptus. Once we have registered the image, we can then run instances of this image using either Amazon EC2 API or Euca2ools.

Eucalyptus Machine Image

A machine image, or more specifically, a Eucalyptus Machine Image (EMI) is a template, stored in Walrus, that can be used to create and run instances. A EMI is a combination of:

• a kernel image,
• a ramdisk image, and
• one or more virtual hard disk image

The kernel image is more specifically referred to as Eucalyptus Kernel Image (EKI) and the ramdisk image is referred to as Eucalyptus Ramdisk Image (ERI).

Each of the images – EKI, ERI, and EMI – have associated xml files containing meta-data about the corresponding images respectively.

The EKIs, ERIs, and EMIs are stored in Walrus and can be referenced by a identifier, Image ID.

Now that we have some background information on EMI’s, let’s get started.

Download Eucalyptus Certified Image

Start by opening a browser and connecting to https://<front-end-ip-address>:8443. In my case, the front-end is 192.168.0.114 (see this post).

Log in with your admin user and password.

Next, click on the Extras tab. And download one of the “Eucalyptus-certified Images”. I picked the CentOS 5.3 i386 image (euca-centos-5.3-i386.tar.gz). I downloaded it on my front-end machine for now.

For the rest of this document, I’ll be referring to the CentOS 5.3 i386 image. Feel free to replace this with the image you downloaded as you proceed through this post.

Once you un-tar the file, you should see a directory structure similar to the following:

where in my case,
centos.5-3.x86.img – is the root disk image
kvm-kernel – directory containing kvm-based kernel and ramdisk (vmlinuz-2.6.28-11-server and initrd.img-2.6.28-11-server)
xen-kernel – directory containing xen-based kernel and ramdisk (vmlinuz-2.6.24-19-xen and initrd.img-2.6.24-19-xen)

Now we can bundle the kernel, ramdisk, and hard disk image, upload it to walrus and register each of them respectively. But before we do this, I’m going to take a small detour.

Detour: Resizing the disk image

Notice in the directory listing of euca-centos-5.3-i386, the size of the centos.5-3.x86.img is only 1 GB. If you bundle, upload and register an EMI out of this image, your instance will have only a 1 GB root partition. Which is fairly small if you plan on installing software on a running instance. For instance, if you try to install Sun Java on it, the installation will fail due to lack of space.

Hence this is a good time to increase the size from 1GB to say, 4GB.

Note: Feel free to skip this step and proceed with the bundling, uploading, and registering of the images.

To re-size the image, let’s start by creating a clean image file, new_centos.5-3.x86.img of size 4GB. We can do this using the dd command.

dd if=/dev/zero of=new_centos.5-3.x86.img bs=1M count-4096

where,
if – source. In this case, we are reading a stream of zeroes from (/dev/zero) device.
of – target. Our new image file (new_centos.5-3.x86.img). The stream of zeroes are copied to this file.
bs – read and write in blocks of 1M
count – copy 4GB of blocks

The above command may take a few minutes to complete. You see output similar to this when done:

Next, associate loop devices with each of the original (centos.5-3.x86.img) and new (new_centos.5-3.x86.img) image files. Make sure that the loop devices that you are going to use are free. You can check the next available loop device using the losetup -f command. In my case, I have /dev/loop1 and /dev/loop2 available.

losetup /dev/loop1 centos.5-3.x86.img
losetup /dev/loop2 new_centos.5-3.x86.img

Confirm the status of the loop devices

losetup /dev/loop1
losetup /dev/loop2

Next, create a ext3 file system on the new image (in our case, associated with loop device /dev/loop2) as follows:

mke2fs -j /dev/loop2

Let’s create a couple of directories that will serve as temporary mount points for our image files. And then mount the images onto those temporary directories

mkdir orig
mkdir new

mount /dev/loop1 orig
mount /dev/loop2 new

Check the list of mounts by running the mount command without any arguments. You should see the above mount points in the listing. Something similar to the following:

Also, you can list the contents of the orig (original image) directory. You should see an entire linux file system.

We can now copy the contents of the original (centos.5-3.x86.img) image to the new image new_centos.5-3.x86.img) as follows:

(cd old; tar zcf - .) | (cd new; tar zxf -)

The above command will take a few minutes to complete. Once it is done you can list the contents of the new (new image) directory to confirm the copy worked ok.

Great we now have a new, bigger image new_centos.5-3.x86.img that we can now bundle, upload and register with Eucalyptus.

Before you proceed, make sure you clean up by un-mounting the images and detaching them from the loop devices

umount /dev/loop1
umount /dev/loop2

losetup -d /dev/loop1
losetup -d /dev/loop2

Bundle, Upload, Register

Now would be a good time to install Euca2ools if you haven’t done so already.

There are 3 steps to making an EMI available:

1. Bundling
2. Uploading
3. Registering

And since an EMI is made up for a kernel image, a ramdisk image and a hard disk image, we need to perform the above 3 steps on each of the kernel, ramdisk and hard disk image.

Eucalyptus Kernel Image

First, we bundle the kernel image using Euca2ools euca-bundle-image command as follows:

euca-bundle-image -i xen-kernel/vmlinuz-2.6.24-19-xen --kernel true --arch i386

where,
xen-kernel/vmlinuz-2.6.24-19-xen – is the path to the kernel image
i386 – is the architecture. In our case, 32-bit architecture
The other argument –kernel true tells the euca-bundle-image that this is a kernel image.

The above command packages the kernel image and generates a manifest.xml file under /tmp directory.

We will pass the full path to the manifest file as an argument to euca-upload-image to upload the image to Walrus:

euca-upload-bundle -b kernel-bucket -m /tmp/vmlinuz-2.6.24-19-xen.manifest.xml

where,
kernel-bucket – is the name of the bucket that the image (kernel image) will be stored under.
/tmp/vmlinuz-2.6.24-19-xen.manifest.xml – is the manifest.xml that was generated by the euca-bundle-image command

As you can see from the output, the kernel image was uploaded to kernel-bucket in Walrus. Note that you could replace kernel-bucket with a name of your choice.

Once we have uploaded the image, we need to register it so that it is available for use. We can do this using the euca-register command:

euca-register kernel-bucket/vmlinuz-2.6.24-19-xen.manifest.xml

where,
kernel-bucket/vmlinuz-2.6.24-19-xen.manifest.xml – is the path to the kernel image in walrus

The output of the euca-register command is an image id. We can use this image id when we bundle up the hard disk image or when we create an instance from a Eucalyptus machine image. The prefix of “eki” implies that the image is a kernel image, more specifically a Eucalyptus Kernel Image (EKI).

Verify that the EKI image is available by running euca2ools euca-describe-images command:

euca-describe-images

Eucalyptus Ramdisk Image

Bundle the ramdisk image as follows:

euca-bundle-image -i xen-kernel/initrd.img-2.6.24-19-xen --ramdisk true --arch i386

where,
xen-kernel/initrd.img-2.6.24-19-xen – is the path to the ramdisk image
i386 – is the architecture. In our case, 32-bit architecture
The other argument –ramdisk true specifies that this is a ramdisk image.

The above command packages the ramdisk image and generates a manifest.xml file in the /tmp directory.

Next, we pass the path to the manifest.xml file as an argument to the euca-upload-bundle command to upload the image to Walrus.

euca-upload-bundle -b ramdisk-bucket -m /tmp/initrd.img-2.6.24-19-xen.manifest.xml

where,
ramdisk-bucket – is the name of the bucket that the ramdisk image will be stored under.
/tmp/initrd.img-2.6.24-19-xen.manifest.xml – is the manifest.xml that was generated in the previous step

Next we register the image as follows:

euca-register ramdisk-bucket/initrd.img-2.6.24-19-xen.manifest.xml

The output of the above euca_register command is an image id. Again, we can use this image id when we bundle up the hard disk image or when we create an instance from a Eucalyptus machine image. The prefix “eri” implies that the image is a ramdisk image, more specifically a Eucalyptus Ramdisk Image (ERI).

Run the describe images command to verify that the ramdisk image is available:

euca-describe-images

Finally we are ready to create the Eucalyptus Machine Image.

Eucalyptus Machine Image

We will use the hard disk image – new_centos.5-3.x86.img – we created in the section “Resizing the disk image“. If you skipped the “resize” section, you could use the base image – centos.5-3.x86.img – provided in the download.

Let’s get started with first bundling the hard disk image as follows:

euca-bundle-image -i new_centos.5-3.x86.img --kernel eki-90461381 --ramdisk eri-E85914C7

where,
new_centos.5-3.x86.img – is the path to the hard disk image

The other two arguments to the above command are the Eucalyptus Kernel Image (EKI) id and the Eucalyptus Ramdisk Image (ERI) id. In this case, eki-90461381 and eri-E85914C7 which we obtained when registering the kernel and ramdisk image respectively.

As expected, the above command packages the disk image and generates a manifest.xml file in the /tmp directory.

Next, we upload the disk image to Walrus passing in the manifest.xml file path obtained from the previous command as follows:

euca-upload-bundle -b image-bucket -m /tmp/new_centos.5-3.x86.img.manifest.xml

where,
image-bucket – is the name of the bucket that the disk image will be stored under.
/tmp/new_centos.5-3.x86.img.manifest.xml – is the manifest.xml that was generated in the previous euca-bundle-image step

Finally, we register the image as follows:

euca-register image-bucket/new_centos.5-3.x86.img.manifest.xml

A quick euca-describe-images reveals the EMI that is available for use:

In the next post, we will look into creating instances of the EMI that we just registered.

• Installed Eucalyptus, an Infrastructure-as-a-Service offering, and
• Configured Eucalyptus to start VMs with IP addresses from a private VLAN

Before we jump into bundling, uploading, and registering images, let’s first download and install Euca2ools.

Euca2ools

Euca2ools are command-line utilities to help:

• Manage Images (bundle, upload, register, delete, etc.)
• Manage Instances (start, reboot, terminate, list, etc.)
• Manage Volumes & Snapshots (attach, detach, list, create, delete, etc.)
• Manage IP addresses (associate, disassociate, list, etc.)
• Manage SSH key pairs (add, delete, list)
• Manage Security groups (add, delete, list, add rules)
• Query availability zones (clusters)

Euca2ools is compatible with Amazon EC2 and S3 web services. In fact, Euca2ools is modeled after Amazon EC2 API and EC2 AMI tools.

Download & Install

Since I have been using CentOS 5 32-bit, I’ll describe the tar ball install process for CentOS 5. Also, in my case I’ll install euca2ools on my front-end machine (192.168.0.114)

Note: You may choose to go the yum (you will need to add the http://www.eucalyptussoftware.com/downloads/repo/euca2ools/$VERSION/yum/centos/ repo first) route instead of the tar ball.

Note: If you are using a Debian-based distro, feel free to install using apt-get (apt-get install euca2ools)

First download the tar.gz package for CentOS 5 32-bit – in this case, euca2ools-1.2-centos-i386.tar.gz

Next un-tar the above package

tar zxvf euca2ools-1.2-centos-i386.tar.gz

Before you install the rpm packages in the euca2ools-1.2-centos-i386 directory, you need to install swig as follows:

yum update
yum install swig

Next, go into euca2ools-1.2-centos-i386 directory and install the rpm packages as follows:

cd euca2ools-1.2-centos-i386
rpm -Uvh python25-2.5.1-bashton1.i386.rpm python25-libs-2.5.1-bashton1.i386.rpm euca2ools-1.2-1.i386.rpm

Confirm that euca2ools is install as follows:

rpm -qa euca2ools

Testing Euca2ools

In the first post I used Amazon EC2 API tools to test the Eucalyptus install. To be precise I had used the ec2-describe-availability-zones command.

Now that we have Euca2ools installed we can use the euca-describe-availability-zones command instead. Remember to source in the eucarc file first.

cd .euca
source eucarc
euca-describe-availability-zones verbose

Nice!

Feel free to play around with some of the other commands. You can get a list of commands by typing “euca” and pressing the TAB button twice. You could also run the command ls /usr/bin | grep euca

To get more information on a particular command, type in the command name following by the argument –help. For instance:

euca-describe-availability-zones --help

As we continue these series, we will see more and more of the Euca2ools commands.

In my previous post on setting up a private cloud, we looked at getting Eucalyptus “Infrastructure-as-a-Service” platform installed and running. If you recall, we stated to keep things simple but still practical enough such that you could use this as a reference guide to “Building and running a private cloud”. Hence in this post we will look at tweaking the default configuration – primarily changing the default out-of-the-box enabled networking to something similar to Amazon EC2.

Networking in Eucalyptus

Eucalyptus comes with 4 networking modes.

• SYSTEM
• STATIC
• MANAGED
• MANAGED-NOVLAN

SYSTEM networking mode

SYSTEM networking mode is the default “no-frills” networking that is offered as part of an out-of-the-box Eucalyptus install.
In this networking mode, Eucalyptus relies on the existence of a DHCP server (non-Eucalyptus controlled) located on the LAN. Virtual machines obtain their IP address from this DHCP server the same way other machines (such as your desktop, laptop, etc.) on the LAN do.
In this mode, you cannot set up create VLANs or isolate network traffic. There is only one IP address that gets assigned to the VMs i.e. the IP address obtained from the DHCP server setup by your administrator.
This mode is useful when you are just getting started with Eucalyptus. If you have only a single machine (server/laptop/desktop) to try out Eucalyptus, then SYSTEM (or STATIC) networking mode is the way to go.

As described in my previous post, I had setup Eucalyptus with 1 Cluster Controller (CC), 2 Node Controllers (NC) and SYSTEM networking mode. But we will tweak this to use MANAGED mode.

MANAGED networking mode

This mode almost resembles the networking setup of Amazon EC2 cloud in that you can define:

• a large private VLAN from which VMs can obtain IP addresses
• a pool of public IP addresses that can be assigned to VMs. Similar to Amazon’s elastic IP addresses
• define security groups where users can define ingress rules that apply to the VM that runs within that security group

In this mode, Eucalyptus maintains a DHCP server, fully manages the local VM instance network and provides all the networking features Eucalyptus currently supports.

Before we jump into configuring Eucalyptus for MANAGED networking there are some requirements that need to be met. Namely:

• Available range of IP addresses unused on the network. These IP addresses will be used to create private VLANs.
• Any switch ports that the Eucalyptus components are connected to allow and forward VLAN tagged packets
• There is either no firewall running on the Cluster Controller or the firewall is compatible with dynamic changes Eucalyptus will make to the front-end netfilter rules

Checklist #1: Available range of IP addresses for private VLAN

Before you start configuring Eucalyptus for MANAGED networking mode, you need to find a range of IP addresses that is unused on the network. We will configure Eucalyptus to use this range to create a private VLAN.

When Eucalyptus boots a virtual machine instance it will assign the instance a private IP address from this range. This is similar to the private IP address assigned to an instance running on Amazon EC2.

In my case, I have 10.10.0.0 – 10.10.255.255 unused as an example.

Checklist #2: Allow/forward VLAN tagged packets

Next we need to verify that the local network will allow/forward VLAN tagged packets between machines running Eucalyptus components.
To test this, we will create, on the front-end and Nodes, virtual ethernet devices assigned with an IP address from the above range of IP addresses. And then attempt to ping them.

Let’s start with the front-end. In my case, 192.168.0.114 if you have been following this series (see post on Setting up a private cloud using Eucalyptus for list of machines involved).

vconfig add eth0 10

where,
eth0 – is the value of the VNET_PRIVINTERFACE in my eucalyptus.conf on my front-end

We can verify we created a VLAN device on eth0 by checking /proc/net/vlan/config

cat /proc/net/vlan/config

We could also run the following command to verify that the VLAN device was created

ip a sh eth0.10

Next let’s pick an IP from the “Available range of IP addresses for private VLAN“, 10.10.0.0 – 10.10.255.255 (in my case). Let’s pick 10.10.1.2

ifconfig eth0.10 10.10.1.2 up

Let’s verify that the virtual ethernet device eth0.10 is up

ip a sh eth0.10

Excellent. Now let’s do the same on the Nodes with the exception that we will pick a different IP address, say 10.10.1.3 and 10.10.1.5 for each of my two Nodes, 192.168.0.19 and 192.168.5.7.

For the sake of brevity, I’ve detailed the steps on one of my Nodes.

vconfig add eth0 10

where,
eth0 – is the value of the VNET_PRIVINTERFACE (and VNET_PUBINTERFACE) in my eucalyptus.conf on my Nodes

Verify that the virtual ethernet device has been created on the Node:

cat /proc/net/vlan/config

Next assign the virtual ethernet device eth0.10 IP address 10.10.1.3 and bring it up

ifconfig eth0.10 10.10.1.3 up
ip a sh eth0.10

Now comes the test. From the above Node, let’s ping the front-end’s virtual ethernet device 10.10.1.2:

ping 10.10.1.2

Next, from the front-end machine, 192.168.0.114, let’s ping the Node’s virtual ethernet device, 10.10.1.3

ping 10.10.1.3

Super cool! This proves that my switch allows/forwards VLAN tagged packets. If for some reason, your results do not resemble mine, then your switch, perhaps, needs to be configured to allow/forward VLAN tagged packets. A little bit of googling on your switch may help.

For cleanup sake, you could go ahead and remove the above test vlan devices. You could do this as follows on the front-end and Node:

ip link set eth0.10 down
vconfig rem eth0.10

cat /proc/net/vlan/config

ip a sh eth0.10

Checklist #3: Firewall configuration on front-end

Finally in the list of things to check, we need to make sure the firewall on the front-end does not interfere with Eucalyptus which will dynamically update the nat and filter rules. In my case my iptables on the front-end reveal the following:

iptables -L

iptables -L -t nat

Ok. Now we are ready to proceed with the MANAGED networking configuration.

Front-end configuration – MANAGED networking mode

Presently to configure Eucalyptus networking, you need to edit eucalyptus.conf. In my installation, this file is located under /etc/eucalyptus.

All of the options that we plan on configuring are located under the “Networking options” section in eucalyptus.conf namely options starting with “VNET_”

We, first, start by configuring VNET_PRIVINTERFACE and VNET_PUBINTERFACE. VNET_PRIVINTERFACE should be set to the ethernet device that is attached to the same physical ethernet as the Nodes. In my case, eth0.

VNET_PRIVINTERFACE="eth0"

Next, if your front-end has a second ethernet device, say eth1, which is used to access the public network, you could configure VNET_PUBINTERFACE to this. In my case, I have only one ethernet device eth0. Therefore I set VNET_PUBINTERFACE to eth0

VNET_PUBINTERFACE="eth0"

Ignore the VNET_BRIDGE since it is only valid for the Nodes.

In MANAGED configuration, Eucalyptus maintains a DHCP server that it uses to dole out IP addresses to virtual machine instances. Therefore it needs the location of dhcp daemon. In my case this is /usr/sbin/dhcpd and in my CentOS OS it is configured to run as root user. Therefore I leave the VNET_DHCPUSER commented out since by default Eucalyptus will setup the DHCPD configuration files/directories to be owned by root user.

VNET_DHCPDAEMON="/usr/sbin/dhcpd"
#VNET_DHCPUSER="root"

If your DHCP daemon is set to run as a non-root user (for example, in Ubuntu it is set to run under dhcpd user), then un-comment VNET_DHCPUSER and update this value accordingly.

Next, comment out VNET_MODE=SYSTEM (out-of-the-box networking mode) since our objective is to use MANAGED networking.

So, let un-comment out the line VNET_MODE=”MANAGED”.

Next, let’s set the values for VNET_SUBNET and VNET_NETMASK. In my case, according to “Checklist #1: Available range of IP addresses for private VLAN“, we define these values as follows:

VNET_SUBNET="10.10.0.0"
VNET_NETMASK="255.255.0.0"

Update your values accordingly. This makes 65536 (256 * 256 = 65536) IP addresses available to Eucalyptus to assign as private IP addresses to virtual machine instances.

Next, I set the number of IP addresses allowed per network (security group per user) to 32 i.e. the option VNET_ADDRSPERNET:

VNET_ADDRSPERNET="32"

The setting above allows for 2048 (65536 / 32 = 2048) networks to be active simultaneously. Depending on your VNET_SUBNET, VNET_NETMASK, and VNET_ADDRSPERNET values, the number of concurrent active networks will be different that mine.

Also, in my case if I end up having, say, 100 users, then each user will have a maximum of 20 networks (2048 / 100 = 20.48) in operation at any given point in time.

Next, we set the VNET_DNS to the same DNS server used by my front-end machine. You can find your DNS server by looking in the /etc/resolv.conf file as follows:

cat /etc/resolv.conf

So I set VNET_DNS with:

VNET_DNS="192.168.0.2"

Next, you will want to assign public IP addresses to your instances – very much like Amazon EC2 instances. This gives users the ability to log into their instances from outside the cluster/front-end. But first you must find a set of public IP addresses that are not in use.

Note: Talk to your LAN administrator at this point to see if he can give you a bunch of IP addresses (a range will be great) that Eucalyptus can use to assign to instances at boot or dynamically at instance run time.

The public IP addresses you pick must be capable of being assigned to the front-end. In my case, I have 192.168.3.1 – 192.168.3.255 addresses available on my LAN network. I confirm that the IP addresses can be assigned to the front-end NIC eth0 as follows:
For example, for IP address 192.168.3.1

ip a add 192.168.3.1/32 dev eth0
ping 192.168.3.1

Perfect. I now configure VNET_PUBLICIPS as follows:

VNET_PUBLICIPS="192.168.3.1-192.168.3.31"

Note: I’ve deliberately, for no particular reason, configured the range to be only allow for 31 public IP addresses.

If you have individual IP addresses instead of range, then separate each IP address with a space.

Next, comes VNET_CLOUDIP and VNET_LOCALIP. Since my Cloud Controller and Cluster Controller are running on the same machine, 192.168.0.114, I leave VNET_CLOUDIP commented out.
Also, since I’m running a single Cluster Controller I leave VNET_LOCALIP commented out as well.

#VNET_LOCALIP="your-public-interface's-ip"
#VNET_CLOUDIP="your-cloud-controller's-ip"

If your installation has multiple Cluster Controllers, and you wish to specify the IP of the Cluster Controller that all other Cluster Controllers can reach, you can set VNET_LOCALIP to that Cluster Controller’s IP address.

That’s all with configuring Eucalyptus to use MANAGED networking mode on the front-end. To summarize my settings on the front-end are as follows:

VNET_PUBINTERFACE="eth0"
VNET_PRIVINTERFACE="eth0"
...
...
VNET_DHCPDAEMON="/usr/sbin/dhcpd"
#VNET_DHCPUSER="root"
...
...
VNET_MODE="MANAGED"
VNET_SUBNET="10.10.0.0"
VNET_NETMASK="255.255.0.0"
VNET_DNS="192.168.0.2"
VNET_ADDRSPERNET="32"
VNET_PUBLICIPS="192.168.3.1-192.168.3.31"
#VNET_LOCALIP="your-public-interface's-ip"
#VNET_CLOUDIP="your-cloud-controller's-ip"
...
...
#VNET_MODE="SYSTEM"

Node configuration – MANAGED networking mode

Next, we need to configure the Nodes for MANAGED networking mode. In my case, my Nodes are 192.168.0.19 and 192.168.5.7.

Again, all the options that we plan on configuring are located under the “Networking options” section in eucalyptus.conf. We are mainly concerned with options VNET_PRIVINTERFACE, VNET_PUBINTERFACE, and VNET_MODE.

Let’s start with VNET_PRIVINTERFACE and VNET_PUBINTERFACE. Both these options should be set to the ethernet device that is attached to the same physical ethernet as the Cluster Controller. In my case, eth0.

VNET_PUBINTERFACE="eth0"
VNET_PRIVINTERFACE="eth0"

We also un-comment the line VNET_MODE=”MANAGED” and comment VNET_MODE=”SYSTEM”

That’s all with configuring Eucalyptus to use MANAGED networking mode on the Nodes. To summarize my settings on the Nodes are as follows:

VNET_PUBINTERFACE="eth0"
VNET_PRIVINTERFACE="eth0"
...
...
VNET_MODE="MANAGED"
#VNET_MODE="SYSTEM"

Now we are ready to restart Eucalyptus on both front-end and Nodes.

Restart Eucalyptus with MANAGED networking mode

On the front-end, first do a “clean” start of the Cluster Controller as follows:

/etc/init.d/eucalyptus-cc cleanstart

Note: A clean start of the Cluster Controller is necessary when you update any Eucalyptus settings for the changes to take effect.

Next, start the Cloud Controller:

/etc/init.d/eucalyptus-cloud start

Verify that Eucalyptus (Cloud Controller, Cluster Controller) is started on the front-end:

ps auxww | grep euca | grep -v euca

Moving on to the Nodes, start the Node Controllers as follows:

/etc/init.d/eucalyptus-nc start

Verify that Eucalyptus (Node Controller) is started on the Nodes:

ps auxww | grep euca | grep -v euca

Testing our Eucalyptus MANAGED networking

Recall in our post on Setting up a private cloud using Eucalyptus, we had installed the Amazon EC2 API Tools. Let’s run a few commands to test Eucalyptus MANAGED networking, especially some of the settings such as the public IP addresses that we set in the Eucalyptus configuration on the front-end:

ec2-describe-availability-zones verbose

You will notice in the above output that it seems like I have some instances running – free < max. Infact I did start an instance (using ec2-run-instances command) so we could see MANAGED networking in action.

ec2-describe-instances

As you can see my instance i-4BAA0834 has a public IP address of 192.168.3.1 and a private IP address of 10.10.1.2.

When Eucalyptus booted up my instance it assigned the instance 192.168.3.1 from the list of public IP addresses (see VNET_PUBLICIPS in section Front-end configuration – MANAGED networking mode). Eucalyptus also assigned the instance a private IP address 10.10.1.2 from the range of unused IP addresses (see VNET_SUBNET in section Front-end configuration – MANAGED networking mode).

Also, if you run ec2-describe-addresses you will see the public IP addresses (VNET_PUBLICIPS) that are in use and available.

ec2-describe-addresses

So now when Eucalyptus boots up instances, the instances will be assigned a private address and a public address, similar to Amazon EC2 instances.

So you are now running a Amazon EC2-like cloud in your datacenter using Eucalyptus!

In up-coming posts in this series of “Building a private Cloud”, we will look how to bundle/register images, run instances, and more importantly how Platform-as-a-Service complements Infrastructure-as-a-Service.

For the sake of keeping things simple but still practical enough, we will have:

• 1 front-end machine. This will house the Cloud Controller (CLC) and Walrus. Since we intend to keep things fairly simple we will limit ourselves to a single cluster and setup the Cluster Controller (CC) and Storage Controller (SC) on this same machine. In my case, this machine has one network interface (NIC) with an IP address of 192.168.0.114.
• 2 machines (Nodes) that will serve as hosts running Xen hypervisor for the virtual machines i.e. each machine will have a Node Controller (NC) installed. In my case, each machine has a single NIC and the IP addresses are 192.168.0.19 and 192.168.5.7 respectively.

Before we install Eucalyptus we need to first prep these machines.

Note: For the rest of this document, run the commands as root user.

This document is organized as below. Feel free to skip any sections if you have already implemented the steps in that section.

Prep work

On the front-end machine, we first install Java and Ant. You can download Sun JDK from here and Ant from here . I’m using JDK version 1.6u20 (jdk-6u20-linux-i586-rpm.bin) and Ant version 1.8.0 (apache-ant-1.8.0-bin.tar.gz).

Once you have downloaded Sun JDK to a directory, install it as follows:

chmod +x jdk-6u20-linux-i586-rpm.bin
./jdk-6u20-linux-i586-rpm.bin

You can confirm that java is on the PATH by running the following command:

java -version

You should output similar to:

Next, install Ant under /opt directory as follows:

cd /opt
mkdir ant
cd ant
tar zxvf ~/apache-ant-1.8.0-bin.tar.gz
ln -s apache-ant-1.8.0 latest

Next, we need to add an environment variable ANT_HOME that points to /opt/ant/latest and append the $ANT_HOME/bin to the PATH environment variable. Add this to the /etc/profile file as follows:

cd /etc
cp profile profile.ORIG
echo "export ANT_HOME=/opt/ant/latest" >> profile
echo "export PATH=\$PATH:\$ANT_HOME/bin" >> profile

Next we need to install a few dependencies (dhcp, bridge-utils, httpd, xen-libs, ntp) and synchronize the system clock on the front-end machine. You can do this as follows:

yum update
yum install dhcp xen-libs httpd bridge-utils ntp
ntpdate pool.ntp.org

I have the following versions installed:

yum list dhcp xen-libs httpd bridge-utils

We also allow the front-end machine to forward IP packets as follows:

cd /etc
cp sysctl.conf sysctl.conf.ORIG
sed -i "s/net.ipv4.ip_forward = 0/net.ipv4.ip_forward = 1/" sysctl.conf

To change this value immediately without rebooting, run the following command:

sysctl -p /etc/sysctl.conf

Next, we need to configure firewall rules to permit the various Eucalyptus communicate with each other. Since we are planning on using security groups in Eucalyptus, let’s start with disabling SELinux on the front-end machine as follows:

cd /etc/selinux
cp config config.ORIG
sed -i "s/SELINUX=permissive/SELINUX=disabled/" config

Let’s reboot the front-end machine at this point.

Next, we need to prep the two Nodes. We start by installing xen hypervisor and also synchronize the system clock on each Node as follows:

yum update
yum install xen ntp
ntpdate pool.ntp.org

I have the following versions of xen installed:

yum list xen

Once we have Xen installed we need to configure it to allow for the hypervisor to be controlled via HTTP from localhost. We can do this by editing /etc/xen/xend-config.sxp file and then restart xen daemon as follows:

cd /etc/xen
cp xend-config.sxp xend-config.sxp.ORIG
sed -i "s/#(xend-http-server no)/(xend-http-server yes)/" xend-config.sxp
sed -i "s/#(xend-address localhost)/(xend-address localhost)/" xend-config.sxp
/etc/init.d/xend restart

Next we need to make sure the correct kernel with xen enabled is started at boot. We do this by editing the GRUB configuration file (grub.conf) under /boot/grub. If grub.conf is not available, then edit menu.lst which should be a file instead of a symlink to grub.conf.
In my case, /boot/grub/grub.conf is:

The default line is the line we want to change. The first title is 0. Since we want title CentOS (2.6.18-164.15.1.el5xen) to be the default kernel we would set default to 0.
We can do this as follows:

cd /boot/grub
cp grub.conf grub.conf.ORIG
sed -i "default=1/default=0/" grub.conf

Next, we disable SELinux on the Node machines as follows:

cd /etc/selinux
cp config config.ORIG
sed -i "s/SELINUX=permissive/SELINUX=disabled/" config

Let’s reboot both Node machines at this point. We are not ready to proceed with the installation of Eucalyptus.

Download Eucalyptus

You could choose to install Eucalyptus via yum if needed which is easier that
You can downloaded Eucalyptus from here. I picked the 32-bit CentOS 5 rpms that come bundled in a gzip compressed tar file.

Note: Different Eucalyptus components need to be installed on the front-end and each of the Node machines. The aforementioned tar.gz file contains all Eucalyptus components though. Therefore download it once on the front-end and then copy this file over to each of the Node machines.

Install Eucalyptus on the front-end

Once you have downloaded Eucalyptus (in my case, eucalyptus-1.6.2-centos-i386.tar.gz) on the front-end, untar it to root’s home folder /root.

tar zxvf eucalyptus-1.6.2-centos-i386.tar.gz
cd eucalyptus-1.6.2-centos-i386

We are ready to install. Let’s start by installing the 3rd-party dependency RPMs included in the eucalyptus-1.6.2-rpm-deps-i386 directory. Install all the rpms in this directory as follows:

cd eucalyptus-1.6.2-rpm-deps-i386
rpm -Uvh aoetools-21-1.el4.i386.rpm euca-axis2c-1.6.0-1.i386.rpm euca-rampartc-1.3.0-1.i386.rpm vblade-14-1mdv2008.1.i586.rpm groovy-1.6.5-1.noarch.rpm vtun-3.0.2-1.el5.rf.i386.rpm lzo2-2.02-3.el5.rf.i386.rpm
cd ..

Note the above “rpm -Uvh…” command might fail with an error about “Failed dependencies…java-sdk > 1.6.0 is needed….“. To get past this error, run the above rpm -Uvh… with –nodeps. The error is because it is trying to look for Openjdk during installation. But we have installed Sun Java instead. Adding –nodeps will get us past this error message. Don’t worry, the Eucalyptus components will start up fine when the time comes to run them.

Next, let’s install the Cloud Controller, Walrus, Cluster Controller, Storage Controller, and a few other dependencies on the front-end machine as follows:

rpm -Uvh eucalyptus-1.6.2-1.i386.rpm eucalyptus-common-java-1.6.2-1.i386.rpm eucalyptus-cloud-1.6.2-1.i386.rpm eucalyptus-walrus-1.6.2-1.i386.rpm eucalyptus-sc-1.6.2-1.i386.rpm eucalyptus-cc-1.6.2-1.i386.rpm eucalyptus-gl-1.6.2-1.i386.rpm

Now let’s move on the installing Eucalyptus components on the Node.

Install Eucalyptus on the Nodes

First, copy (or download) the Eucalyptus tar.gz file on each Node. Untar it to root’s home folder /root.

Note: The steps in this section need to be performed on each Node (in my case on each of the two Nodes).

Let’s begin by installing a few 3rd-party dependency RPMs.

tar zxvf eucalyptus-1.6.2-centos-i386.tar.gz
cd eucalyptus-1.6.2-centos-i386
cd eucalyptus-1.6.2-rpm-deps-i386
rpm -Uvh aoetools-21-1.el4.i386.rpm euca-axis2c-1.6.0-1.i386.rpm euca-rampartc-1.3.0-1.i386.rpm
cd ..

Next, we install the Node Controller (and a couple of dependencies) on each Node as follows:

rpm -Uvh eucalyptus-1.6.2-1.i386.rpm eucalyptus-gl-1.6.2-1.i386.rpm eucalyptus-nc-1.6.2-1.i386.rpm

Next, confirm that the user eucalyptus can connect with the hypervisor through libvirt.

su eucalyptus -c "virsh list"

The output of the above command should look something like:

Note: If you don’t have libvirt installed/running on the Nodes, you could install it: yum install libvirt

That’s it with the installation!

Running Eucalyptus

You are now ready to start Eucalyptus up.

SSH to the front-end machine and start the Cluster Controller and Cloud Controller as follows:

/etc/init.d/eucalyptus-cc start
/etc/init.d/eucalyptus-cloud start

Run ps command to confirm Eucalyptus is running on the front-end:

ps auxww | grep euca

Next, SSH to each Node and start Node Controller as follows:

/etc/init.d/eucalyptus-nc start

Confirm eucalyptus is running on the Nodes:

ps auxww | grep euca

Registering Eucalyptus components

Now that you have started all components, you will need to register them so that they can talk to each other.

SSH to the front-end machine (in my case, 192.168.0.114) and run the following commands:

euca_conf --register-walrus 192.168.0.114
euca_conf --register-cluster rosh-cluster1 192.168.0.114
euca_conf --register-sc rosh-cluster1 192.168.0.114

where,
192.168.0.114 – is the IP address of my front-end machine which has CLC, Walrus, CC and SC installed/running. Replace this with the IP address of your front-end machine in all the above commands.
rosh-cluster1 – is the cluster name that I used. Replace it with your own cluster name.

Next, we need to register the 2 Nodes. On the front-end machine, run the following command:

euca_conf --register-nodes "192.168.0.19 192.168.5.7"

where,
192.168.0.19, 192.168.5.7 – are the 2 Nodes in my case. Replace the above IP addresses with the IP addresses of your Nodes. Add additional Nodes separated with a space.

You can verify that the nodes are registered by verifying that value of the NODES element in the eucalyptus.conf file on the front-end reflects the node IP addresses added via the above euca_conf –register-nodes command. In my case:

grep NODES /etc/eucalyptus/eucalyptus.conf

We are done with registering the Eucalyptus components.

First-time Configuration

We are now ready to perform some quick configuration.

Using a browser, browse to https://<front-end-ip-address>:8443. In my case, https://192.168.0.114:8443. You will get a warning page stating that the “site’s security certificate is not trusted“. Since Eucalyptus is using a self-signed certificate which is not verified by a third-party that the browser trusts, shows you this warning. Accept the certificate and you will be prompted for a user_id/password. Enter admin for both.

Once you have logged in for the first time, you will be asked to change the password, set the admin email address, etc. Enter the relevant details and hit “Submit”.

On the “Configuration” web page you will see Cloud Configuration, Walrus Configuration, Clusters, etc. These should all be pre-populated. You could make changes to the configurations if you wish. I left these unchanged for now.

Next, browse to “Credentials” web page and click the “Download Credentials” zip file. Save the “euca2-admin-x509.zip” to a directory. You will need these credentials when you use client tools such as euca2ools to manage virtual machines, images, etc.
Create a .euca folder and unzip the contents of this file in this folder. Run the following command from under .euca folder:

unzip euca2-admin-x509.zip

Once you have unzipped the contents, you will find a .eucarc file that exports some variables. The EC2_URL in this case will point to your front-end machine. In my case, 192.168.0.114.

cat .eucarc

Before you run any client tools, you will need to source this file.

Testing our Eucalyptus install

To keep things simple and quickly test our Eucalyptus installation, download Amazon EC2 API Tools. Unzip the downloaded ec2-api-tools.zip to under the .euca folder that you created in the “First-time Configuration” section.

unzip ec2-api-tools.zip

Next source the .eucarc file under and run the ec2-describe-availability-zones command provided by the ec2-api-tools. From under .euca folder run the following commands:

cd .euca
source .eucarc
cd ec2-api-tools-1.3-46266/bin
ec2-describe-availability-zones verbose

You should see output similar to the following:

where,
rosh-cluster1 – is the cluster I registered using euca_conf and in my case, it corresponds to the Cluster Controller running on my front-end machine (192.168.0.114)

If you see something like the above, give yourself a pat on the back!

You are now ready to bundle images and create instances from those images on your own private infrastructure cloud!

Related Articles:
Configuring your private cloud

Note: These instructions should work with other Debian flavors as well.

Download Appistry-CloudIQ
Download the rpm package for Appistry-CloudIQ from the Community site. The free Community edition is great to start building a small enterprise cloud on-premise. For the purpose of this post, I’m Appistry-CloudIQ Platform 4.2 Community edition for RHEL 5.x, x86 32-bit version.

After you have downloaded, you should see something like this:

Create .deb package
Once you have the package downloaded, use alien to convert the rpm to a .deb package using the commands below. The package alien allows you to convert rpm packages to debian packages and then install them via dpkg. See how-to install alienhow-to.

Convert the rpm package to the debian package using the following command:

roshan@ubuntuWorkerB:~$ sudo alien -kc appistry-cloudiq-4.2.1.2-rhel5.rpm

Install the .deb pacakge
Now that you have the .deb package, install it using dpkg.

roshan@ubuntuWorkerB:~$ sudo dpkg -i appistry-cloudiq_4.2.1.2-rhel5_i386.deb

The install fails to create the user “fabricuser”. So let’s create it and add it to group “fabricuser”.

roshan@ubuntuWorkerB:~$ sudo useradd -s /bin/bash -g fabricuser fabricuser

You can confirm that the user was created by running the following command:

roshan@ubuntuWorkerB:~$ cat /etc/passwd | grep fabricuser

Change ownership of /usr/local/appistry/cloudiq to fabricuser
Change ownership of the /usr/local/appistry/cloudiq directory recursively to fabricuser using the following command:

roshan@ubuntuWorkerB:~$ sudo chown -R fabricuser:fabricuser /usr/local/appistry/cloudiq

Update scripts and create /bin/arch script
First, update the start-up scripts – fabric_keeper and fabric_system_service to explicitly use the /bin/bash interpreter. These two scripts are located under /etc/init.d directory.

roshan@ubuntuWorkerB:~$ sudo sed -ie 's/\/bin\/sh/\/bin\/bash/g' /etc/init.d/fabric_system_service
roshan@ubuntuWorkerB:~$ sudo sed -ie 's/\/bin\/sh/\/bin\/bash/g' /etc/init.d/fabric_keeper

The reason for doing this is because of the bashisms that exist in the two scripts.

Next, we need to create an executable script arch under the /bin directory.

roshan@ubuntuWorkerB:~$ echo "uname -m" > arch
roshan@ubuntuWorkerB:~$ sudo mv arch /bin/.; sudo chmod +x /bin/arch

The reason for doing this is because the start-up scripts source in /etc/fabric_env references /bin/arch which does not exist in Ubuntu since version 7.10. This causes the start-up scripts to fail. See https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/148511 for more information.

Set up symlinks for libssl.so.6 and libcrypto.so.6 shared libraries
Since the start up scripts attempt to load the libssl.so.6 and libcrypto.so.6 shared libraries, create symlinks with these names under /usr/lib directory.

roshan@ubuntuWorkerB:/usr/lib$ sudo ln -s libssl.so.0.9.8 libssl.so.6
roshan@ubuntuWorkerB:/usr/lib$ sudo ln -s libcrypto.so.0.9.8 libcrypto.so.6

If you do not create the above symlinks you will get errors like:

and

Set up init script links to start Appistry CloudIQ at boot
Next, we need to set up init scripts links to have Appistry CloudIQ start up at boot. We do this as follows:

roshan@ubuntuWorkerB:~$ sudo update-rc.d fabric_system_service start 99 2 3 4 5 . stop 01 0 1 6 .
roshan@ubuntuWorkerB:~$ sudo update-rc.d fabric_keeper start 99 2 3 4 5 . stop 01 0 1 6 .

Reboot to start Appistry CloudIQ
Reboot Ubuntu. You should see the following messages when Ubuntu is restarting.

Also, start log_monitor. You should see heartbeat messages.

roshan@ubuntuWorkerB:~$ . /etc/fabric_env
roshan@ubuntuWorkerB:~$ log_monitor 239.255.0.1

You now have the world’s best cloud platform – Appistry CloudIQ – running on Ubuntu.

In future posts, we will see how to create/migrate/manage infrastructure and applications to the “cloud” using Appistry CloudIQ

Downloads
Download the following script and place it alongside the appistry-cloudiq-X.x.rpm that you downloaded from the Appistry Community website.

cloudiq-ubuntu-install.sh

Next, run the script as follows:

roshan@ubuntuWorkerB:~$ sudo sh cloudiq-ubuntu-install.sh appistry-cloudiq-4.2.1.2-rhel5.rpm

where, appistry-cloudiq-4.2.1.2-rhel5.rpm is the version that I downloaded.

What is the Cloud, really?

As the author of the above post noted – it is certainly a buzz word (when you add computing to it) and according to the New Oxford American Dictionary you could even call it “vapor”

noun

1. a visible mass of condensed water vapor floating in the atmosphere, typically high above the ground.

But what is the cloud, really? I’m going to plagiarize the following definition from an article on CIO.com

“a pool of abstracted, highly-scalable, and managed compute infrastructure capable of hosting end-customer applications and billed by consumption”

But, IMHO, the above definition does not give the term cloud computing enough justice. But it comes close! Therefore let’s look at this definition a little closer:

pool – 2 or more
compute infrastructure (read machines) – could be commodity ($200 workstations – Ugly Betty style) or high-end multi-processor, multi-core (Paris Hilton style) machines. I’ll leave it at that for now and not elaborate further on infrastructure (it’s a loaded word).

Hmmm…so far we have 2 or more machines (starting to look like a cluster or group to me)

Now let’s understand the other words in this context:
abstract – meaning the complexity of the infrastructure is taken away or hidden from the user.
Or in other words, the user does not care whether the machines (hardware) are commodity or expensive, OR what flavor of Operating Systems (OSes) (could be one of the many Linux, Linux or Windows distros) are being used, or whether machines are runing single OSes or housing guest OSes (software)

This leaves us with: A cluster of abstract hardware/software (we’ll call it machines so it’s not a mouthful) hosting end-customer applications.

Going back to a few more words in the definition:
scalable – the ability to expand and handle additional load without having to change the software or business applications. In “Infrastructure Clouds” (another buzz word – I’ll touch on this in a future post), this means adding more machines or more images (using virtualization) to meet growing demands on the end-customer applications.

managed – select, plan, organize, control
i.e. some effort is required in supporting the infrastructure – a system or network administrator in most cases.

Again, this leaves us with:
A cluster of select machines hosting end-customer applications capable of growing when load on the customer applications increases.

Holy potatoes! Does this mean have we been using cloud computing all this while? To a certain extent, yes.

For example, when web-based email first came out you and I hopped happily on-board. Little did (and even today) we care about how many machines or what flavor of OS was (and still is) servicing our requests. All we care is that it is available 24/7. Perhaps we should add “Highly Available” the above definition.

Or how about that application that your company built in-house and then deployed on one server only to add another server and another and so on as the application got more and more popular. The application users do not know or care if there is 1 or more servers housing your application. Again, all they care about is that it meets their business needs.

Of course we all know that clustering means having something load-balance the load across each server based on some algorithm such as round-robin, cpu load per server, etc.

So is cloud computing plain old clustering front-ended with a load-balancer? Partly true – Clustering with load-balancing is only one aspect of the cloud. And it comes packaged with the cloud!

An integrated clustering solution

Certain cloud products such as Appistry EAF or cloud providers such as Amazon EC2 offer a load-balancing solution out of the box.

In traditional clustering/load-balancing, you would need to setup and configure a load-balancer to spread the load across the machines. This generally proves cumbersome if you are new to the load-balancer product/solution. In other words, you are spending:

• time – researching, configuring, setting up, managing a load balancer
• optionally, money – if you have purchased one or planning on purchasing one

By using a cloud product or cloud provider, you could save time and costs. And therefore, time and cost to market your business application is drastically reduced.

Another key point that separates cloud computing from traditional clustering is how static traditional load-balancing/clustering is. Imagine needing a physical or virtual machine to scale out your business application as it gets popular. But not being able to do it quickly because:

1. you have to procure a machine or create an image
2. provision the machine/image with your business application
3. configure the load-balancer to recognize this machine as a part of the cluster and start spreading requests to it

Using a cloud product or cloud provider alleviates such issues since you will have machines/images at your disposal right away. And when you provision and deploy your machine/image the load-balancing solution that is provided out of the box will recognize the new machine/image and start spraying client/application requests to it.

Food for thought: Imagine the situation where your application does not need the machine/image you just added and you want to take a machine out of the cluster and add it to a different cluster of machines serving another business application? Elasticity, anyone?

I’ll stop here for now on traditional clustering vs cloud computing. A more detailed pros and cons in another post perhaps.

So let’s see who can benefit from this right away?

• Companies who do not have the time and resources to invest in a IT department to manage their applications/data. Such companies could either use Amazon EC2 or some other public cloud (another buzz word – more on this in a later post) offering.
• Software-as-a-Service (more buzz words) or business application vendors who want to stay focussed on their core business and want to have a hassle-free option of hosting their applications. Like the above, such companies stand to benefit quickly from cloud offerings.
• Businesses who are spending huge amounts of money out-sourcing the management of their data centers to old fashioned IT Consulting firms who either host the data/applications off-site or place pricey consultants on-site. Such companies could also go with a public cloud or private cloud offering.
• What about developers? As a developer I’ve worked on prototypes – for customer demonstrations, proof of concepts, test some theory. And during this time, if I needed multiple machines, it was generally hard to find or I would need to jump through hoops to get my hands on these resources. Suddenly having a cloud of resources certainly sounds appealing, doesn’t it! And not having to deal with load-balancers is a bonus.

In this post I talked about about only one aspect of cloud – clustering with load-balancing which isn’t anything new. But what separates traditional clustering from cloud level clustering is the dynamic nature:

• The ability to add/remove machines and provision applications easily and quickly
• The luxury of not having to deal with the complexities of a load-balancer but still have a load-balancing solution underneath the covers

So for companies not wanting to invest time in researching, setting up clusters and managing load-balancers, but still wanting to quickly market their products, cloud is a great solution.

In the next post in this series, I’ll talk how another characteristic of cloud – scalability made easy.

]]> http://blogs.plexibus.com/2009/02/03/cloud-series-part-1-what-is-the-cloud-really/feed/ 1 http://blogs.plexibus.com/2008/06/21/code-example-using-spring-beans-with-appistry-eaf-38/ http://blogs.plexibus.com/2008/06/21/code-example-using-spring-beans-with-appistry-eaf-38/#comments Sun, 22 Jun 2008 05:13:57 +0000 roshanallan http://blogs.plexibus.com/2008/06/21/code-example-using-spring-beans-with-appistry-eaf-38/ The following is a guide to fabric-enabling (or grid-enabling) your existing Spring beans using Appistry EAF. We will use Maven for build configuration.

Pre-requisites to running the code sample provided:

• JDK 1.5+
• Appistry EAF 3.8 Release
  
• Maven 2.0.7+.
  
• An IDE to write/edit code. I use Eclipse 3.3.2 and the code in the zip file has eclipse-related files. But you could easily download the code and use it with an IDE of your choice.
• m2eclipse plugin – maven integration for Eclipse. *This is optional*, but if you are using Eclipse as your IDE, I recommend using it.
  

Installing the above software
See here on how to install Appistry EAF 3.8 Release on Cent OS, RHEL, and SuSE.
See here on how to install Appistry EAF 3.8 Release on Cent OS, RHEL, and SuSE.


Installation instructions for Java, Maven and Eclipse are provided on their respective web sites.

Creating the Maven project
This example is based on the tutorial samples (spring_hello_world – see tutorial_samples.zip or tutorial_samples.tgz) provided on this page.
Allright, let’s get started by designing some components:

HelloWorldSpringService – interface that exposes a single “greet” method. Remember – program to interfaces!

HelloWorldSpringServiceBean – provides the implementation for HelloWorldSpringService.

HelloWorldSpringClient – client that calls the HelloWorldSpringService.greet method


Since HelloWorldSpringService interface is implemented by HelloWorldSpringServiceBean and is used by HelloWorldSpringClient, let’s put HelloWorldSpringService into a common module, say HelloWorldCommon.

HelloWorldSpringServiceBean could go into a HelloWorldService module while HelloWorldSpringClient could go into a HelloWorldClient module.The above structure provides a clean separation between the interface, implementation(s), and the client.Now that we have the structure laid out, let’s start by creating these maven modules. I’m not going to use the m2eclipse plugin to create a multi-module project in eclipse, but instead will use mvn from the command line to create the modules. You are free to use the m2eclipse plugin to create the project and modules. Usage of m2eclipse is beyond the scope of this article.First, let’s create a project, HelloWorldSpring, that will be the parent of HelloWorldCommon, HelloWorldService, and HelloWorldClient. Open a shell, change to your eclipse workspace directory, type in the following command and hit enter:

mvn archetype:create
-DgroupId=com.appistry.samples
-DartifactId=HelloWorldSpring


The above command will create a HelloWorldSpring maven project.
* Non m2eclipse plugin users: If you do not plan on using m2eclipse, from the command line change directory to HelloWorldSpring directory, type the following command and hit enter:

mvn eclipse:eclipse


Since we want this to be the parent project, go ahead and delete the “src” directory under HelloWorldSpring. Next, edit the HelloWorldSpring’s pom.xml – change the packaging element’s value from jar to pom.

Next we need to create the HelloWorldCommon, HelloWorldService, and HelloWorldClient modules. Go back to your command line, and change directory to <workspace>/HelloWorldSpring. Type the following commands and hit enter:

mvn archetype:create -DgroupId=com.appistry.samples -DartifactId=HelloWorldCommon
mvn archetype:create -DgroupId=com.appistry.samples -DartifactId=HelloWorldService
mvn archetype:create -DgroupId=com.appistry.samples -DartifactId=HelloWorldClient


The above 3 commands create the HelloWorldCommon, HelloWorldService, and HelloWorldClient modules under <workspace>/HelloWorldSpring directory.
* Non m2eclipse plugin users: If you do not plan on using m2eclipse, from the command line change directory to HelloWorldSpring directory, type the following command and hit enter:

mvn eclipse:eclipse


Next, fire up Eclipse and select the workspace that you created the above maven project under. Now, create a new Project by clicking “File -> New -> Project…”. Select “General -> Project” and hit “Next”. Type “HelloWorldSpring” for Project Name and click “Finish”.
You should see, a HelloWorldSpring project with HelloWorldCommon, HelloWorldService, and HelloWorldClient under it.
If you plan on using m2eclipse plugin, now would be a good time to right-click on the project and select “Maven -> Enable Dependency Management” and then right-click again on the project and select “Maven -> Enable Nested Modules”.
* Non m2eclipse plugin users: Right-click “java” (under HelloWorldSpring -> HelloWorldCommon -> src -> main), select “Build Path -> Use as Source Folder”. Do the same for “java” under HelloWorldSpring -> HelloWorldService -> src -> main and HelloWorldSpring -> HelloWorldClient -> src -> main.Now you have the HelloWorldSpring multi-module project configured under Eclipse! See Figure 1




Figure 1


Next we need to update the poms.

HelloWorldSpring pom.xml


<?xml version=”1.0″ encoding=”UTF-8″?>
<project xmlns=”http://maven.apache.org/POM/4.0.0″ xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd”>
<modelVersion>4.0.0</modelVersion>
<groupId>com.appistry.samples</groupId>
<artifactId>HelloWorldSpring</artifactId>
<packaging>pom</packaging>
<version>1.0.0-SNAPSHOT</version>
<name>HelloWorldSpring</name><properties>
<!– Default JDK Level –>
<jdkLevel>1.5</jdkLevel>
</properties><modules>
<module>HelloWorldService</module>
<module>HelloWorldClient</module>
<module>HelloWorldCommon</module>
</modules><dependencyManagement>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>HelloWorldCommon</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</dependencyManagement><dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.4</version>
<scope>test</scope>
</dependency>
</dependencies><build>
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<!–<version>2.0.2</version>–>
<executions>
<execution>
<id>compile</id>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>unit-test-compile</id>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<source>${jdkLevel}</source>
<target>${jdkLevel}</target>
</configuration>
</plugin><plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<excludes>
<exclude>**/*$*</exclude>
</excludes>
</configuration>
</plugin><plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.1</version>
</plugin>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.0</version>
</plugin>
</plugins>
</pluginManagement>
</build>

</project>

HelloWorldService pom.xml
Add HelloWorldCommon dependency to HelloWorldService

<?xml version=”1.0″?>
<project>
<parent>
<artifactId>HelloWorldSpring</artifactId>
<groupId>com.appistry.samples</groupId>
<version>1.0.0-SNAPSHOT</version>
</parent> <modelVersion>4.0.0</modelVersion>
<artifactId>HelloWorldService</artifactId>
<name>HelloWorldService</name><dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>HelloWorldCommon</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</project>

HelloWorldClient pom.xml
Add HelloWorldCommon dependency to HelloWorldClient. You will also need to add Spring as a dependency. I’m using Spring 2.5.4.


<?xml version=”1.0″?>
<project>
<parent>
<artifactId>HelloWorldSpring</artifactId>
<groupId>com.appistry.samples</groupId>
<version>1.0.0-SNAPSHOT</version>
</parent><modelVersion>4.0.0</modelVersion>
<artifactId>HelloWorldClient</artifactId>
<name>HelloWorldClient</name><dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>HelloWorldCommon</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
<version>2.5.4</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.1</version>
</dependency>
</dependencies>
</project>




Now we are ready to code the HelloWorldSpringService, HelloWorldSpringServiceBean, and HelloWorldSpringClient.

HelloWorldSpringService
The HelloWorldSpringService has one method greet which takes a String parameter.


package com.appistry.samples.spring.helloworld;

public interface HelloWorldSpringService {
    String greet(String name);
}

This interface goes under the HelloWorldCommon module under HelloWorldCommon/src/main/java.

HelloWorldSpringServiceBean
The HelloWorldSpringServiceBean class implements HelloWorldSpringService interface. It goes under the HelloWorldService/src/main/java.

package com.appistry.samples.spring.helloworld.impl;

import com.appistry.samples.spring.helloworld.HelloWorldSpringService;

public class HelloWorldSpringServiceBean implements HelloWorldSpringService {

	public String greet(String name) {
		return "Hello World, " + name;
	}
}

HelloWorldSpringClient
Next, add the client that calls HelloWorldSpringService.greet. HelloWorldSpringClient class goes under HelloWorldClient/src/main/java.

package com.appistry.samples.spring.helloworld;

import org.springframework.context.support.ClassPathXmlApplicationContext;

public class HelloWorldSpringClient {

    public static void main(String[] args) {
	ClassPathXmlApplicationContext appContext = new ClassPathXmlApplicationContext("helloWorldClient-appContext.xml");

        HelloWorldSpringService localHWS = (HelloWorldSpringService) appContext.getBean("localHelloWorldBean");
System.out.println(localHWS.greet("locally."));

    }
}

helloWorldClient-appContext.xml is detailed below and it goes under HelloWorldClient/src/main/resources (add the “resources” directory under HelloWorldClient/src/main as a “source” folder in Eclipse):

<?xml version=”1.0″?>
<!DOCTYPE beans PUBLIC “-//SPRING//DTD BEAN//EN” “http://www.springframework.org/dtd/spring-beans.dtd”><beans>
<bean id=”localHelloWorldBean” class=”com.appistry.samples.spring.helloworld.impl.HelloWorldSpringServiceBean”/>
</beans>


Let’s package, install and run HelloWorldSpring. From the command, type the following command from under <workspace>/HelloWorldSpring to package and install HelloWorldSpring artifacts to your local maven repository.

mvn clean install


From Eclipse, run HelloWorldSpringClient as a Java Application. You should see an output like this:


…
…
Jun 20, 2008 4:35:26 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@18a7efd: defining beans [localHelloWorldBean,fabricHelloWorldBean,fabric]; root of factory hierarchy
Hello World, locally.


The HelloWorldSpringClient application loads the helloWorldClient-appContext.xml spring application context, gets a reference to the HelloWorldSpringServiceBean and calls the greet method on it. So far we have seen nothing with regards to the fabric/grid. If you would like to take a break, now would be a good time to do so. Go on. I’ll be right here.

Exposing the HelloWorldSpringServiceBean as a fabric bean
To expose our existing HelloWorldSpringServiceBean in the fabric, we need to package and deploy the HelloWorldSpringServiceBean into the fabric.
Let’s begin by first defining the HelloWorldSpringServiceBean in a spring application context (spring_hello_world_bean.xml).

<?xml version=”1.0″?>
<!DOCTYPE beans PUBLIC “-//SPRING//DTD BEAN//EN” “http://www.springframework.org/dtd/spring-beans.dtd”><beans>
<bean id=”HelloWorldBean” class=”com.appistry.samples.spring.helloworld.impl.HelloWorldSpringServiceBean”/>
</beans>

Place this file under HelloWorldService/src/main/resources
Note: Do we need to expose this in the fabric as a spring bean? No. We could have exposed it as a regular POJO. But the purpose of this example is to show Spring integration with Appistry EAF.
Now that we have wired the HelloWorldSpringServiceBean in a spring container, we need to define this bean as a Fabric Java Component. This is accomplished using the EAF Component Definition XML.

springHWSvc-component-defn.xml

<?xml version=”1.0″ encoding=”utf-8″?>
<java-components xmlns=”http://www.appistry.com/ns/component” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://www.appistry.com/ns/component eaf-component.xsd”><component name=”mvnSpringHWSvcComponent” default-timeout=”5″>
<spring-bean name=”HelloWorldBean”/>
</component>
</java-components>


where,
the name attribute of the spring-bean element should match the bean id defined in the spring_hello_world_bean.xml spring application context. In this case, HelloWorldBean.
The default-timeout specifies how long to wait for a call (to a method) to return before timing out. In this case, we have specified 5 seconds.

Place this file under HelloWorldService/src/main/scripts for now.

So how does the EAF tie the component definition file and the spring application context? This is done via the EAF Fabric Application Definition XML.The fabric application definition file describes the various files that comprise a EAF fabric application. This file is used by the Fabric Package (which we will encounter momentarily) to bundle these files into “.fabric” package.

springHWSvc-fabric-app-defn.xml

<?xml version=”1.0″?>
<!DOCTYPE app SYSTEM “FabricApp.dtd”><app name=”mvnSpringHelloWorldSvc” version=”1.0.0″>
<spring>
<bean-factory-locator bean=”myBeanFactory” resource-location=”beanRefFactory.xml”/>
</spring><components>
<file name=”springHWSvc-component-defn.xml”/>
</components><java-libs>
<file name=”HelloWorldCommon.jar”/>
<file name=”HelloWorldService.jar”/>
<file name=”spring.jar”/>
<file name=”commons-logging.jar”/>
</java-libs>
</app>


where,
components, file – is component definition file that we defined earlier
java-libs – dependant java libraries that will be included in the .fabric application.

Place this file under HelloWorldService/src/main/scripts

The interesting element in this file is the . The bean-factory-locator element provides the Fabric with an implementation of Spring BeanFactory to enable it to access spring bean containers / application contexts. The BeanFactory (provided to the Fabric) is itself described in a spring application context. In this case, beanRefFactory.xml.

beanRefFactory.xml



<?xml version=”1.0″ encoding=”UTF-8″?>
<!DOCTYPE beans PUBLIC “-//SPRING//DTD BEAN//EN” “http://www.springframework.org/dtd/spring-beans.dtd”><beans>
<bean id=”myBeanFactory” class=”org.springframework.context.support.ClassPathXmlApplicationContext”>
<constructor-arg value=”spring_hello_world_bean.xml”/>
</bean>
</beans>

The bean id in beanRefFactory.xml should match the “bean” attribute of bean-factory-locator in the fabric application definition xml. In this case, myBeanFactory.
In the above application context, we have configured the ClassPathXmlApplicationContext (a super set of the BeanFactory) and pass it the name of the application context, spring_hello_world_bean.xml, that contains our HelloWorldSpringServiceBean.

Place the beanRefFactory.xml under HelloWorldService/src/main/resources

Now we are ready to package our HelloWorldSpringServiceBean, so we can deploy it on the fabric. As you can see we have not modified the java code at all. All we have done so far is configure the HelloWorldSpringServiceBean POJO in a spring application context, describe the bean as a fabric Java component, and tie these elements (along with the BeanFactory) together using a fabric definition file. Non-intrusive, eh?

Build and Package HelloWorldSpringServiceBean
Packaging the application into .fabric file can be accomplished using the fabric pkg command. The fabric package command takes a fabric application definition file as input and bundles the application files into a single package (.fabric). We can configure the fabric package in the HelloWorldService pom.xml. But before we do this, we need to first download the dependant jars listed in the java-libs section of the fabric application definition xml (springHWSvc-fabric-app-defn.xml) to a particular directory. This can be defined in the HelloWorldService pom.xml as well. Add the following snippet (after the dependencies element and before the closing project tag) to the HelloWorldService pom.xml:

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.0</version>
<executions>
<execution>
<id>copy-required-dependencies</id>
<phase>install</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<stripVersion>true</stripVersion>
<artifactItems>
<artifactItem>
<groupId>${project.groupId}</groupId>
<artifactId>HelloWorldCommon</artifactId>
<version>${project.version}</version>
</artifactItem>
<artifactItem>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<version>${project.version}</version>
</artifactItem>
<artifactItem>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
<version>2.5.4</version>
</artifactItem>
<artifactItem>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.1</version>
</artifactItem>
</artifactItems>
<outputDirectory>${project.build.directory}/fabric-package</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>


This directs Maven (at install phase) to copy the HelloWorldCommon, HelloWorldService, spring and commons-logging jar files (with the versions stripped) from the local maven repository to the fabric-package directory under HelloWorldService/target directory. Since I’ve listed the jar files without version numbers in the element (in springHWSvc-fabric-app-defn.xml file), I tell Maven to strip the version numbers from the jar files while it’s performing the copy.

Now that we have addressed the jar dependencies, we are ready to configure the fabric_pkg command. Let’s configure the fabric package in ant build file target and call the ant build file from maven.
Note: You don’t have to necessarily configure the fabric_pkg in an ant build file. You could have configured it within the pom itself.

First let’s check out the ant build file.

build.xml (goes under HelloWorldService/src/main/scripts) – just displaying the necessary snippets. The attached HelloWorldSpring-source.zip contains all the source code listed here.

<!– copy the fabric application defn xml and component defn xml to fabric-package directory –>
<target name=”init”>
<copy todir=”target/fabric-package”>
<fileset dir=”src/main/scripts”>
<include name=”**/*-defn.xml”/>
</fileset>
</copy>
</target><!– execute fabric-pkg command –>
<target name=”fabric-package” depends=”init”>
<exec executable=”fabric_pkg”>
<arg line=”target/fabric-package/springHWSvc-fabric-app-defn.xml” />
</exec>
</target>

Now we need to update the HelloWorldService pom.xml to call the above ant build file. Add the following snippet (after the previous plugin definition and before the closing plugins tag) to the HelloWorldService pom.xml.

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.1</version>
<executions>
<execution>
<id>fabric-package</id>
<phase>install</phase>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<configuration>
<tasks>
<ant antfile=”src/main/scripts/build.xml” target=”fabric-package” />
</tasks>
</configuration>
</plugin>



Here we direct Maven (again, at install phase) to call the “fabric-package” target of build.xml. The fabric-package target first copies the springHWSvc-fabric-app-defn.xml and springHWSvc-component-defn.xml files to the HelloWorldService/target/fabric-package directory and then calls the fabric_pkg executable to bundle the application files into “mvnSpringHelloWorldSvc.fabric” file. You will notice that the name of the fabric file is the same as the name attribute of the app element in springHWSvc-fabric-app-defn.xml.

We are done with the package configuration. Now, open a command shell, change directory to /HelloWorldSpring, type the following command and hit enter:

mvn clean install


The above command will compile the sources, the application jars, install the application jars to the local maven repository on your local machine, and then create the mvnSpringHelloWorldSvc.fabric under the HelloWorldSpring/HelloWorldService directory.

Now you are ready to deploy the mvnSpringHelloWorldSvc.fabric to the EAF fabric.

Deploy HelloWorldSpringServiceBean
Let’s use the instructions detailed on the Appistry EAF wiki page – Building, Packaging, and Deploying a Fabric application. The fabric_ctl command can be used to deploy a fabric application.

To deploy the HelloWorldSpringServiceBean bundled in the mvnSpringHelloWorldSvc.fabric, type the following command (from under <workspace>/HelloWorldSpring/HelloWorldService) with the correct fabric adminstrative user, password, and fabric address. If you have stuck with the defaults, the command is:

fabric_ctl -d 239.255.0.1:30000 -u fabric-admin/fabric-admin deploy mvnSpringHelloWorldSvc.fabric


You can verify if the deployment is successful or not by following the instructions on this wiki page.

If the above command didn’t throw any errors, you have successfully deployed the HelloWorldSpringServiceBean, without any code changes, to the fabric.

Now that we have HelloWorldSpringServiceBean deployed to the fabric, how do we call the greet method on it from HelloWorldSpringClient?

Call HelloWorldSpringServiceBean remotely
Calling HelloWorldSpringServiceBean from HelloWorldSpringClient is achieved by a simple configuration in helloWorldClient-appContext.xml. And adding Appistry EAF provided fabric.jar as a dependency to the HelloWorldClient pom.xml

Since Appistry EAF fabric.jar is not available on a public maven repository as of yet, we need to install it to our local maven repository on our local machine. To do this, open a command shell, change directory to $FABRIC_HOME/classes, type the following command and hit enter:

mvn install:install-file -Dfile=fabric.jar -DgroupId=com.appistry -DartifactId=fabric -Dversion=3.8.0.15 -Dpackaging=jar


The above command will install fabric.jar to your local maven repository.

Next we need to add fabric.jar as a dependency to HelloWorldClient’s pom.xml:

<dependency>
<groupId>com.appistry</groupId>
<artifactId>fabric</artifactId>
<version>3.8.0.15</version>
</dependency>


Now we are ready to configure helloWorldClient-appContext.xml.
Append the following bean definitions to helloWorldClient-appContext.xml located under <workspace>/HelloWorldSpring/HelloWorldClient:

<bean id=”fabricHelloWorldBean” class=”com.appistry.spring.FabricProxyFactoryBean”>
<property name=”serviceInterface” value=”com.appistry.samples.spring.helloworld.HelloWorldSpringService”/>
<property name=”fabric”>
<ref local=”fabric”/>
</property>
<property name=”applicationName” value=”mvnSpringHelloWorldSvc”/>
<property name=”componentName” value=”mvnSpringHWSvcComponent”/>
</bean><bean id=”fabric” class=”com.appistry.fabric.Fabric”>
<constructor-arg value=”239.255.0.116″/>
<constructor-arg value=”31000″/>
</bean>


The bean fabricHelloWorldBean defines a com.appistry.spring.FabricProxyFactoryBean which proxies any calls made on HelloWorldSpringService to an instance of HelloWorldSpringService on the fabric. The concept behind FabricProxyFactoryBean is very similar to Spring Remoting. Infact FabricProxyFactoryBean uses Spring Remoting classes to proxy requests to the fabric.
The FabricProxyFactoryBean requires:

• the interface on which methods will be called,
• a reference to com.appistry.fabric.Fabric,
• an applicationName which should match the name attribute of the app element in the fabric application definition xml (springHWSvc-fabric-app-defn.xml),
• a componentName which should match the name attribute of the component element in the component definition xml (springHWSvc-component-defn.xml).

Let’s now add some code to HelloWorldSpringClient to call the fabricHelloWorldBean that we just defined above.

Let’s append the following two lines to the HelloWorldSpringClient’s main method:

HelloWorldSpringService fabricHWS = (HelloWorldSpringService) appContext.getBean("fabricHelloWorldBean");
System.out.println(fabricHWS.greet("from the fabric."));

Now we are ready to run HelloWorldSpringClient once again.

From Eclipse, run HelloWorldSpringClient as a Java Application. You should see an output like this:


…
…
Jun 20, 2008 4:35:26 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO:
Pre-instantiating singletons in
org.springframework.beans.factory.support.DefaultListableBeanFactory@18a7efd:
defining beans [localHelloWorldBean,fabricHelloWorldBean,fabric]; root
of factory hierarchy
Hello World, locally.
Hello World, from the fabric.


The HelloWorldSpringClient application loads the helloWorldClient-appContext.xml spring application context, gets a reference to the fabricHelloWorldBean and calls the greet method on it. The FabricProxyFactoryBean proxies the greet request to the fabric. An instance of HelloWorldSpringServiceBean on the fabric executes the greet method and returns the results to the HelloWorldSpringClient which prints the result to the standard out.

As you can see, we haven’t changed any code on the client either. Just a little bit of configuration and that was it.

Download

Extract the contents of the zip into your eclipse workspace. This will create a HelloWorldSpring folder under your workspace.

HelloWorldSpring-source.zip


Appistry EAF Developer Portal

To learn more visit the Appistry EAF Developer Portal

]]> http://blogs.plexibus.com/2008/06/21/code-example-using-spring-beans-with-appistry-eaf-38/feed/ 0