Passing info to EC2 instance on startup.

Amazon EC2 infrastructure provide basic but good methods to pass data to images at startup.

I have created EC2 bundled images that runs a set of scripts on stratup and configures the instance according to data received on launch.

$EC2BIN/ec2-run-instances ami-NNNNNN -d "whenibootupmakemea=webserver" -k My_Key_Pair_Name

What? The command will launch image ami-NNNNNN with my ssh keys My_Key_Pair_Name and will "whenibootupmakemea=webserver" available to the instance. See EC2 docs on how to create the key pairs.

How to get the message?
curl http://169.254.169.254/latest/user-data
Contacts the EC2 control server and request the user-data. This service is provided by amazon.

I use the following command to pass the parameter to my startup scripts.
STARTUP_REQ=`curl --silent http://169.254.169.254/latest/user-data | perl -ne " if (@a=m/whenibootupmakemea=(.*)/) {print @a[0];}"

This is so nice and clean thanks Amazon.

Installing the glusterFS modified version of the FUSE Linux Kernel module onto an EC2 system

Building the fuse module as supplied by Gluster:
The 27 Step Process: (your welcome to skip some or add your own variety)

Launch the Amazon developer image (ami-26b6534f) and ssh in.

[root@domU-12-31-35-00-29-74 fuse-2.7.0-glfs1]# history
1 gcc -v # get gcc version
2 modinfo dm_mod # see what gcc was used to compile a kernel module
3 rpm -qa fuse
4 rpm -e `rpm -qa fuse`
5 rpm -e fuse-sshfs-1.7-1.fc4.i386
6 rpm -e fuse-encfs-1.3.1-2.fc4.i386
7 rpm -e fuse-2.6.0-2.fc4 #Remove the old Fuse Utilities
8 cd /tmp
9 mkdir gluster
10 cd gluster
11 wget -c http://ftp.zresearch.com/pub/gluster/glusterfs/fuse/fuse-2.7.0-glfs1.tar.gz
12 tar xzvf fuse-2.7.0-glfs1.tar.gz
13 ls -l /usr/src/linux-`uname -r`
14 cd fuse-2.7.0-glfs1
15 ./configure --enable-kernel-module --with-kernel=/usr/src/linux-`uname -r`
16 make
17 make install
18 ls /lib/modules/`uname -r`
19 ls /lib/modules/`uname -r`/ker/ # In need Coffee
20 ls /lib/modules/`uname -r`/kernel # aaar I am brain
21 ls /lib/modules/`uname -r`/kernel/fs # dead, where
22 ls /lib/modules/`uname -r`/kernel/fs/fuse # did they hide
23 ls -l /lib/modules/`uname -r`/kernel/fs/fuse # the fuse module
24 ls -l /lib/modules/`uname -r`/kernel/fs/fuse/fuse.ko # gotcha
25 tar cvjf /tmp/kernel_modules.tar.bz2 /lib/modules/`uname -r`/kernel/
26 ifconfig # get the ip so i could scp the tar file to the image I am bundling.
27 history # show you guys what i did, this output
[root@domU-12-31-35-00-29-74 fuse-2.7.0-glfs1]#

Next I scp'd the file to the image I was testing on.
The installation is pretty simple. (this ones from memory, so let hope)
umount /data/clusterfs # or where ever you mounted your cluster.
rmmod fuse # remove the module
lsmod
cp fuse.ko /lib/modules/`uname -r`/kernel/fs/fuse/fuse.ko
depmod # sort out all module dependences
modprobe fuse # load fuse module

Now your back online and you can mount your gluster filesystem with much better stability. I have been trying to break fuse-2.7.0 but so far it has stood up to the tests.

The biggest problem that people have with compiling kernel modules is the version of gcc. For your own sanity always make sure, see step 1 and 2, that you use the same gcc version for both compiling the kernel and later compiling you kernel module.

I had to recompile the fuse module for two reason. First was stability, I had several core dumps under high load, version 2.7 of the Fuse module solved it. I am again happy with Linux, great job guys. The second was to tune fuse for better performance with glusterFS. More detail http://www.gluster.org/docs/index.php/Guide_to_Optimizing_GlusterFS

A little bit of load.

root@web0:/# uptime
02:41:39 up 9:57, 1 user, load average: 104.38, 91.41, 71.59
root@web0:/# uptime
02:41:42 up 9:57, 1 user, load average: 104.38, 91.41, 71.59
root@web0:/# uptime
02:41:43 up 9:57, 1 user, load average: 102.99, 91.34, 71.67

Load testing Amazons EC2 servers.
I am quite happy.