This chapter deals with the starting of the Pegasus submit host virtual machine on FutureGrid using the Nimbus IaaS (preferred), or the Eucalyptus IaaS. The goal is to start a Pegasus tutorial VM in one of FutureGrid's IaaS clouds, and connect to it using your local ssh client within a terminal emulator.
Figure 2.1. Figure: Communication between your machine and the Pegasus tutorial VM in the FutureGrid Nimbus cloud.
The Nimbus cloud-client software is required to run the virtual
machine on your computer. If you do not already have it installed, as per
Nimbus tutorial above, download the latest cloud client and install it
from the Nimbus
download page. The client only requires a Java runtime environment
(JRE) to run on your system. Please make sure that you have also installed
the FutureGrid nimbus_creds.tar.gz configuration
files into your cloud client installation, as per Nimbus tutorial.
At this point, please open your terminal application, as as applicable for your local operating system. Windows users will have to use the cmd.exe application for local commands that involve the cloud client, and not the putty.exe client just yet. For Unix and Mac OS X users, please open the terminal application detailled in the first chapter.
In the next step, in your terminal, please change into the base
directory of where you installed the cloud client software, as per Nimbus
tutorial. This directory will be named
nimbus-cloud-client-020 or similar. Your operating
system's search feature may help you locate this directory.
The Pegasus tutorial image is called
pegasus-tutorial.x64.gz on the Nimbus clouds
foxtrot, hotel and
sierra and
pegasus-tutorial.x64.kvm.gz on Nimbus cloud
alamo. All these images are equivalent. You pick one
cloud, start the image, and work against that image. Assuming the
FutureGrid hotel cloud, you can query all existing
images like this:
$ bin/cloud-client.sh --conf conf/hotel.conf --list
...
[Image] 'pegasus-tutorial.x64.gz' Read only
Modified: Mar 14 2012 @ 12:38 Size: 886225239 bytes (~845 MB)
...
Similarily, if you replace the argument to the
--conf option, you can query the Nimbus images available
to you on the sierra cloud:
$ bin/cloud-client.sh --conf conf/sierra.conf --list
...
[Image] 'pegasus-tutorial.x64.gz' Read only
Modified: Mar 14 2012 @ 10:06 Size: 886225239 bytes (~845 MB)
...
You will see a number of images provided by different users and
administrators. Please look in the list for the name starting with
pegasus-tutorial.x64 string. Currently, the
alamo site uses a slightly different suffix
convention than the other three Nimbus clouds hotel,
sierra and foxtrot in
FutureGrid. Note the name of the tutorial image including all suffices.
Some details, like the size or time of last modification, may differ from
the screen shot above. Not to worry, this is perfectly normal.
Once you have identified the proper image name, you can start it providing an maximum run-time for the image. Typical FutureGrid users are limited to a maximum of 24 hours. After the maximum run-time expired without you terminating your image, Nimbus will terminate it for you. This prevents users from unecessarily blocking resources when forgetting to shut down their image instance.
Note
However, it is good behavior to terminate any running image once you are done with it. Remember that when using commercial cloud environments, you are paying for the time an image is running.
Starting the Pegasus tutorial image for 8 hours, retype (or copy and
paste) the image string, passing it as argument to the option labeled
--name:
$ bin/cloud-client.sh --conf conf/hotel.conf --run --namepegasus-tutorial.x64.gz--hours 8 ... Launching workspace. Workspace Factory Service: https://svc.uc.futuregrid.org:8443/wsrf/services/WorkspaceFactoryService Creating workspace "vm-105"... done. IP address:149.165.148.103Hostname:vm-103.uc.futuregrid.orgStart time: Fri Dec 09 12:55:08 PST 2011 Shutdown time: Fri Dec 09 20:55:08 PST 2011 Termination time: Fri Dec 09 21:05:08 PST 2011 Waiting for updates. "vm-105" reached target state: Running Running: 'vm-105'
Of course, your screen output will look different. The essential pieces of information you need to note are:
- Workspace (instance handle)
The handle for the VM in the example above is
vm-105. A handle is specific to the location of the cloud client. You will need the handle in order to terminate the VM when you are done.
- IP address
This is the public address you need to contact your image. In the example above, it's
149.165.148.103. A reliable hostname may not always be available, so the IP address is more useful.
Should the image you are attempting to start not reach the target state of Running, please try once more. Very rarely a hiccup may prevent it from running.
Should the image fail to start a second time, please try a different
FutureGrid Nimbus cloud by varying the argument to the
--conf option when invoking the
cloud-client. This is similar to the
--list example at the beginning of the section. In
effect, by varying the argument, you attempt to run at a different cloud.
For instance, if you were using an argument of
conf/hotel.conf previously, try with an argument of
conf/sierra.conf instead. Please remember that
alamo uses a different image naming
convention.
If your start attempts continues to fail, you will need to contact the FutureGrid helpdesk. However, none of this should happen to you.
Once the Image reached the running state, you
can log onto the image, i.e. contact it using the ssh client. The Pegasus
tutorial images are set up with a tutorial account that
is mapped to your public ssh key you provided to the FutureGrid portal.
The Pegasus tutorial works by using the tutorial
account. Please replace the IP address in the following command with the
one reported by the cloud client:
$ ssh -l tutorial 149.165.148.103
The authenticity of host '149.165.148.103 (149.165.148.103)' can't be established.
RSA key fingerprint is 55:08:84:70:2a:f8:61:02:a8:08:e5:c1:ab:d4:62:01.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '149.165.148.103' (RSA) to the list of known hosts.
/usr/bin/xauth: creating new authority file /home/tutorial/.Xauthority
tutorial@vm-103.uc.futuregrid.org:~ $
Note
In the output above, you may notice that ssh put the cryptographic hash of the remote host into a local file. Should you connect later to the same address, but a different virtual machine, ssh won't let you connect. In this case, you need to clean the offending address from your known hosts file.
The specific output, again, may differ in specifics in your case.
For instance, in the Windows case, you open the
putty.exe application, and type in the remote IP
address, as returned from the cloud client, into the host box. Once you
click connect, you will see a dialog asking you to
accept the host key, and then you will be logged in. At this time, you
should see a similar prompt to the one shown above.
Once you have logged into the remote virtual machine, you will be able run the exercises on that machine.
Figure 2.2. Figure: Communication between your machine and the Pegasus tutorial VM in the FutureGrid Eucalyptus cloud.
Sorry, but at this time, we have no Eucalyptus instructions for Windows users.
The Eucalyptus euca2ools software is required to run the virtual
machine on your computer. If you do not already have it installed, as per
Eucalyptus tutorial above, download the latest version of euca2ools for
your operating system. The source and some pre-assembled packages are
available from the Eucalyptus Community
pages. For Linux, you may try one of the pre-compiled binary packages.
Conveniently, for the RHEL-based family of Linux OS, there is a simple
file euca.repo you can enter into directory
yum.repos.d
[eucatools] name=Euca2ools baseurl=http://www.eucalyptussoftware.com/downloads/repo/euca2ools/1.3.1/yum/centos enabled=1 priority=10
If you installed the repo file above, you can simply tell your system to install the new software like this:
# yum install euca2ools.`uname -p` --nogpgcheckFor Mac OS X, you download the source tarball and install from source. In order to compile software, you need to have some version (any recentish version) of XCode installed. You need to tinker with the Makefile to make the compilation of euca2ools work on Mac. Replace
install -g root
with
install -g wheel
in all instances. If you decide to use the prefix
/usr/local please make sure that your shell setup
will find it. I have no solution for Windows users, except to log onto one
of the supported operating systems and work from there.
To use the Pegasus tutorial on Eucalyptus, you are need to tell
Eucalyptus that you will connect via ssh and http to the VM image. You use
a Eucalyptus security group to enable connections
from the outside world to your virtual machine. We are going to create a
security group called pegasus and use it for the
tutorial VM. The group has to be set up once per Eucalyptus domain. Before
you can run any euca2ool commands, you have to source
the eucarc file for the target Eucalyptus domain, as
will be shown in the next example below, and in the next section
below.
$. $HOME/.euca/india/eucarc # yes, the command is the period $ euca-add-group -d 'Pegasus Tutorial VM' pegasus $ euca-authorize -P icmp -t 1:-1 -s 0.0.0.0/0 pegasus $ euca-authorize -P icmp -t 8:-1 -s 0.0.0.0/0 pegasus $ euca-authorize -P tcp -p 22 -s 0.0.0.0/0 pegasus $ euca-authorize -P tcp -p 80 -s 0.0.0.0/0 pegasus $ euca-authorize -P udp -p 9614-9618 -s 0.0.0.0/0 pegasus $ euca-authorize -P tcp -p 9614-9618 -s 0.0.0.0/0 pegasus $ euca-authorize -P udp -p 49152-53247 -s 0.0.0.0/0 pegasus $ euca-authorize -P tcp -p 49152-53247 -s 0.0.0.0/0 pegasus $ euca-describe-groups pegasus GROUPvoecklerpegasus Pegasus tutorial VM PERMISSIONvoecklerpegasus ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0 PERMISSIONvoecklerpegasus ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0 PERMISSIONvoecklerpegasus ALLOWS tcp 9614 9618 FROM CIDR 0.0.0.0/0 PERMISSIONvoecklerpegasus ALLOWS tcp 49152 53247 FROM CIDR 0.0.0.0/0 PERMISSIONvoecklerpegasus ALLOWS icmp 1 -1 FROM CIDR 0.0.0.0/0 PERMISSIONvoecklerpegasus ALLOWS icmp 8 -1 FROM CIDR 0.0.0.0/0 PERMISSIONvoecklerpegasus ALLOWS udp 9614 9618 FROM CIDR 0.0.0.0/0 PERMISSIONvoecklerpegasus ALLOWS udp 49152 53247 FROM CIDR 0.0.0.0/0
The order of your output may differ.
Euca2ools, once installed, will work with both, Eucalyptus and
OpenStack. While the tutorials recommend to include the Euca or OpenStack
setup into your login files, I strongly advise against this. It is better
to keep the setup files separately, and to source it when you need to work
with any system. By keeping the IaaS setup files separate from your login
scripts, it is possible to work first with Eucalyptus on
india, then OpenStack on india,
and finally with Eucalyptus on sierra from the same
terminal, just by sourcing the eucarc file of the
cloud you want to work with.
The author tends to keep an setup file and his identity for
Eucalyptus on india in
$HOME/.euca/india/, the setup file and his identify
for Eucalyptus on sierra in
$HOME/.euca/sierra/ and the setup file and his
identity for OpenStack on india in
$HOME/.nova/. It is only necessary to source the
setup file once per session, assuming sub-sequent euca2ool command refer
to that cloud. The following prepares my terminal session to work with
Eucalyptus on india:
$ . $HOME/.euca/india/eucarc # yes, the command is the periodThe following prepares a terminal session to work with OpenStack on india instead:
$ . $HOME/.nova/india/novarc # yes, the command is the periodThe Pegasus tutorial image is called
pegasus-tutorial.x64.gz on the Eucalyptus clouds
sierra and india and the
OpenStack cloud india. All these images are
equivalent. You pick one cloud, start the image, and work against that
image.
Warning
The rest of the examples in this section show commands run against Eucalyptus. Unfortunately, the commands to be run against OpenStack subtly differ in some important details.
Using the FutureGrid Eucalyptus india cloud, you can query the public images like this:
$ euca-describe-images -o voeckler | sort -k3,3 ... IMAGEemi-31C41278pegasus/pegasus-tutorial.x64.manifest.xml voeckler available public x86_64 machineeki-78EF12D2 eri-5BB61255...
In this case, note the image identifier in the second column. Please
note that the identifier may change as the image gets updated. You need
the identifier to start a VM. To get your feet wet and start an image, you
need to decide which size (and thus, cost) of VM you are requesting.
Please refer to the Eucalyptus tutorial for details pertaining to the
image size. If you just want to see, if something runs, use
m1.small. If you plan to later save the image,
which is rather tedious and difficult in Eucalyptus, you should use at
least m1.large or even
m1.xlarge.
$ euca-run-instances -t m1.small -g pegasus -n 1 --addressing publicemi-31C41278RESERVATIONr-4AB00876voecklerpegasus INSTANCEi-556E08FCemi-31C412780.0.0.0 0.0.0.0 pending 0 m1.small2012-02-08T17:55:31.393Zindiafg eki-78EF12D2 eri-5BB61255
Unlike Nimbus, Eucalyptus and OpenStack use asynchronous
communication. The request above is placed with the remote cloud, and the
command returns. The command above requests a small instance. It starts
the VM with the firewall settings of pegasus activated.
Your default group will probably not have the proper
settings. Finally, we request a public IP address right away, because we
intend to connect to this machine from outside of FutureGrid. The last
argument is the image handle from the previous command.
You have to poll any instances you started, and wait for it to reach
running state. If you haven't installed the
HybridFox extension into your Firefox browser, the
watch command is a pretty decent polling tool:
$ watch euca-describe-instancesi-556E08FCRESERVATIONr-4AB00876voecklerpegasus INSTANCEi-556E08FC emi-31C41278149.165.159.12910.128.2.67pending 0 m1.small2012-02-08T17:55:31.393Zindiafg eki-78EF12D2 eri-5BB61255 ... RESERVATIONr-4AB00876voecklerpegasus INSTANCEi-556E08FC emi-31C41278149.165.159.12910.128.2.67running 0 m1.small2012-02-08T17:55:31.393Zindiafg eki-78EF12D2 eri-5BB61255
Once the instance has reached running state, you can attempt to log into it, using the public IP address from the output of euca-describe-instances.
Once the Image reached the running state, you
can log onto the image, i.e. contact it using the ssh client. The Pegasus
tutorial images are set up with a tutorial account that
is mapped to your public ssh key you provided to the FutureGrid portal.
The Pegasus tutorial works by using the tutorial
account. Please replace the IP address in the following command with the
one reported by the euca2ools:
Note
For Eucalyptus and OpenStack, you must tell ssh the identity with which you are login in. This identity was provided to you when you signed up for Eucalyptus according to the Eucalyptus tutorial.
$ ssh -i$HOME/.euca/india/jens.private-l tutorial149.165.159.129The authenticity of host '149.165.159.129 (149.165.159.129)' can't be established. RSA key fingerprint is 8a:a3:74:28:47:14:5c:76:ca:13:65:e9:71:2b:de:5b. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '149.165.159.129' (RSA) to the list of known hosts. /usr/bin/xauth: creating new authority file /home/tutorial/.Xauthority tutorial@fg-10-128-2-67.india.futuregrid.org:~ $
Note
In the output above, you may notice that ssh put the cryptographic hash of the remote host into a local file. Should you connect later to the same address, but a different virtual machine, ssh won't let you connect. In this case, you need to clean the offending address from your known hosts file.
The specific output, again, may differ in specifics in your case.
For instance, in the Windows case, you open the
putty.exe application, and type in the remote IP
address, as returned from the cloud client, into the host box. Once you
click connect, you will see a dialog asking you to
accept the host key, and then you will be logged in. At this time, you
should see a similar prompt to the one shown above.
Once you have logged into the remote virtual machine, you will be able run the exercises on that machine.
