7.7. Remote Cluster using CREAMCE

CREAM is a webservices based job submission front end for remote compute clusters. It can be viewed as a replaced for Globus GRAM and is mainly popular in Europe. It widely used in the Italian Grid.

In order to submit a workflow to compute site using the CREAMCE front end, the user needs to specify the following for the site in their site catalog

  1. pegasus profile style with value set to cream

  2. grid gatewaydefined for the site with contact attribute set to CREAMCE frontend and scheduler attribute to remote scheduler.

  3. a remote queue can be optionally specified using globus profile queue with value set to queue-name

An example site catalog entry for a creamce site looks as follow in the site catalog

<sitecatalog xmlns="http://pegasus.isi.edu/schema/sitecatalog"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://pegasus.isi.edu/schema/sitecatalog http://pegasus.isi.edu/schema/sc-4.0.xsd"
             version="4.0">
      
    <site  handle="creamce" arch="x86" os="LINUX">
        <grid type="cream" contact="https://ce01-lcg.cr.cnaf.infn.it:8443/ce-cream/services/CREAM2" scheduler="LSF" jobtype="compute" />
        <grid type="cream" contact="https://ce01-lcg.cr.cnaf.infn.it:8443/ce-cream/services/CREAM2" scheduler="LSF" jobtype="auxillary" />

         <!-- Scratch directory on the cluster -->
        <directory type="shared-scratch" path="/home/virgo034">
            <file-server operation="all" url="gsiftp://ce01-lcg.cr.cnaf.infn.it/home/virgo034"/>
        </directory>

        <!-- cream is the style to use for CREAMCE submits -->                                                                                                                                                                                                                               
        <profile namespace="pegasus" key="style">cream</profile>

        <!-- the remote queue is picked up from globus profile --> 
        <profile namespace="globus" key="queue">virgo</profile>

        <!-- Staring HTCondor 8.0 additional cream attributes 
             can be passed by setting cream_attributes -->
        <profile namespace="condor" key="cream_attributes">key1=value1;key2=value2</profile>
    </site>

 </sitecatalog>

The pegasus distribution comes with creamce examples in the examples directory. They can be used as a starting point to configure your setup.

Tip

Usually , the CREAMCE frontends accept VOMS generated user proxies using the command voms-proxy-init . Steps on generating a VOMS proxy are listed in the CREAM User Guide here .