org.griphyn.cPlanner.transfer.refiner
Class Bundle

java.lang.Object
  org.griphyn.cPlanner.transfer.AbstractRefiner
      org.griphyn.cPlanner.transfer.MultipleFTPerXFERJobRefiner
          org.griphyn.cPlanner.transfer.refiner.Default
              org.griphyn.cPlanner.transfer.refiner.Bundle

All Implemented Interfaces:

Refiner

public class Bundle
extends Default

An extension of the default refiner, that allows the user to specify the number of transfer nodes per execution site for stagein and stageout.

Version:

$Revision: 226 $

Author:

Karan Vahi

Nested Class Summary

private class	Bundle.PoolTransfer A container to store the transfers that need to be done on a single pool.
private class	Bundle.TransferContainer A container class for storing the name of the transfer job, the list of file transfers that the job is responsible for.

Field Summary

static String	DEFAULT_STAGE_IN_BUNDLE_FACTOR The default bundling factor that identifies the number of transfer jobs that are being created per execution pool for stageing in data for the workflow.
static String	DEFAULT_STAGE_OUT_BUNDLE_FACTOR The default bundling factor that identifies the number of transfer jobs that are being created per execution pool while stageing data out.
static String	DESCRIPTION A short description of the transfer refinement.
private int	mCurrentLevel The current level of the jobs being traversed.
private ReplicaCatalogBridge	mRCB The handle to the replica catalog bridge.
private Map	mRelationsMap The map indexed by compute jobnames that contains the list of stagin job names that are being added during the traversal of the workflow.
private Map	mSetupMap The map indexed by staged executable logical name.
private Map	mSIBundleMap The map containing the stage in bundle values indexed by the name of the pool.
private Map	mStageInMap The map containing the list of stage in transfer jobs that are being created for the workflow indexed by the execution poolname.
private Map	mStageOutMapPerLevel A map indexed by site name, that contains the pointer to the stage out PoolTransfer objects for that site.

Fields inherited from class org.griphyn.cPlanner.transfer.refiner.Default

mFileTable, mLogMsg, mPPS

Fields inherited from class org.griphyn.cPlanner.transfer.AbstractRefiner

mDAG, mLogger, mPOptions, mProps, mTPT, mTXInterImplementation, mTXStageInImplementation, mTXStageOutImplementation, mXMLStore

Fields inherited from interface org.griphyn.cPlanner.transfer.Refiner

INTER_POOL_PREFIX, REGISTER_PREFIX, STAGE_IN_PREFIX, STAGE_OUT_PREFIX

Fields inherited from interface org.griphyn.cPlanner.engine.Refiner

VERSION

Constructor Summary

Bundle(ADag dag, PegasusProperties properties, PlannerOptions options)
The overloaded constructor.

Method Summary

void	addStageInXFERNodes(SubInfo job, Collection files) Adds the stage in transfer nodes which transfer the input files for a job, from the location returned from the replica catalog to the job's execution pool.
void	addStageOutXFERNodes(SubInfo job, Collection files, ReplicaCatalogBridge rcb, boolean deletedLeaf) Adds the stageout transfer nodes, that stage data to an output site specified by the user.
void	done() Signals that the traversal of the workflow is done.
String	getDescription() Returns a textual description of the transfer mode.
protected int	getSISiteBundleValue(String site, String deflt) Determines the bundle factor for a particular site on the basis of the stage in bundle value associcated with the underlying transfer transformation in the transformation catalog.
protected int	getSOSiteBundleValue(String site, String deflt) Determines the bundle factor for a particular site on the basis of the stage out bundle value associcated with the underlying transfer transformation in the transformation catalog.
Bundle.PoolTransfer	getStageOutPoolTransfer(String site, int num) Returns the appropriate pool transfer for a particular site.
private void	resetStageOutMap() Resets the stage out map.

Methods inherited from class org.griphyn.cPlanner.transfer.refiner.Default

addInterSiteTXNodes, addJob, addRelation, addRelation, addStageOutXFERNodes, appendAttribute, constructFileKey, createRegistrationJob, logRefinerAction

Methods inherited from class org.griphyn.cPlanner.transfer.MultipleFTPerXFERJobRefiner

loadImplementations

Methods inherited from class org.griphyn.cPlanner.transfer.AbstractRefiner

getWorkflow, getXMLProducer, isSiteThirdParty, logConfigMessages, runTPTOnRemoteSite

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DESCRIPTION

public static final String DESCRIPTION

A short description of the transfer refinement.

See Also:

Constant Field Values

DEFAULT_STAGE_IN_BUNDLE_FACTOR

public static final String DEFAULT_STAGE_IN_BUNDLE_FACTOR

The default bundling factor that identifies the number of transfer jobs that are being created per execution pool for stageing in data for the workflow.

See Also:

Constant Field Values

DEFAULT_STAGE_OUT_BUNDLE_FACTOR

public static final String DEFAULT_STAGE_OUT_BUNDLE_FACTOR

The default bundling factor that identifies the number of transfer jobs that are being created per execution pool while stageing data out.

See Also:

Constant Field Values

mStageInMap

private Map mStageInMap

The map containing the list of stage in transfer jobs that are being created for the workflow indexed by the execution poolname.

mRelationsMap

private Map mRelationsMap

The map indexed by compute jobnames that contains the list of stagin job names that are being added during the traversal of the workflow. This is used to construct the relations that need to be added to workflow, once the traversal is done.

mSIBundleMap

private Map mSIBundleMap

The map containing the stage in bundle values indexed by the name of the pool. If the bundle value is not specified, then null is stored.

mSetupMap

private Map mSetupMap

The map indexed by staged executable logical name. Each entry is the name of the corresponding setup job, that changes the XBit on the staged file.

mStageOutMapPerLevel

private Map mStageOutMapPerLevel

A map indexed by site name, that contains the pointer to the stage out PoolTransfer objects for that site. This is per level of the workflow.

mCurrentLevel

private int mCurrentLevel

The current level of the jobs being traversed.

mRCB

private ReplicaCatalogBridge mRCB

The handle to the replica catalog bridge.

Constructor Detail

Bundle

public Bundle(ADag dag,
              PegasusProperties properties,
              PlannerOptions options)

The overloaded constructor.

Parameters:

dag - the workflow to which transfer nodes need to be added.

properties - the PegasusProperties object containing all the properties required by Pegasus.

options - the options passed to the planner.

Method Detail

addStageInXFERNodes

public void addStageInXFERNodes(SubInfo job,
                                Collection files)

Adds the stage in transfer nodes which transfer the input files for a job, from the location returned from the replica catalog to the job's execution pool.

Overrides:

addStageInXFERNodes in class Default

Parameters:

job - SubInfo object corresponding to the node to which the files are to be transferred to.

files - Collection of FileTransfer objects containing the information about source and destURL's.

addStageOutXFERNodes

public void addStageOutXFERNodes(SubInfo job,
                                 Collection files,
                                 ReplicaCatalogBridge rcb,
                                 boolean deletedLeaf)

Adds the stageout transfer nodes, that stage data to an output site specified by the user.

Overrides:

addStageOutXFERNodes in class Default

Parameters:

job - SubInfo object corresponding to the node to which the files are to be transferred to.

files - Collection of FileTransfer objects containing the information about source and destURL's.

rcb - bridge to the Replica Catalog. Used for creating registration nodes in the workflow.

deletedLeaf - to specify whether the node is being added for a deleted node by the reduction engine or not. default: false

done

public void done()

Signals that the traversal of the workflow is done. At this point the transfer nodes are actually constructed traversing through the transfer containers and the stdin of the transfer jobs written.

Overrides:

done in class Default

getDescription

public String getDescription()

Returns a textual description of the transfer mode.

Overrides:

getDescription in class Default

Returns:

a short textual description

getSISiteBundleValue

protected int getSISiteBundleValue(String site,
                                   String deflt)

Determines the bundle factor for a particular site on the basis of the stage in bundle value associcated with the underlying transfer transformation in the transformation catalog. If the key is not found, then the default value is returned. In case of the default value being null the global default is returned.

Parameters:

site - the site at which the value is desired.

deflt - the default value.

Returns:

the bundle factor.

See Also:

#DEFAULT_BUNDLE_STAGE_IN_FACTOR

getSOSiteBundleValue

protected int getSOSiteBundleValue(String site,
                                   String deflt)

Determines the bundle factor for a particular site on the basis of the stage out bundle value associcated with the underlying transfer transformation in the transformation catalog. If the key is not found, then the default value is returned. In case of the default value being null the global default is returned.

Parameters:

site - the site at which the value is desired.

deflt - the default value.

Returns:

the bundle factor.

See Also:

DEFAULT_STAGE_OUT_BUNDLE_FACTOR

getStageOutPoolTransfer

public Bundle.PoolTransfer getStageOutPoolTransfer(String site,
                                                   int num)

Returns the appropriate pool transfer for a particular site.

Parameters:

site - the site for which the PT is reqd.

num - the number of Stageout jobs required for that Pool.

Returns:

the PoolTransfer

resetStageOutMap

private void resetStageOutMap()

Resets the stage out map.