org.griphyn.cPlanner.toolkit
Class CPlanner

java.lang.Object
  org.griphyn.cPlanner.toolkit.Executable
      org.griphyn.cPlanner.toolkit.CPlanner

public class CPlanner
extends Executable

This is the main program for the Pegasus. It parses the options specified by the user and calls out to the appropriate components to parse the abstract plan, concretize it and then write the submit files.

Version:

$Revision: 464 $

Author:

Gaurang Mehta, Karan Vahi

Field Summary

static String	CLEANUP_DIR The basename of the directory that contains the submit files for the cleanup DAG that for the concrete dag generated for the workflow.
static String	DEFAULT_MEGADAG_MODE The default megadag mode that is used for generation of megadags in deferred planning.
private static String	EMPTY_FINAL_WORKFLOW_MESSAGE The final successful message that is to be logged.
private NumberFormat	mNumFormatter The number formatter to format the run submit dir entries.
private PlannerMetrics	mPMetrics The PlannerMetrics object storing the metrics about this planning instance.
private PlannerOptions	mPOptions The object containing all the options passed to the Concrete Planner.
private String	mUser The user name of the user running Pegasus.
static String	SUBMIT_DIRECTORY_PREFIX The prefix for the submit directory.
private static String	SUCCESS_MESSAGE The message to be logged in case of empty executable workflow.

Fields inherited from class org.griphyn.cPlanner.toolkit.Executable

mLogger, mLogMsg, mProps, mVersion

Constructor Summary

CPlanner()
Default constructor.

Method Summary

protected String	createSubmitDirectory(ADag dag, String dir, String user, String vogroup, boolean timestampBased) Creates the submit directory for the workflow.
protected String	createSubmitDirectory(String label, String dir, String user, String vogroup, boolean timestampBased) Creates the submit directory for the workflow.
private void	doDeferredPlanning() This ends up invoking the deferred planning code, that generates the MegaDAG that is used to submit the partitioned daxes in layers.
protected void	doPartitionAndPlan(PegasusProperties properties, PlannerOptions options) Partitions and plans the workflow.
void	executeCommand(PlannerOptions options) Executes the command on the basis of the options specified.
void	executeCommand(String[] args) Executes the command on the basis of the options specified.
gnu.getopt.LongOpt[]	generateValidOptions() Tt generates the LongOpt which contain the valid options that the command will accept.
private String	getPegasusRunInvocation(boolean nodatabase) Returns the pegasus-run invocation on the workflow planned.
protected String	getRandomDirectory(ADag dag) Sets the basename of the random directory that is created on the remote sites per workflow.
void	loadProperties() Loads all the properties that are needed by this class.
protected boolean	logEntryInWorkCatalog(ADag dag, String baseDir, String relativeDir) Logs an entry in the work catalog about the workflow being planned.
private void	logSuccessfulCompletion(boolean nodatabase) Logs the successful completion message.
static void	main(String[] args) The main program for the CPlanner.
PlannerOptions	parseCommandLineArguments(String[] args) Parses the command line arguments using GetOpt and returns a PlannerOptions contains all the options passed by the user at the command line.
void	printLongVersion() Prints the long description, displaying in detail what the various options to the command stand for.
void	printShortVersion() Prints out a short description of what the command does.
protected static void	sanityCheck(File dir) Checks the destination location for existence, if it can be created, if it is writable etc.
boolean	submitWorkflow(String invocation) Submits the workflow for execution using pegasus-run, a wrapper around pegasus-submit-dag.
protected boolean	writeOutMetrics(PlannerMetrics pm) Writes out the planner metrics to the global log.

Methods inherited from class org.griphyn.cPlanner.toolkit.Executable

convertException, getEnvValue, getGVDSVersion, log, setupLogging

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_MEGADAG_MODE

public static final String DEFAULT_MEGADAG_MODE

The default megadag mode that is used for generation of megadags in deferred planning.

See Also:

Constant Field Values

SUBMIT_DIRECTORY_PREFIX

public static final String SUBMIT_DIRECTORY_PREFIX

The prefix for the submit directory.

See Also:

Constant Field Values

CLEANUP_DIR

public static final String CLEANUP_DIR

The basename of the directory that contains the submit files for the cleanup DAG that for the concrete dag generated for the workflow.

See Also:

Constant Field Values

EMPTY_FINAL_WORKFLOW_MESSAGE

private static final String EMPTY_FINAL_WORKFLOW_MESSAGE

The final successful message that is to be logged.

See Also:

Constant Field Values

SUCCESS_MESSAGE

private static final String SUCCESS_MESSAGE

The message to be logged in case of empty executable workflow.

See Also:

Constant Field Values

mPOptions

private PlannerOptions mPOptions

The object containing all the options passed to the Concrete Planner.

mPMetrics

private PlannerMetrics mPMetrics

The PlannerMetrics object storing the metrics about this planning instance.

mNumFormatter

private NumberFormat mNumFormatter

The number formatter to format the run submit dir entries.

mUser

private String mUser

The user name of the user running Pegasus.

Constructor Detail

CPlanner

public CPlanner()

Default constructor.

Method Detail

main

public static void main(String[] args)

The main program for the CPlanner.

Parameters:

args - the main arguments passed to the planner.

loadProperties

public void loadProperties()

Loads all the properties that are needed by this class.

Specified by:

loadProperties in class Executable

executeCommand

public void executeCommand(String[] args)

Executes the command on the basis of the options specified.

Specified by:

executeCommand in class Executable

Parameters:

args - the command line options.

executeCommand

public void executeCommand(PlannerOptions options)

Executes the command on the basis of the options specified.

Parameters:

options - the command line options.

parseCommandLineArguments

public PlannerOptions parseCommandLineArguments(String[] args)

Parses the command line arguments using GetOpt and returns a PlannerOptions contains all the options passed by the user at the command line.

Parameters:

args - the arguments passed by the user at command line.

Returns:

the options.

logEntryInWorkCatalog

protected boolean logEntryInWorkCatalog(ADag dag,
                                        String baseDir,
                                        String relativeDir)

Logs an entry in the work catalog about the workflow being planned.

Parameters:

dag - ADag

baseDir - String

relativeDir - String

Returns:

boolean

submitWorkflow

public boolean submitWorkflow(String invocation)

Submits the workflow for execution using pegasus-run, a wrapper around pegasus-submit-dag.

Parameters:

invocation - the pegasus run invocation

Returns:

boolean indicating whether could successfully submit the workflow or not.

doPartitionAndPlan

protected void doPartitionAndPlan(PegasusProperties properties,
                                  PlannerOptions options)

Partitions and plans the workflow. First step of merging DAGMan and Condor

Parameters:

properties - the properties passed to the planner.

options - the options passed to the planner.

getRandomDirectory

protected String getRandomDirectory(ADag dag)

Sets the basename of the random directory that is created on the remote sites per workflow. The name is generated by default from teh flow ID, unless a basename prefix is specifed at runtime in the planner options.

Parameters:

dag - the DAG containing the abstract workflow.

Returns:

the basename of the random directory.

generateValidOptions

public gnu.getopt.LongOpt[] generateValidOptions()

Tt generates the LongOpt which contain the valid options that the command will accept.

Specified by:

generateValidOptions in class Executable

Returns:

array of LongOpt objects , corresponding to the valid options

printShortVersion

public void printShortVersion()

Prints out a short description of what the command does.

Specified by:

printShortVersion in class Executable

printLongVersion

public void printLongVersion()

Prints the long description, displaying in detail what the various options to the command stand for.

Specified by:

printLongVersion in class Executable

doDeferredPlanning

private void doDeferredPlanning()

This ends up invoking the deferred planning code, that generates the MegaDAG that is used to submit the partitioned daxes in layers.

createSubmitDirectory

protected String createSubmitDirectory(ADag dag,
                                       String dir,
                                       String user,
                                       String vogroup,
                                       boolean timestampBased)
                                throws IOException

Creates the submit directory for the workflow. This is not thread safe.

Parameters:

dag - the workflow being worked upon.

dir - the base directory specified by the user.

user - the username of the user.

vogroup - the vogroup to which the user belongs to.

timestampBased - boolean indicating whether to have a timestamp based dir or not

Returns:

the directory name created relative to the base directory passed as input.

Throws:

IOException - in case of unable to create submit directory.

createSubmitDirectory

protected String createSubmitDirectory(String label,
                                       String dir,
                                       String user,
                                       String vogroup,
                                       boolean timestampBased)
                                throws IOException

Creates the submit directory for the workflow. This is not thread safe.

Parameters:

label - the label of the workflow

dir - the base directory specified by the user.

user - the username of the user.

vogroup - the vogroup to which the user belongs to.

timestampBased - boolean indicating whether to have a timestamp based dir or not

Returns:

the directory name created relative to the base directory passed as input.

Throws:

IOException - in case of unable to create submit directory.

sanityCheck

protected static void sanityCheck(File dir)
                           throws IOException

Checks the destination location for existence, if it can be created, if it is writable etc.

Parameters:

dir - is the new base directory to optionally create.

Throws:

IOException - in case of error while writing out files.

writeOutMetrics

protected boolean writeOutMetrics(PlannerMetrics pm)

Writes out the planner metrics to the global log.

Parameters:

pm - the metrics to be written out.

Returns:

boolean

logSuccessfulCompletion

private void logSuccessfulCompletion(boolean nodatabase)

Logs the successful completion message.

Parameters:

nodatabase - boolean indicating whether to add a nodatabase option or not.

getPegasusRunInvocation

private String getPegasusRunInvocation(boolean nodatabase)

Returns the pegasus-run invocation on the workflow planned.

Parameters:

nodatabase - boolean indicating whether to add a nodatabase option or not.

Returns:

the pegasus-run invocation