11.6. pegasus-cwl-converter

pegasus-cwl-converter - converts a Common Workflow Language (CWL) workflow into a Pegasus workflow. Note that this utility is currently a work in progress and we are working towards providing more comprehensive support of the CWL specification.

pegasus-cwl-converter [-h] [-d]

11.6.1. Description

pegasus-cwl-converter is a command-line utility for converting CWL workflows into Pegasus workflows.

11.6.2. Additional Python Dependencies

Pegasus does not ship with the cwl-utils v0.11 and jsonschema v3.2.0 packages used by this utility, and must be installed prior: pip install pegasus-wms[cwl]

11.6.3. Positional Arguments


Path to the file containing the CWL workflow class.


YAML file specifying initial inputs to the CWL workflow.


YAML file specifying Pegasus specific transformation parameters. This file must be formatted as:

<executable name0>:
    site: <str>
    is_stageable: <bool>
<executable nameN>:
    site: <str>
    is_stageable: <bool>

Where each <executable name> is the name of the executable specified in the baseCommand field of a CWL CommandLineTool. Note that if baseCommand has something like /usr/bin/gcc, the corresponding executable name that you would place in this file is just gcc. The site field specifies the site on which this transformation resides (must be one of the sites specified in your Pegasus site catalog).


Desired path of the generated Pegasus workflow file.

11.6.4. Optional Arguments


Prints a concise help and exits.


Enables debug mode in the logger.

11.6.5. Example

Say that you have the following directory, my_cwl_workflow, that contains a main workflow CWL file, job CWL files, input specifications, input files, and scripts or executables used by the jobs:

├── tar.cwl            <-- CWL Job class that invokes /usr/bin/tar
├── compile_1.cwl      <-- CWL Job class that invokes /usr/bin/gcc
├── compile_2.cwl      <-- CWL Job class that invokes /usr/bin/gcc
├── get_file_sizes.cwl <-- CWL Job class that invokes ./get_file_sizes.sh
├── get_file_sizes.sh  <-- stageable script
├── src_tarball        <-- initial input file
├── input.yml          <-- workflow input specs
└── workflow.cwl       <-- CWL Workflow class that describes steps

In order to use the converter, you must create a transformation spec file. This file specifies Pegasus specific properties about each CWL CommandLineTool which are required for the conversion. In this example, the CommandLineTool files have the following baseCommand fields:











Assuming that your Pegasus site catalog has two sites local and condorpool, the transformation spec file would look something like:

    site: condorpool
    is_stageable: False
    site: condorpool
    is_stageable: False
    site: local
    is_stageable: True

Note that, gcc is only referenced once. Now that the transformation spec file has been created (let’s call this tr_specs.yml), pegasus-cwl-converter can be used to convert workflow.cwl into Pegasus’s native format by calling:

pegasus-cwl-converter workflow.cwl input.yml tr_specs.yml  pegasus_workflow.yml