Saving workflows

To save a workflow call the WorkflowGenerator.save() method:

wf.save('workflow.cwl')

By default, the paths in the run field of workflow steps are absolute. This means that a workflow created on one machine cannot be run on another machine. However, there are multiple options for creating portable workflows.

Saving workflows with relative paths

To get relative paths in the run field of workflow steps, use mode='rel':

wf.save('workflow.cwl', mode='rel')

The paths in the run field are relative to where the workflow is saved. This option is convenient when you are creating workflows using a single directory with possible workflow steps.

Using a working directory

If you have multiple directories containing workflow steps and the locations of these directories may differ depending on where software is installed (for example, if you want to use the generic NLP steps from nlppln, but also need project specific data processing steps), it is possible to specify a working directory when creating the WorkflowGenerator object. If you this, all steps are copied to the working directory. When you save the workflow using mode='wd', the paths in the run fields are set to the basename of the step (because all steps are in the same directory).

from scriptcwl import WorkflowGenerator

with WorkflowGenerator(working_dir='path/to/working_dir') as wf:
  wf.load(steps_dir='some/path/')
  wf.load(steps_dir='some/other/path/')

  # add inputs, steps and outputs

  wf.save('workflow', mode='wd')

The workflow is saved in the working directory and then copied to the specified location. To be able to run the workflow, use the copy in the working directory (please note that the working directory is not deleted automatically).

Also, steps from urls are not copied to the working directory.

Pack workflows

Another way to create workflows with all steps in one file is to save it with mode='pack':

wf.save('workflow.cwl', mode='pack')

Please note that packed workflows cannot be used as a building block in scriptcwl. If you try to load a packed workflow, you will get a warning.

Saved With mode='pack', the example workflow looks like:

{
    "cwlVersion": "v1.0",
    "$graph": [
        {
            "class": "CommandLineTool",
            "baseCommand": [
                "python",
                "-m",
                "scriptcwl.examples.add"
            ],
            "inputs": [
                {
                    "type": "int",
                    "inputBinding": {
                        "position": 1
                    },
                    "id": "#add.cwl/x"
                },
                {
                    "type": "int",
                    "inputBinding": {
                        "position": 2
                    },
                    "id": "#add.cwl/y"
                }
            ],
            "stdout": "cwl.output.json",
            "outputs": [
                {
                    "type": "int",
                    "id": "#add.cwl/answer"
                }
            ],
            "id": "#add.cwl"
        },
        {
            "class": "CommandLineTool",
            "baseCommand": [
                "python",
                "-m",
                "scriptcwl.examples.multiply"
            ],
            "inputs": [
                {
                    "type": "int",
                    "inputBinding": {
                        "position": 1
                    },
                    "id": "#multiply.cwl/x"
                },
                {
                    "type": "int",
                    "inputBinding": {
                        "position": 2
                    },
                    "id": "#multiply.cwl/y"
                }
            ],
            "stdout": "cwl.output.json",
            "outputs": [
                {
                    "type": "int",
                    "id": "#multiply.cwl/answer"
                }
            ],
            "id": "#multiply.cwl"
        },
        {
            "class": "Workflow",
            "inputs": [
                {
                    "type": "int",
                    "id": "#main/num1"
                },
                {
                    "type": "int",
                    "id": "#main/num2"
                }
            ],
            "outputs": [
                {
                    "type": "int",
                    "outputSource": "#main/multiply-1/answer",
                    "id": "#main/final_answer"
                }
            ],
            "steps": [
                {
                    "run": "#add.cwl",
                    "in": [
                        {
                            "source": "#main/num1",
                            "id": "#main/add-1/x"
                        },
                        {
                            "source": "#main/num2",
                            "id": "#main/add-1/y"
                        }
                    ],
                    "out": [
                        "#main/add-1/answer"
                    ],
                    "id": "#main/add-1"
                },
                {
                    "run": "#multiply.cwl",
                    "in": [
                        {
                            "source": "#main/add-1/answer",
                            "id": "#main/multiply-1/x"
                        },
                        {
                            "source": "#main/num2",
                            "id": "#main/multiply-1/y"
                        }
                    ],
                    "out": [
                        "#main/multiply-1/answer"
                    ],
                    "id": "#main/multiply-1"
                }
            ],
            "id": "#main"
        }
    ]
}

Workflow validation

Before the workflow is saved, it is validated using cwltool. Validation can also be triggered manually:

wf.validate()

It is also possible to disable workflow validation on save:

wf.save('workflow.cwl', validate=False)

File encoding

By default, the encoding used to save workflows is utf-8. If necessary, a different encoding can be specified:

wf.save('workflow.cwl', encoding='utf-16')