Hands-on: Importing workflows

• 10 min
• Easy

LabID supports importing workflows from various sources, including git repositories, WorkflowHub and Galaxy.
By Workflow here we include traditional ones from workflow platforms (Nextflow, Snakemake, Galaxy) but also scripts and notebooks, which can be equally registered in LabID.

'Manual' workflows with desktop applications

Workflows relying on desktop applications for which there is no dedicated files representing the workflow can also be registered in LabID, by creating a "blank" workflow version, simply documenting the version and name of the software used and ideally adding a short text file describing the steps to perform. This is however not ideal for reproducibility.

To import a workflow, from the left menu, navigate to Workflows > Workflow.
On the top left of the page, click on New workflow to see the different import options.

Workflows can be imported from various sources or created from scratch

Import from git

Workflows can be imported from a public (GitHub, GitLab...) or privately hosted (e.g institutional) git repo.
Here we will import a Nextflow workflow from nf-core, direcly from the URL of the workflow repository (https://github.com/nf-core/scrnaseq/).

Support for git institutional repository

To import workflows from an institutional repository, your admin needs to configure authentication via ssh.

1. From the list of options under New workflow (as shown in screencapture above), select Import from git.

2. Paste the Github URL above (or any GitHub, GitLab or other repository URL) in the Repository URL field.

3. Review the version selected in the dropdown menu Workflow version commit. The last tagged version is selected by default, feel free to select another commit or tag. Note that versions already imported in LabID are labeled as such.

4. Review the list of files to display with the workflow version. Add/remove files or change the default type as needed.

About the workflow files

LabID will list some "key" files from the repository, that are "representative" of the version (main workflow file, license, readme, config). This selection is based on the filename, and currently has for main purpose to provide a quick way to view the file content in LabID. There is thus no need to add every single file in the repo to this list, the created Workflow Version anyway retains a link to the repo and the commit used to import the workflow version.

Once all files of interest are listed, clicking Import version x.x.x will create a corresponding Workflow version in LabID.
A parent Workflow will also be created if this is the first version imported from this URL.

When importing a workflow from a git repository, the last tagged version is pre-selected. You can choose another version via the dropwdon "workflow version commit".

The parent Workflow keeps a reference to the original Git repository, while the associated Workflow versions have a reference to the original commit hash, as shown on the screenshots below.

Resulting workflow and associated workflow versions in LabID

Import from a workflow repository (ex : WorkflowHub, Galaxy...)

Workflows can also be imported from workflow repositories such as WorkflowHub (and dev.WorkflowHub) and Galaxy instances.
The repositories you can import from, need to be configured by an admin (see WorkflowHub Integration of the admin documentation).

To import from a workflow repository :

1. Go to the Workflows section of LabID, you should see the list of workflows currently available.
2. At the top left of the page, click New workflow , and select the corresponding Import from ... option.

Importing workflows from repositories

Missing 'Import from ...' ?

If this option is missing, your LabID instance is not configured to import from that repository, in this case an administrator needs to add it following these instructions.

Now, you can either search across the workflow repository using the built-in search functionality of LabID or paste the URL to a public workflow in that repository.

Only public workflows can be imported

You can only import workflows that are public within these repositories.
If you work on a institutional Galaxy instance, "public" here means visible to all the users of this instance (not to the outside world). Alternatively, you can keep your workflow private and import into LabID any galaxy invocation using this workflow : an associated workflow version will automatically be created.

Importing a workflow using its direct URL

1. Select the workflowhub.eu repository under Repository platform
2. Select the tab "Import from RO-Crate URL".
3. Directly paste the URL to the workflow version from workflowhub.eu. For example, https://workflowhub.eu/workflows/1325?version=3.
4. Click Import

Importing a workflow from WorkflowHub using a URL. LabID automatically adds a "/rocrate" to the workflow URL.

Upon successful import, a new Workflow and Workflow Version are created. Click on the button in the notification window informing about the successful import to be taken to the workflow page.

Both the Workflow and Workflow version retain the original link to the workflow repository as an annotation.

Workflow versions imported from WorkflowHub are packaged as Workflow RO-Crate. This RO-Crate is associated to the Workflow Version via the Attachments functionality, so you can always download the original crate for this workflow version.

Workflow versions imported from Workflow Hub retain the link to the workflow page and a copy of the imported Workflow RO-Crate

Supported URL format for WorkflowHub

When importing from a WorkflowHub URL, the main URL of the workflow page cannot be used directly for import (ex : workflowhub.eu/workflows/1325/). The reason is that the URL does not point to a specific version, rather to the last available version at the time the link is visited.
LabID enforces here the use of version-specific URL to ensure reliable identification of imported workflow versions (ex : workflowhub.eu/workflows/1325?version=3).
Such version-specific links are available under Version history at the bottom of the WorkflowHub page, simply copying the hyperlink under the version identifier.

Example workflow from WorkflowHub. Make sure to use the link to a specific version for import

Next, we'll see how to search workflow versions using specific keywords.

Searching for workflows with keywords

Besides using the URL to a workflow, LabID supports searching across a workflow repository using free text keywords.

1. Select the usegalaxy.eu repository under Repository platform
2. Select the tab "Search XXX"
3. Enter some keywords in the input field ; for example type in "segmentation" and validate with the
4. Below the workflow name, select the version you want to import (in case of multiple versions)
5. Click Import x.x on the right side of the workflow

A pop-up window should notify you that the workflow was correctly imported. Click on the link in the pop up to go to the Workflow page listing all imported versions.

Searching for workflows matching the keyword "segmentation" within the Galaxy EU instance

When importing from Galaxy, the workflow version has a copy of the main Galaxy file (.ga).

Workflow and workflow version after importing workflow from a Galaxy instance