SODAR App Development

This document provides information and resources on accessing and developing different apps used in the SODAR server. Some of these originate in the SODAR Core framework, while others are apps native to the SODAR server repository.

SODAR Core Apps

SODAR uses the following apps from SODAR Core:

For details on each app as well as the APIs potentially offered, see their respective documentation as linked above. Note that issues and changes to these apps need to be raised and implemented in the separate SODAR Core repository and introduced to SODAR via the django-sodar-core dependency.

General App Development Guidelines

This documentation assumes you are familiar in the way Django apps are set up. General conventions apply unless otherwise noted.

SODAR apps define plugins. In addition to the typical Django app setup, apps with plugins need to be registered into the database before they can be run on your development site. This can be done with the following command:

$ ./manage.py syncplugins

The command is also run automatically together with ./manage.py migrate.

Samplesheets

In addition to the typical SODAR Core based Django app, the samplesheets app consists of multiple subsystems. This section is split into multiple subsections detailing each subsystem.

Django App

In addition to the standard SODAR Core app components, the samplesheets Django app contains the following modules:

constants: Application constants are stored here.
io: Sample sheet importing and exporting between ISA-Tab TSV files and a Django database model. Uses the altamISA parser.
rendering: Sample sheet rendering from the graph-based Django database model into tables, for UI rendering and mapping iRODS collections to table rows.
sheet_config: Management of sample sheet editing and display configurations.
tasks_celery: Celery tasks for asynchronous actions.

The Django app provides the standard framework for views and backend functionality. The app also provides the Ajax API views required for the Vue.js app to retrieve data from the server side.

Except for the sample sheet viewer and editor implemented as a Vue.js app, it is recommended to develop views in the app as server-side Django views.

Vue.js App

The Vue.js app is located under samplesheets/vueapp. It is used as the sample sheet viewer and editor. The app is embedded into the Django app view ProjectSheetsView and the associated project_sheets.html template using django-webpack-loader.

The app uses ag-grid for table rendering and interactivity. It is recommended to get familiarized with ag-grid before developing this app.

The app is set up with strict linting, which should make following code conventions straightfoward.

At the moment, the app is implemented on Vue v2. Upgrading to Vue v3 will be considered in the future.

Study Sub-Apps

Study sub-apps are stored under samplesheets/studyapps. They are used to dynamically introduce study table shortcuts for specific sample sheet configurations.

The following study sub-apps currently exist:

cancer: Study sub-app for cancer studies. Organizes shortcuts by case.
germline: Study sub-app for germline studies. Organizes shortcuts by pedigree.

These apps generally consist of a plugin implementing certain methods, but can also include e.g. views to be linked from the UI elements generated by the plugin.

To create a new study sub-app, you should first start a Django app under samplesheets/studyapps. In plugins.py, you need to implement a plugin based on samplesheets.plugins.SampleSheetStudyPluginPoint.

Member variables:

name: ID of the sub-app plugin. It is recommended to prefix this with samplesheets_study_*, so your plugin’s name should be in the form of samplesheets_study_yourapp.
title: Display title for your sub-app. Used in the UI.
config_name: Sample sheet configuration name. Must correspond to the expected name used in the ISA-Tab investigation file comment labeled Created with Configuration.
description: Verbose description for the sub-app.
permission: Required permission to view data with this app (optional).

Methods:

get_shortcut_column(): Return a dictionary containing shortcuts for an extra study table links column in a sample sheet study table. These shortcuts can either be direct links or open a modal with parameters for retrieving data into the modal.
get_shortcut_links(): Return links for a shortcut modal opened by the links column.
update_cache(): Update the sodarcache entry for the app related to a specific cache item or project, or for all projects.

An example of expected output for get_shortcut_column():

{
    'schema': {
        'your_link': {
            'type': 'link',
            'icon': 'mdi:info',
            'title': 'Printable title for link',
        }
        'your_modal': {
            'type': 'modal',
            'icon': 'mdi:info',
            'title': 'Printable title for modal opening',
        }
    },
    'data': [  # Data for the entire table, array item per table row
        'your_link': {
            'url': some_url,
            'enabled': True,
        },
        'your_modal': {
            'query': {
                'key': some_parameter,
                'value': some_value_like_source_id,
            },
            'enabled': True
        }
    ]
}

An example of expected output for get_shortcut_links():

{
    'title': 'Title for shortcut modal',
    'data': {
        'item': {
            'title': 'Printable title for item type',
            'files': [
                {
                    'label': 'Link label (e.g. related extract name)',
                    'url': some_url,
                    'title': 'Mouseover title'
                    'extra_links': {  # Additional links for the same item
                        'label': 'Mouseover label',
                        'icon': 'mdi:info',  # Icon for link button
                        'url': some_extra_url
                    }
                }
            ]
        }
    }
}

The use of each study sub-app in sample sheets should be documented to the user in metadata_advanced.

Assay Sub-Apps

Similar to study sub-apps, assay sub-apps provide a dynamic way to link content within assays based on the assay type. They are placed under samplesheets/assayapps.

The following assay sub-apps currently exist:

cytof: Protein expression profiling / mass cytometry assay app.
dna_sequencing: DNA sequencing assay app.
generic_raw: Generic assay app providing a top level “raw data” collection under each assay.
meta_ms: Metabolite profiling / mass spectrometry assay app.
microarray: Microarray assay app.
pep_ms: Protein expression profiling / mass spectrometry assay app.

These apps consist of a plugin implementing certain methods. To create a new assay sub-app, you should first start a Django app under samplesheets/assayapps. In plugins.py, you need to implement a plugin based on samplesheets.plugins.SampleSheetAssayPluginPoint.

Member variables:

name: ID of the sub-app plugin. It is recommended to prefix this with samplesheets_assay_*, so your plugin’s name should be in the form of samplesheets_assay_yourapp.
title: Display title for your sub-app. Used in the UI.
app_name: App name for dynamic reference to app in e.g. caching. Should be defined in the module.
assay_fields: Identifying assay fields. These link the sub-app to a specific assay. It is given as a list of dicts each containing two keys, measurement_type and technology_type. These are checked against the corresponding paramters in the Assay model and the first successful hit returns true.
description: Verbose description for the sub-app.
permission: Required permission to view data with this app (optional).
display_row_links: Toggle displaying row-based iRODS links in the assay table, if relevant to this assay type.

Methods:

get_row_path(): Return iRODS path for a specific assay table row.
update_row(): Update table row with e.g. links.
get_shortcuts(): Return assay level iRODS shortcuts.
update_cache(): Update the sodarcache entry for the app related to a specific cache item or project, or for all projects.

The use of each assay sub-app in sample sheets should be documented to the user in metadata_advanced.

Landingzones

The landingzones app is used to manage landing zones in iRODS related to the project sample sheets.

Because this app relies heavily on data created by the samplesheets app, hard-coded imports from samplesheets are allowed as opposed to usual SODAR Core conventions.

Configuration Sub-Apps

It is possible to define specific configuration sub-apps for special cases of landing zones where extra functionality is required. These are implemented in a style similar to the study and assay sub-apps in samplesheets.

To create a new configuration sub-app, you should first start a Django app under landingzones/configapps. In plugins.py, you need to implement a plugin based on landingzones.plugins.LandingZoneConfigPluginPoint.

Member variables:

name: ID of the sub-app plugin. It is recommended to prefix this with landingzones_config_*, so your plugin’s name should be in the form of landingzones_config_yourapp.
title: Display title for your sub-app. Used in the UI.
config_name: Configuration name. Used to identify plugin by configuration string.
config_display_name: Configuration name to be displayed in the UI.
description: Verbose description for the sub-app.
menu_items: Additional items for the landing zone dropdown menu.
api_config_data: Fields from LandingZone.config_data to be displayed in zone list API.
permission: Required permission to view data with this app (optional).

Methods:

cleanup_zone(): Perform custom actions before landing zone deletion (optional).
get_extra_flow_data(): Return custom parameters for taskflowbackend (optional).

Irodsadmin

The irodsadmin app provides helper management commands for iRODS data. At this moment, it contains the irodsorphans command. It will search for orphaned iRODS collections in both project sample data repositories and landing zones. It returns a list of collections not tied into any project as recognized by SODAR.

Irodsbackend

The irodsbackend app provides an API for iRODS connections and common iRODS helpers. If enabled, it can be included in your application as follows:

from projectroles.plugins import get_backend_api
irods_backend = get_backend_api('omics_irods')

The API retrieves iRODS connection parameters from the Django configuration. It uses an admin account to perform actions on the iRODS server. The IRODS_ENV_BACKEND setting can be used to override environment values for these backend connections.

To create and access the iRODS session, it is recommended to do it using the backend API using get_session() via a context manager:

with irods_backend.get_session() as irods:
    pass  # Your session code here

Alternatively, you can get the object directly with get_session_obj(). This requires you to call cleanup() manually and it is mostly recommended for testing.

irods = irods_backend.get_session_obj()
pass  # Your session code here
irods.cleanup()

Irodsinfo

The irodsinfo app simply displays iRODS server information and is used to generate a client configuration for the user.

Ontologyaccess

The ontologyaccess app is used for parsing .obo and .owl format ontologies, which are then queried by the samplesheets app in its UI. The app provides both a site app and a backend app plugin for managing local ontologies and providing an API for ontology list, respectively. Ajax API views for ontology queries are also provided.

Import operations can be found in the io module. Note that for parsing the OMIM catalog is done using a separate import_omim() method to convert it into a dummy OBO-compatible format.

Taskflowbackend

This backend application handles project data specific transactions in the iRODS data management system, with rollback capability on errors. It is based on the formerly separate SODAR Taskflow repository. The Taskflow backend is invoked as follows:

taskflow = get_backend_api('taskflowbackend')

Jobs are submitted using the submit() method similar to the following example:

taskflow.submit(
    project=project,
    flow_name='sheet_colls_create',
    flow_data=flow_data,
)

Taskflowbackend uses the project modify API introduced in SODAR Core v0.11 to set up project data in iRODS according to project and role assignment changes.

The syncmodifyapi management command can be used to sync all projects into iRODS. This should only be used in development when there is need to e.g. recreate a local iRODS database.