flowr: Streamline your workflows¶
The documentation is outdated, please use: docs.flowr.space
This framework allows you to design and implement complex pipelines, and deploy them on your institution’s computing cluster. This has been built keeping in mind the needs of bioinformatics workflows. However, it is easily extendable to any field where a series of steps (shell commands) are to be executed in a (work)flow.
Highlights¶
Effectively process a pipeline multi-step pipeline, spawning it across the computing cluster
- Example:
- A typical case with next-generation sequencing, a sample with tens of fastq files)
- Each file can be processed (aligned) individually, each using multiple cores
- Say 50 files using 10 cores each, totalling 500 cores across several machines, for one sample
- flowr further supports processing multiple samples in parrellel, spawning thousands of cores.
Reproducible, with cleanly structured execution logs
Track and re-run flows
Lean and Portable, with easy installation
Supports multiple platforms (torque, lsf, sge, slurm ...)
A few lines, to get started:¶
## From the official R repository (may be a few versions behind)
install.packages("flowr")
## OR
install.packages(devtools)
devtools::install_github("sahilseth/flowr")
library(flowr) ## load the library
setup() ## copy flowr bash script; and create a folder flowr under home.
run('sleep', execute=TRUE, platform='moab') ## submit a simple example
- Here is a shiny app, flow_creator which helps you build a flow.
- A few slides providing a quick overview.
Contents:¶
- Get started
- Ingredients for building a pipeline
- Submission types
- Dependency types
- Relationships
- Available Pipelines
- Cluster Support
- Example of building a pipeline
- FAQs
- Help on Available functions
Indices and tables¶
Aknowledgements¶
- Jianhua Zhang
- Samir Amin
- Kadir Akdemir
- Ethan Mao
- Henry Song
- An excellent resource for writing your own R packages: r-pkgs.had.co.nz