Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
20,066
result(s) for
"Workflow software"
Sort by:
Introducing Microsoft Flow : automating workflows between apps and services
\"Use Microsoft Flow in your business to improve productivity through automation with this step-by-step introductory text ... You'll see the prerequisites to get started with this cloud-based service, including how to create a flow and how to use different connectors. [It] takes you through connecting with SharePoint, creating approval flows, and using mobile apps. ... The second half of the book continues with managing connections and gateways, where you'll cover the configuration, creation,, and deletion of connectors and how to connect to a data gateway. The final topic is Flow administration and techniques to manage the environment.\"--Back cover.
Book
Sustainable data analysis with Snakemake version 2; peer review: 2 approved
Data analysis often entails a multitude of heterogeneous steps, from the application of various command line tools to the usage of scripting languages like R or Python for the generation of plots and tables. It is widely recognized that data analyses should ideally be conducted in a reproducible way. Reproducibility enables technical validation and regeneration of results on the original or even new data. However, reproducibility alone is by no means sufficient to deliver an analysis that is of lasting impact (i.e., sustainable) for the field, or even just one research group. We postulate that it is equally important to ensure adaptability and transparency. The former describes the ability to modify the analysis to answer extended or slightly different research questions. The latter describes the ability to understand the analysis in order to judge whether it is not only technically, but methodologically valid.
Here, we analyze the properties needed for a data analysis to become reproducible, adaptable, and transparent. We show how the popular workflow management system Snakemake can be used to guarantee this, and how it enables an ergonomic, combined, unified representation of all steps involved in data analysis, ranging from raw data processing, to quality control and fine-grained, interactive exploration and plotting of final results.
Journal Article
Preventing Temporal Violations in Scientific Workflows: Where and How
Due to the dynamic nature of the underlying high-performance infrastructures for scientific workflows such as grid and cloud computing, failures of timely completion of important scientific activities, namely, temporal violations, often take place. Unlike conventional exception handling on functional failures, nonfunctional QoS failures such as temporal violations cannot be passively recovered. They need to be proactively prevented through dynamically monitoring and adjusting the temporal consistency states of scientific workflows at runtime. However, current research on workflow temporal verification mainly focuses on runtime monitoring, while the adjusting strategy for temporal consistency states, namely, temporal adjustment, has so far not been thoroughly investigated. For this issue, two fundamental problems of temporal adjustment, namely, where and how, are systematically analyzed and addressed in this paper. Specifically, a novel minimum probability time redundancy-based necessary and sufficient adjustment point selection strategy is proposed to address the problem of where and an innovative genetic-algorithm-based effective and efficient local rescheduling strategy is proposed to tackle the problem of how. The results of large-scale simulation experiments with generic workflows and specific real-world applications demonstrate that our temporal adjustment strategy can remarkably prevent the violations of both local and global temporal constraints in scientific workflows.
Journal Article
The JEDI event-based infrastructure and its application to the development of the OPSS WFMS
The development of complex distributed systems demands the creation of suitable architectural styles (or paradigms) and related runtime infrastructures. An emerging style that is receiving increasing attention is based on the notion of event. In an event-based architecture, distributed software components interact by generating and consuming events. An event is the occurrence of some state change in a component of a software system, made visible to the external world. The occurrence of an event in a component is asynchronously notified to any other component that has declared some interest in it. This paradigm (usually called \"publish/subscribe\", from the names of the two basic operations that regulate the communication) holds the promise of supporting a flexible and effective interaction among highly reconfigurable, distributed software components. In the past two years, we have developed an object-oriented infrastructure called JEDI (Java event-based distributed infrastructure). JEDI supports the development and operation of event-based systems and has been used to implement a significant example of distributed system, namely, the OPSS workflow management system (WFMS). The paper illustrates the main features of JEDI and how we have used them to implement OPSS. Moreover, the paper provides an initial evaluation of our experiences in using the event-based architectural style and a classification of some of the event-based infrastructures presented in the literature.
Journal Article
Cytoscape Automation: empowering workflow-based network analysis
Cytoscape is one of the most successful network biology analysis and visualization tools, but because of its interactive nature, its role in creating reproducible, scalable, and novel workflows has been limited. We describe Cytoscape Automation (CA), which marries Cytoscape to highly productive workflow systems, for example, Python/R in Jupyter/RStudio. We expose over 270 Cytoscape core functions and 34 Cytoscape apps as REST-callable functions with standardized JSON interfaces backed by Swagger documentation. Independent projects to create and publish Python/R native CA interface libraries have reached an advanced stage, and a number of automation workflows are already published.
Journal Article
A Software Testing Workflow Analysis Tool Based on the ADCV Method
Based on two progressive aspects of the modeling problems in business process management (BPM), (1) in order to address the increasing complexity of user requirements on workflows underlying various BPM application scenarios, a more verifiable fundamental modeling method must be invented; (2) to address the diversification of software testing processes, more formalized advanced modeling technology must also be applied based on the fundamental modeling method. Aiming to address these modeling problems, this paper first proposes an ADCV (acquisition, decomposition, combination, and verification) method that runs through the core management links of four types of business processes (mining, decomposition, recombination, and verification) and then describes the compositional structure of the ADCV method and the design of corresponding algorithms. Then, the software testing workflow is managed and monitored using the method, and the corresponding analysis tool is implemented based on Petri nets. At the same time, the tool is applied to the case processing of the software testing workflow. Specifically, the workflow models are established successively through ADCV during the process of business iteration. Then, the analysis tool developed with the ADCV method, the model–view–controller (MVC) design pattern, and Java Swing technology are applied to instances of the software testing workflow to realize the modeling and management of the testing processes. Thus, the analysis tool can guarantee the accuracy of the parameter estimations of related software reliability growth models (SRGMs) and ultimately improve the quality of software products.
Journal Article
fMRIPrep: a robust preprocessing pipeline for functional MRI
Preprocessing of functional magnetic resonance imaging (fMRI) involves numerous steps to clean and standardize the data before statistical analysis. Generally, researchers create ad hoc preprocessing workflows for each dataset, building upon a large inventory of available tools. The complexity of these workflows has snowballed with rapid advances in acquisition and processing. We introduce fMRIPrep, an analysis-agnostic tool that addresses the challenge of robust and reproducible preprocessing for fMRI data. fMRIPrep automatically adapts a best-in-breed workflow to the idiosyncrasies of virtually any dataset, ensuring high-quality preprocessing without manual intervention. By introducing visual assessment checkpoints into an iterative integration framework for software testing, we show that fMRIPrep robustly produces high-quality results on a diverse fMRI data collection. Additionally, fMRIPrep introduces less uncontrolled spatial smoothness than observed with commonly used preprocessing tools. fMRIPrep equips neuroscientists with an easy-to-use and transparent preprocessing workflow, which can help ensure the validity of inference and the interpretability of results.
Journal Article
The Perseus computational platform for comprehensive analysis of (prote)omics data
Perseus is a comprehensive, user-friendly software platform for the biological analysis of quantitative proteomics data. It is intended to help biologists with little bioinformatics training to interpret protein expression, post-translational modification and interaction data. Also in this issue, see the Perspective by Röst
et al
.
A main bottleneck in proteomics is the downstream biological analysis of highly multivariate quantitative protein abundance data generated using mass-spectrometry-based analysis. We developed the Perseus software platform (
http://www.perseus-framework.org
) to support biological and biomedical researchers in interpreting protein quantification, interaction and post-translational modification data. Perseus contains a comprehensive portfolio of statistical tools for high-dimensional omics data analysis covering normalization, pattern recognition, time-series analysis, cross-omics comparisons and multiple-hypothesis testing. A machine learning module supports the classification and validation of patient groups for diagnosis and prognosis, and it also detects predictive protein signatures. Central to Perseus is a user-friendly, interactive workflow environment that provides complete documentation of computational methods used in a publication. All activities in Perseus are realized as plugins, and users can extend the software by programming their own, which can be shared through a plugin store. We anticipate that Perseus's arsenal of algorithms and its intuitive usability will empower interdisciplinary analysis of complex large data sets.
Journal Article
ATLAS: a Snakemake workflow for assembly, annotation, and genomic binning of metagenome sequence data
Background
Metagenomics studies provide valuable insight into the composition and function of microbial populations from diverse environments; however, the data processing pipelines that rely on mapping reads to gene catalogs or genome databases for cultured strains yield results that underrepresent the genes and functional potential of uncultured microbes. Recent improvements in sequence assembly methods have eased the reliance on genome databases, thereby allowing the recovery of genomes from uncultured microbes. However, configuring these tools, linking them with advanced binning and annotation tools, and maintaining provenance of the processing continues to be challenging for researchers.
Results
Here we present ATLAS, a software package for customizable data processing from raw sequence reads to functional and taxonomic annotations using state-of-the-art tools to assemble, annotate, quantify, and bin metagenome data. Abundance estimates at genome resolution are provided for each sample in a dataset. ATLAS is written in Python and the workflow implemented in Snakemake; it operates in a Linux environment, and is compatible with Python 3.5+ and Anaconda 3+ versions. The source code for ATLAS is freely available, distributed under a BSD-3 license.
Conclusions
ATLAS provides a user-friendly, modular and customizable Snakemake workflow for metagenome data processing; it is easily installable with conda and maintained as open-source on GitHub at
https://github.com/metagenome-atlas/atlas
.
Journal Article
Software and computing for Run 3 of the ATLAS experiment at the LHC
The ATLAS experiment has developed extensive software and distributed computing systems for Run 3 of the LHC. These systems are described in detail, including software infrastructure and workflows, distributed data and workload management, database infrastructure, and validation. The use of these systems to prepare the data for physics analysis and assess its quality are described, along with the software tools used for data analysis itself. An outlook for the development of these projects towards Run 4 is also provided.
Journal Article