We are happy to annouce the release of Pegasus 4.3.1. Pegasus 4.3.1 is a minor release, that has minor enhancements and fixes bugs to Pegasus 4.3.0 release.
Support for Fixed Output Mapper
Using this output mapper, users can specify an externally accessible URL in the properties file, pointing to a directory where the output files needs to be transferred to. To use this mapper, set the following properties
The Pegasus team will be at SC'13! Please come and talk to us at one of the following events.
Paper presentation at The 8th Workshop on Workflows in Support of Large-Scale Science (WORKS13) http://works.cs.cardiff.ac.uk/index.php Talk by Rafael Ferreira da Silva, the newest Pegasus team member: Sunday 12:00-12:30 Toward Fine-Grained Online Task Characteristics Estimation in Scientific Workflows R. Silva, G. Juve, E. Deelman, T. Glatard, F. Desprez, D. Thain, B. Tovar, and M. Livny
The Pegasus group recently organized a software carpentry workshop at USC on September 17-18th, 2013.
The workshop focussed on helping bioinformatics scientists and engineers become more productive by teaching them basic computing skills like program design, version control, testing, and task automation. Over a course of two days, short tutorials alternated with hands-on practical exercises.
The topics covered were
using the shell to do more in less time
using version control to manage and share information
basic Python programming
how (and how much) to test programs
HPCC Computing Resources
introduction to workflows and Pegasus WMS
RNASeq bioninformatics pipeline through Pegasus WMS
We are excited to invite the community to a SC'13 BOF we are involved with:
Abstract: The purpose of this Birds-of-a-Feather (BOF) session is for computational scientists, scientific workflow software researchers, cyberinfrastructure architects, and cyberinfrastructure operations leaders to share ideas on how to best use cyberinfrastructure such as the NSFs XSEDE to tackle the most challenging and scientifically valuable distributed computing problems. The emphasis will be on discussing open, unsolved scientific problems that will benefit from using multiple XSEDE, Open Science Grid, campus, and international cyberinfrastructure resources in coordinated fashion. The BOF will be mediated by the XSEDE Workflow Community Applications Team.
The July 24, 2013 issue of iSGTW is featuring a Pegasus workflow running on the Open Science Grid:
“We had contributing nodes on OSG totaling near 1.4 million hours for the first 19.5 million jobs, which ran for about a month,” Quick says. “Since then we’ve done two more runs and we’re now up to more than 30 million docking jobs consuming over 3 million CPU-hours from opportunistic OSG resources.”
Speaker: Thomas Fahringer, University of Innsbruck
Abstract: Researchers from many different domain sciences create and use ever-increasing amounts of complex data, and rely more and more on compute-intensive modelling, simulation and analysis. Scientific workflows have become a key paradigm for managing complex tasks and have emerged as a unifying mechanism for handling scientific data.
Unfortunately, workflows developed for one workflow system are normally not compatible with workflows of other workflow systems. Every workflow system usually has its own user interface/API, description language, and enactment engine which are not based on standards and therefore prevent interoperable workflow systems. Workflow integration or reuse therefore is currently impractical, thereby inhibiting the growth in uptake and proliferation of workflows in scientific practice.
The following seminar was held at USC/ISI on 6/19/13
Abstract: ExoGENI is a new GENI testbed that links GENI to two advances in virtual infrastructure services outside of GENI: open cloud computing (OpenStack) and dynamic circuit fabrics. ExoGENI orchestrates a federation of independent cloud sites located across the US and circuit providers, like NLR and Internet2 through their native IaaS API interfaces, and links them to other GENI tools and resources. In this talk we explore the capabilities and internal architecture of ORCA (Open Resource Control Architecture) - the software that powers ExoGENI, and its use as a platform for GENI experimentation and for running complex distributed applications
An atlas compiled from more than 18 million images reveals in unprecedented detail the structure of the Milky Way's galactic plane—the center line of the galaxy that contains most of its stars. Researchers used the NSF-funded Pegasus Workflow Management System (Pegasus WMS) to produce this map of the light that the interstellar medium and dense molecular clouds absorb in the near-infrared and subsequently re-emit at longer wavelengths.
Notes from the recent OSG CIC Pegasus webinar were posted in the OSG newsletter:
Freeform From the OSG Blogs – Notes from the Campus Infrastructure Community Pegasus webinar
Mats Rynge gave the webinar on “Managing HTC Workflows with Pegasus” from the OSG Campus Infrastructures website: http://www.campusgrids.org/. This webinar is available from the WEBCASTS tab or from the following direct link to materials on the Pegasus website. There were 18 participants.
Abstract: Distributed computing infrastructures are commonly used through scientific gateways, but operating these gateways requires important human intervention to handle operational incidents. In this work, we present a self-healing process that quantifies incident degrees of workflow executions from metrics measuring long-tail effect, task granularity, and fairness among workflows. These metrics are simple enough to be computed online and they make little assumptions on the application or resource characteristics. Incidents are classified in levels and associated to sets of healing actions that are selected based on association rules modeling correlations between incident level. The healing process is parametrized on real application traces acquired in production on the European Grid Infrastructure.