News

Abstract: Distributed computing infrastructures are commonly used through scientific gateways, but operating these gateways requires important human intervention to handle operational incidents. In this work, we present a self-healing process that quantifies incident degrees of workflow executions from metrics measuring long-tail effect, task granularity, and fairness among workflows. These metrics are simple enough to be computed online and they make little assumptions on the application or resource characteristics. Incidents are classified in levels and associated to sets of healing actions that are selected based on association rules modeling correlations between incident level. The healing process is parametrized on real application traces acquired in production on the European Grid Infrastructure.

Maciej Malawski from AGH University of Science and Technology gave a seminar at ISI on April 11th titled "Cloud Platform for VPH Applications".

We are proud to announce new changes to Precip - Pegasus Repeatable Experiments for the Cloud in Python

Two features were added to the precip API:

First of them is the retry feature. Some of the cloud instances fail to boot correctly and they will reach timeout in the middle of the experiments. So in the provisioning phase of the precip whenever an instance has booting problems, it will retry the booting phase according to the times user asked for.

The second feature is the change to the way the SSH keypair is stored. In order to use precip from multiple machines or accounts at the same time the way the keys are stored is changed. Now at the beginning of every connection between the submit host and the cloud a unique id is generated for each account and the keys are stored in "precip_id". In this case different machines with different ids can use it at the same time.

The upcoming edition of the US News and World Report, which will become public on March 12, 2013, ranks the USC Viterbi School of Engineering in the top ten graduate programs in engineering.

Pegasus 4.2 is a major release of Pegasus which contains 

• several improvements on data management capabilities,
• a new web based monitoring dashboard
• job submission interfaces supported. CREAM CE is now supported
• new replica catalog backends. 
• support for PMC only workflows and IO forwarding for PMC clustered jobs
• anonymous usage metrics reporting

Workflows, especially data-driven workflows and workflow ensembles are becoming a centerpiece of modern computational science. However, scientists lack the tools that integrate the operation of workflow-driven science applications on top of dynamic infrastructures that link campus, institutional and national resources into connected arrangements targeted at solving a specific problem. These tools must (a) orchestrate the infrastructure in response to application demands, (b) manage application lifetime on top of the infrastructure by monitoring various workflow steps and modifying slices in response to application demands, and (c) integrate data movement with the workflows to optimize performance.

Next week the annual Supercomputing conference will be held in Salt Lake City, Utah. If you are planning to attend the conference we encourage you to stop by one of the Pegasus-related talks that are scheduled during the week.

Several of the talks will be given at the workshop on Workflows in Support of Large-Scale Science (WORKS) on Monday. More information can be found on the WORKS website:  http://works.cs.cardiff.ac.uk/program.php

If you would like to discuss the Pegasus Workflow Managment System or other projects such as running workflows on the Cloud, resource provisioning, or our new Precip: Pegasus Repeatable Experiments for the Cloud in Python software, please come to the USC booth during Ewa's talks.

The complete list of Pegasus-related talks is as follows:

Monday 10:50AM: "Peer-to-Peer Data Sharing for Scientific Workflows on Amazon EC2", Gideon Juve, WORKS Workshop, Room 155-A

From the article:

Over the past several years, the US National Science Foundation has been funding the development of collaborative web sites or ‘collaboratories’ and scientific workflow technologies. Many communities have adopted the HUBzero platform to create collaboratories called ‘hubs’ where they can share ideas, models, experiences, publications, and data in pursuit of research and education. Scientists have also been using the Pegasus Workflow Management System (WMS) to manage complex analyses running on their campus and on large-scale cyberinfrastructure, such as the Open Science Grid, DiaGrid, and XSEDE.

Read the full article at the ISGTW website