The Collaborative and Adaptive Sensing of the Atmosphere (CASA), has the goal to improve our ability to observe, understand, predict, and respond to hazardous weather events. CASA presents data movement challenges and the need to elastically scale resources on demand. Moment data from a network of seven weather radars, located in Dallas/Fort Worth (DFW), flow continuously to CASA’s data repository and new processing tasks are triggered as new data arrive. Additionally the severity of weather events increases the processing time. In these cases, in order to satisfy CASA’s QOS, more resources have to be provisioned dynamically, since data processing cannot be postponed until resources become available again.
Two of CASA’s pipelines have been ported to Pegasus workflows and are taking advantage of the improved Mobius capabilities to instantiate compute resources across different cloud infrastructures and setup dedicated network links between the cloud resources, as well as between CASA’s data repository and all cloud resources, using Software Defined Networks (SDNs). The first pipeline is called Wind Speed, while the second pipeline is called Nowcast.
The Wind Speed pipeline, ingests radar moment data, especially radial velocity, but also other variables for noise reduction purposes. The first step in the process is the merging of an arbitrary number of individual radar data files. The wind speed algorithm attempts to create a grid of the maximum observed velocity, corrected for the height of the observation. The output is a single grid representing the estimate of the wind speed near the ground.
Wind Workflow DAG
The Nowcast pipeline ingests asynchronous individual radar data that have previously been combined into a merged grid. These gridded data are accumulated over time for a certain initialization period and then are used as input to the nowcast algorithm. In its current version, the algorithm is processing data every minute and produces a series of gridded reflectivity data, where each grid corresponds to the projected conditions of each minute within the span of the next 30 minutes into the future.
Nowcast Workflow DAG