HOOVER: Distributed, Flexible, and Scalable Streaming Graph Processing on OpenSHMEM
Presentation Date:
Presentation Slides:
Many problems can benefit from being phrased as a graph processing or graph analytics problem: infectious disease modeling, insider threat detection, fraud prevention, social network analysis, and more. These problems all share a common property: the relationships between entities in these systems are crucial to understanding the overall behavior of the systems themselves. However, relations are rarely if ever static. As our ability to collect information on those relations improve (e.g. on financial transactions in fraud prevention), the value added by large-scale, high-performance, dynamic/streaming (rather than static) graph analysis becomes significant.
This paper introduces HOOVER, a distributed software framework for large-scale, dynamic graph modeling and analysis. HOOVER sits on top of OpenSHMEM, a PGAS programming system, and enables users to plug in application-specific logic while handling all runtime coordination of computation and communication. HOOVER has demonstrated scaling out to 24,576 cores, and is flexible enough to support a wide range of graph-based applications, including infectious disease modeling and anomaly detection.
Knocking the Wind Out of Hurricane Harvey: Using Data to Strengthen Our Communities in the Face of Emergent Natural Disasters
Presentation Date:
Friday, August 10, 2018
Presentation Slides:
Hurricane Harvey was the most significant tropical cyclone rainfall event in US history, resulting in 300,000 confirmed flooded structures. At Harvey’s peak, it was classified as a Category 4 Hurricane with wind guests of up to 132 miles per hour and an aggregate rainfall greater than 44 inches in some areas. As context, the total precipitation for Houston in 2016 was 60.96 inches.
During and immediately following Hurricane Harvey, Rice University launched a data-driven disaster response effort that used a four prong strategy of data collection, data fusion, data analysis, and communications/outreach. The insights gained from live data streams were used to immediately and efficiently allocate limited university resources to those most in need.
However, it was immediately apparent that Harvey’s impact (and the impact of storms like it) would not be contained to the few days of the storm itself – indeed, the health, housing, financial, mental, and emotional impacts of Harvey on our Houston community will be felt for years to come. As emergent natural disasters become more common in Houston and around the world, the importance of studying their impact along as many axes as possible is magnified.
In response to this need, the Hurricane Harvey Registry (HHR) was launched – a health registry for collecting and securely storing the impact of Harvey on the Greater Houston region. Built around the HHR is a collaborative outreach, analysis, and (eventually) policy effort led by community stakeholders, including Rice University, the Houston Health Department, Harris County Health Department, Montgomery County Health Department, Fort Bend County Health Department, and the Environmental Defense Fund.
This talk will take us through the genesis of the HHR: from our data-driven disaster response during Harvey, to the insights and motivation gained through that experience, to the stand-up of the HHR, and finally closing with initial learnings and continuing challenges.
Contact
Rice University
(713) 348-4834
6100 Main St.
Duncan Hall 3122
Houston, TX 77005-1892
max.grossman@rice.edu