Chin-Yung Hsu - Poster presentation entry 2016

Browse: More posters, Abstracts, About the HPCC Systems 2016 Poster Presentation Competition

Elastic Computing Support for HPCC Systems

Users of cloud computing only want to use and pay for the resources they actually need. Elastic computing provides the means for easily scaling up or down resources giving users flexible computing power when and where it is required. Chin Jung Hsu is a PhD student at North Carolina State University who has been working on a project to evaluate whether we can add more complete elasticity support to HPCC Systems. He started this project focusing on data replacement for the Roxie cluster a year ago.

It’s hard for workloads from real-world applications to be uniform, which means that loads are not evenly distributed among the Roxie nodes. Chin’s hypothesis was that for better system performance, the number of data replicas and their placement need to match the workloads. The aim of this project was to determine the best number of data replicas and their locations to fit the anticipated workloads. His initial results show that the proposed ‘rainbow’ data placement, a fine-grained data placement scheme which maximizes the number of unique data replicas per node, brings significant performance improvements for both query throughput and latency, of around 76% and 31% respectively as well as improving query throughout by as much as 1.55 times (in extreme cases).

The next step on this project is to look into the resource configurations for Thor on AWS. He plans to develop a cost and performance model to help the HPCC Systems Community make decisions about deploying onto AWS.

Chin's poster presentation entered into our competition held on Community Day at the HPCC Systems Engineering Summit in 2016.