Tech Talk 31 - February 20th 2020

11am

Guest Speakers and subjects:

André Felipe Santos Martins, Student, Federal Institute of Espírito Santo (IFES) - Campus Serra, Brazil - Watch Recording
Infrastructure analysis of elementary schools in Brazil using HPCC Systems
Historically, Brazil’s public primary education has always been characterized as a sector suffering from a lack of financial resources. Due to its continental dimensions and to the fact that there are almost 200,000 primary schools spread across the country, the distribution of the basic education public funds tends to be more concentrated around the largest urban centers. In many cases, it is possible to identify poor and hard to reach regions that are far away from these centers, whose schools have critical infrastructure issues, such as lack of sanitation. These conditions may not only immediately reflect on the quality of education delivered for primary students, but also in the long term intellectual, financial and life quality development of the communities located on these regions. Recent surveys, for example, show that infrastructure problems were responsible for closing more than 10,000 schools in rural areas of Brazil between the years of 2010 and 2015.
In order to remediate this scenario, in April of 2019, the Brazilian government and the Brazilian Ministry of Education (MEC) announced a relocation of 30% from the budget originally targeted for public higher education over to public primary education (approximately R$5.8 billions). Despite this announcement, this year, MEC paradoxically halted the investment of approximately R$2.4 billions originally planned for primary education programs and the construction of new schools. This is, therefore, an opportune moment for a better understanding of the Brazil´s basic education infrastructure, in order to ensure the continuity and correct allocation of such public investments.
Based on the current context of the basic education in Brazil, this study aims to analyze the current infrastructure situation of primary schools across the country, by focusing on the conditions and resources that these institutions provide to their students and teachers, such as basic sanitation, transportation, and internet access. The data from this analysis can further highlight how the recent cuts in public investment can negatively affect even more schools in poor areas of the country, such as institutions located in indigenous reserves and rural areas that are difficult to access.
This study has leveraged the HPCC Systems platform for data manipulation and analysis. The datasets utilized are public, provided by the National Institute for Educational Studies and Research Anísio Teixeira (INEP), between the years of 2015 and 2018. The database is composed of data referring to the schools, its classes, teachers and enrolled students.
Starting from the principle that the public schools must provide good infrastructure conditions to their students, teachers and support staff in order for them to develop their educational activities accordingly, it is expected that the information gathered in this study will contribute for the promotion of public policies that guarantee quality education in all regions of Brazil.
Yash Mishra, PhD Student, Clemson University - Watch Recording
Deploying HPCC Systems on Microsoft Azure
Massive computation workflows utilize High Performance Computing Cluster (HPCC) Systems environments that require significant resources, but these resources are only needed for the time of the actual processing. Many commercial cloud resources are available to deploy HPCC Systems. The existing HPCC Systems® Instant Cloud for AWS offers flexibility to instantly create, manage and terminate cluster resources on AWS cloud [1].

This project is a new effort to provision and deploy HPCC Systems on Microsoft Azure Cloud. Azure Resource Manager (ARM) provides resource templates to deploy infrastructure as a code [2].

The idea is to use ARM templates and provision HPCC systems on Azure. A working deployment on Azure would enable us to analyze some research questions such as, should a HPCC Systems environment on Azure have large number of nodes with small memory or small number of nodes with large memory? What type of storage would be best to handle massive workload results? What could be an optimal auto-scaling setting for Thor and Roxie instances? By answering these questions we aim to identify the best configuration options for HPCC Systems on Azure for a candidate synthetic workload that Lily Xu is testing for massive data analysis. At a later stage, we intend to work on containerization environment and strategy for HPCC Systems using Azure Arc [3].

References:
[1] HPCC Systems® Instant Cloud for AWS, Version 7.6.10-1. Boca Raton Documentation Team, 2019, https://d2wulyp08c6njk.cloudfront.net/releases/CE-Candidate-7.6.10/ docs/EN_US/InstantCloud_for_AWS_EN_US-7.6.10-1.pdf.
[2] Templates overview - Azure Resource Manager. Retrieved from https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/overview
[3] Azure Arc.Retrieved from https://azure.microsoft.com/en-us/services/azure-arc/

Yash Mishra is a graduate student in Computer Science at Clemson University. He received his B.S degree in Computer Science from University of Wisconsin, Milwaukee. He is a Research Assistant at Data Intensive Computing Ecosystems (DICE) Lab at Clemson University, working under the supervision of Dr. Amy Apon. He has a growing interest in Cloud Computing, and architecting cloud-based solutions for domain specific workloads. He was introduced to HPCC Systems in the Cloud Computing Architecture class at Clemson and has been involved in identifying different configuration options to deploy HPCC Systems on commercial cloud. In his new project, he is working on provisioning HPCC Systems on Microsoft Azure to test a custom data-intensive workload on the cloud.