Browse: Abstracts, Winners and runners up, Awards Ceremony (Watch Recording / View Slides), Posters by HPCC Systems Interns, Posters by Academic Partners, Poster Judges, Virtual Judging, Home

Open

Guilherme Santos da Silva is studying for a Bachelor of Computer Engineering at Federal Technological University of Paraná (UTFPR) in Brazil. Guilherme is currently an intern at LexisNexis Risk Solutions Group in Brazil. He has always been very connected to technology and is now studying a Computer Engineering course that covers both software and hardware. Guilherme is interested in working with data in different ways and aims to use his knowledge in the area to contribute to society.  This project originates from foundations laid by David Wheelock and is a project of importance that contributes greatly to the team. Guilherme liked the idea of being able to help the team with this valuable project.

Poster Abstract

A cluster is a connection between two or more computers with the purpose of improving the performance of systems in performing different tasks. In the cluster, each computer is called “node” and there is no limit to how many nodes can be interconnected. Then, computers start to act within a single system, working together in processing, analyzing and interpreting data, information and/or performing simultaneous tasks. It is interesting to know information about a cluster, such as its capacity and availability. This monitoring can help in the maintenance and management of this type of system, avoiding the upload of very large files that will occupy a large part of its capacity or informing the user of the ideal size that can be used. Currently, there is no such control and monitoring in clusters, and the system administrator only finds out that the cluster is out of memory when teams notify it.

Therefore, this project proposes a cluster monitoring system. This is a system that maintains a list of all files in each of the Thor clusters and presents a dashboard that can enable a user to drill into disk utilization on any of them. It is the most efficient way to track multiple data sources as it provides, in real time and in a single location, all the information needed to track a company's performance. As data is displayed in real time, long hours are not required to interpret all indicators and the time to communicate results is shorter and more efficient.

The project consists of a structure formed by .ecl files, that includes a section to run the build hourly in the cron, a section to collect all the information from each server of a cluster, collate it, and create a key to be used by the service, and the service itself; and an HTML file which is a dashboard that shows the file system usage across all clusters. The user will be able to select the desired cluster and thus have access to data and usage graphs over time (virtual size and real occupied area of the disk). Each dot on these charts, when clicked, will call a roxie service showing the five main scopes and their relative usage in a pie chart, all using the Visualizer Bundle. With the tool we will provide an alert that is triggered when the cluster is running out of memory. The ultimate goal of this project is to provide the tool for people to use in any cluster.

Presentation

In this Video Recording, Guilherme provides a tour and explanation of his poster content.

HPCC Systems File Usage Monitor 

Click on the poster for a larger image. 

Open