Page tree
Skip to end of metadata
Go to start of metadata

HPCC Systems 10 Year Open Source Anniversary

June 15, 2021 marks the 10th anniversary of HPCC Systems as an open source offering in the big data analytics market. Our 10 Year Anniversary Podcast Series showcases members of our HPCC Systems Community commemorating this milestone event. Hosted by Flavio Villanustre, VP Technology and CISO, LexisNexis Risk Solutions Group, who leads the HPCC Systems open source initiative, this series of interviews highlights how the move to open source has made a positive impact to our users and to the platform through the many contributions from the community. Interviewees include colleagues from LexisNexis Risk Solutions Group who were involved from the beginning of the open source movement and others whose role is dedicated to supporting the community. Others include our academic partners and interns who use HPCC Systems in research and in the classroom, and key industry users who base their business model and leverage HPCC Systems as their technology platform. Listen to the stories they share on how the platform has evolved over the years, the success stories from community users who are now advocates and influencers, and announcements on new functionality in the next major release, upcoming training courses, and more.

Keep checking back for more interviews!

Celebrating 10 years of innovation, growth, and success in a vibrant community...



Now Available

Interviewee(s) and AffiliationEpisode SummaryResources

Arjuna Chala
Senior Director, HPCC Systems Solutions Lab
LexisNexis Risk Solutions Group

Flavio kicks off with the introduction of this new podcast series to commemorate the 10 year anniversary of HPCC Systems as open source. He then welcomes Arjuna Chala as his first guest and one of the key people who helped bring HPCC Systems into the open source community. They begin by discussing how Hadoop was the only known player in the open source big data space and there was very little knowledge in the community about HPCC Systems, and how we overcame that challenge of educating people on what big data really means and emphasizing the efficiency of ECL. The conversation continues with how the terminology evolved along with the platform, and how today, COVID-19 is a classic Data Lake problem and HPCC Systems is a natural fit for drilling into uncover data insights easily and quickly. They conclude with sharing a few examples of customer success stories, including DataSeers, and how these use cases not only help the customer, but also contribute back to the platform – the true beauty of open source!

Bob Foreman
Senior Software Engineer
LexisNexis Risk Solutions Group

Richard Taylor
Senior Consulting Software Engineer
LexisNexis Risk Solutions Group

Flavio interviews Sr Software Engineer and ECL instructor, Bob Foreman, along with Sr Consulting Software Engineer and Chief Trainer, Richard Taylor, both with LexisNexis Risk Solutions Group to chat about their experiences in the training world of teaching ECL and HPCC Systems over the past ten years. First up, Richard tells the story of how the HPCC Systems platform was born in the late 1990s when there was a critical need for massively parallel processing of massive data sets – our first supercomputer. At that time Richard was the only trainer around to teach the platform and ECL language to our users, which then were our own employees and customers. Fast forward to 2011 when Bob Foreman joined the team as a second trainer which was perfect timing as that was when HPCC Systems became open source. Bob shares his passion and love for teaching and knew this was a huge step, and challenge, to introduce the platform and language to a new community. Training plays a vital role when wanting to build a strong community of users to learn, adopt and ultimately contribute back to the platform. This was no small task. Richard goes on to explain all the courses which were taught for a proprietary system had to be rewritten to fit the open source paradigm. Bob discusses how internal lingo and phrases had to change in order to make sense to a new set of users. They both were ready and eager to shift to a new way of teaching to accommodate an open source model. The benefits of open source brings in a wider community base across the globe, but with just two trainers, something innovative had to happen in order to support the reach of our growing community. This push gave birth to our first set of online self-paced classes, or “MOOCs” as Flavio likes to call them, and remote training. This became a game changer. Classes were filling up and hitting record breaking attendance with students from various locations around the world.


Over the years, the training materials and lessons evolved to meet the needs of the open source community and helping businesses solve real world data problems. Bob, the voice of the online lessons, shares the best satisfaction he receives from teaching is seeing one of his students excel from what they learned in his class and go on to build impressive solutions, not only for their business, but for ours too. The open source world introduced unique case studies we have never seen before and it is a true win to see how HPCC Systems is being leveraged across a variety of industries from our external community. Richard also touches on his new 3-book series, with the first one, Definitive HPCC Systems, available now on Amazon, which covers an overview of the platform, with the following books focusing more on ECL.


A big announcement is also unveiled! To commemorate the 10th anniversary with our HPCC Systems community, we are giving away the entire suite of online courses for FREE all year long in 2021! And there is more! Bob talks about the new set of Applied ECL courses covering Machine Learning, and coming soon, Embedded Languages. They are excited to expand the course material to support the evolution of the platform with the move to the cloud and all the 8.0 functionality ahead. 10 courses and counting – all free! The discussion closes with a recap of the excitement of all the success from the last 10 years and looking forward to what the next 10 years will bring. And don’t forget – FREE training in 2021!

Bahar Fardanian
Technology Evangelist
LexisNexis Risk Solutions Group

In this episode, Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions, has a chat with Bahar Fardanian, Technology Evangelist, LexisNexis Risk Solutions Group, whose role was specifically created to support our growing open source HPCC Systems community.

Bahar begins by sharing how her career evolved and her love of big data by working in healthcare analytics shortly after graduating with her Masters degree. She was hooked and knew this is where she wanted to focus her career! After attending a career fair looking for companies who were in the big data industry, she landed her first role with LexisNexis Risk Solutions on the data scoring team. Leveraging all the material found on the HPCC Systems website, she became very familiar with the platform and infrastructure prior to her first interview. It wasn’t too long until Bahar became one of our key HPCC Systems ambassadors working alongside many of our community members helping them with their big data projects. Seeing how this seemed to be a perfect fit with her passion, skills and influence, she landed a new role as a technology evangelist. This unique role also has a special focus on supporting our academic community where she has the opportunity to run hackathons, workshops - and new this year, she is teaching her very first big data course at her alma-mater, Kennesaw State University. She shares that it is an amazing experience to help students who start out with very little knowledge of big data, to seeing them now solve real-world problems as a result from her mentorship and teaching.

The benefit of having a data lake platform as open source allows these students to experience industry focused problems to ramp up their skills so they can be work-ready upon graduation. She looks forward to continuing to work with different universities and schools in the coming year. Bahar uses her own success story when speaking to students, especially females, and encourages them to follow their passion to find a job that allows them to do what they love. Everybody can do this! What a great feeling to know that the college student who stumbled across the LexisNexis booth at a Career Fair a few years ago, would now be the one at the “other side of the table”, evangelizing about the HPCC Systems platform and giving students the same opportunities to find their love with big data!

Dr G Shobha
Professor, CSE Department
R V College of Engineering
Bengaluru, India

This next segment features our 2020 HPCC Systems Community Recognition Award winner, Dr G Shobha, Professor, CSE Department from R V College of Engineering (RVCE) in Bengaluru, India speaking with our host, Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions Group. Dr Shobha begins by sharing how she learned about HPCC Systems in 2016 through a conference where Arjuna Chala delivered a keynote. Seeing how he was an alumni of RVCE, an instant connection was made and their collaboration journey started. The discussion continues with Dr Shobha highlighting the number of projects her team has worked – seven in total so far, covering various aspects of HPCC Systems components. She explains how the benefits of working with an open source platform allows her students to contribute back to the community in the form of talks, papers and bundles. This gives them great exposure to real-world experience and also gives great visibility to the university as well. Dr Shobha announces the new course she is teaching at the university on big data analytics leveraging HPCC Systems and looking forward to facilitating additional workshops and hackathons, not only at RVCE, but other local colleges in India, helping to further expand our International community!

Dan Camper
Senior Architect, HPCC Systems Solutions Lab
LexisNexis Risk Solutions Group

Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions Group, introduces Dan Camper, one of our founding HPCC Systems Community members, who now is an employee of LexisNexis Risk Solutions Group. Dan explains how he stumbled upon HPCC Systems by accident who was doing a Google search on something else. HPCC Systems appeared in the results and caught his interest and decided to “give it a whirl”. At the time he was looking for something to manage lots and lots of data and HPCC Systems was a natural fit. Dan shared he is a programmer at heart and loves to learn and build new languages. His passion with coding and working with data led him to a key position on the HPCC Systems core team which he has been advising and mentoring other users within our community. He is looking forward to the cloud movement and Kubernetes work with HPCC Systems to make it more dynamically scalable, which will be a huge game changer in this next release! Dan provided an update on the Data Patterns bundle he created and new data profiling capabilities available based on the needs and requests coming from the open source community. They conclude with Dan sharing new functionality users can might leverage with the move to the Cloud and interoperability with other languages. Dan ends with giving advice to inspiring programmers to don’t sell ECL short and don’t wait to “give it a whirl”.

David Dasher
Chief Technical Officer
CPL Learning

Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions Group, welcomes David Dasher, CTO, CPL Learning, as the guest for this episode. His company was one of the first adopters of HPCC Systems for his business in the hospitality industry. David explains the challenges he faced at the time with slow performance and complex and incomplete reporting for the huge amounts of data they had. Fortunately, he was in the right place in the right time attending a meetup where his good friend Jo Prichard was presenting on HPCC Systems. David then had his ah-ha moment and immediately downloaded the platform and built his first cluster and was blown away with the results! He said it was love at first sight. Over the years, his business has grown in magnitudes and having HPCC Systems at its core has been a huge advantage to support that growth and success. David is looks forward to the new Azure cloud offering, visualization enhancements, and ECL Watch improvements which is something his team heavily relies on to fine-tune performance levels which are critical to his business. David goes on to say how much he appreciates the nature of open source and informative documentation, forum posts and blogs that are readily available to the community. As a presenter for many of our Community Day events, he anxiously awaits the day we can all come together again to share knowledge and brainstorm with the core platform team and fellow Community members. He ends by saying his job is now his hobby and his love story with HPCC Systems continues!

Rodrigo Pastrana
Architect
LexisNexis Risk Solutions Group

James McMullan 
Sr Software Engineer
LexisNexis Risk Solutions Group


This next episode features our host, Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions Group, interviewing two esteemed technologists from our core HPCC Systems platform team, Rodrigo Pastrana, Architect, and James McMullan, Sr Software Engineer. Rodrigo joined the team right around the time the platform went open source, so he shares his insight on how the platform has evolved over the years by exposing our technology to the community. James started midway through our open source journey with one of his first projects tackling the integration of Spark with HPCC Systems. Both Rodrigo and James have developed many connectors and plugins to help build the bridge for working with datasets that utilized other open source projects. They explain this has not only helped expand our ecosystem, but also opened the door to new opportunities for creating additional interfaces to accommodate any data size or format. In order to support and grow our open source community, they continue to develop these modules that extend the capabilities of HPCC Systems, which are free to the community for integrating third party tools with the platform. They close with giving an update on some of the new work they are doing to help users better manage their data in the cloud, including improved logging, easy to read visualizations, and enhanced data movement across various formats. Yet another example of the benefits of open source, allowing the developer community to contribute different technology and come together to build something great!

Lili Xu
Software Engineer III
LexisNexis Risk Solutions Group

Yash Mishra 
Software Engineer II
LexisNexis Risk Solutions Group


In this next episode of the HPCC Systems 10 year anniversary series, Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions Group, interviews two members from our academic community – who are now employees. Flavio welcomes Lili Xu and Yash Mishra, both who joined our community from Clemson University and also participated in our HPCC Systems internship program. Lili begins by sharing how she learned about HPCC Systems during her first PhD year through the Clemson University DICE lab managed by Dr Amy Apon. Her first taste of ECL was reading the ECL Programmers Guide which got her hooked to learn more about the platform. Today, you can see her mark in many projects including Machine Learning, NLP, DBSCAN, and her latest development, HSQL – a new language allowing SQL developers to easily work with HPCC Systems without ECL knowledge. Lili also shares her involvement as a mentor now, and how she enjoys working with students to learn HPCC Systems through facilitating workshops, hackathons and other events.

Yash has a similar story, also being part of the DICE lab, but learning about HPCC Systems through a cloud computing architecture class taught by Dr Apon. He was intrigued with the different aspects HPCC Systems could interact with the cloud. Seeing this was a major direction for our roadmap, Yash landed an internship working on leveraging Azure with HPCC Systems. He talks about his experience from working on the project and using Kubernetes to help fine-tune performance across the environment, which also helps with cost assessment for anyone who is venturing into the cloud native journey. Yash concludes by sharing what his new role as a Software Engineer on the MVR team entails and looks forward to the new challenges ahead.

Both Lili and Yash have made many contributions to the platform and will always be key members of the HPCC Systems community, and now, we are honored to have them as colleagues!

HPCC Systems Platform Team Mini-Series
These talented technologists forming the platform team are the early adopters of new technology and get to work on the interoperability of many different tools with HPCC Systems. More recently, the move to the cloud is at the top of the roadmap with the team exploring, testing and configuring different methods to make your clustering environment in the cloud at its best. The HPCC Systems platform team is excited to sit on the front lines of working with different operating systems, containers and other bleeding edge technology to bring HPCC Systems into the next generation! Check back for additional episodes coming soon featuring this team!

Stuart "Stu" Ort
Sr Director Software Engineering
LexisNexis Risk Solutions Group

Kunal Aswani 
Software Engineer III
LexisNexis Risk Solutions Group

Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions Group, kicks off the first of this mini-series as part of the 10 Year Anniversary podcast series featuring the platform team supporting HPCC Systems. In this episode, Flavio welcomes his first two guests, Stuart “Stu” Ort, Sr Director Software Engineering, and Kunal Aswani, Software Engineer III, both also with LexisNexis Risk Solutions Group. Having been with the company for 20 years now, Stu has been involved in a variety of roles from the early days through following the open source movement, so has experience from both sides with supporting internal users as well as the wider open community. He shares how changes to accommodate the open source community not only positively impacted the platform, but also career growth across his team. Stu explains the methods and approaches that were once used to serve the internal community had to adapt in order to serve a diverse and expanded userbase. He looked at this as an opportunity to improve and enhance a number of the responsibilities his team owns with support, security, documentation, installation practices and more.

Joining the team in 2014, a few years after going open source, Kunal’s main focus is working on the frontend of ECL Watch. His involvement and contributions has helped ECL Watch grow from a simple webpage to a more rich and robust web browser interface giving the users a visual story of their environment performance. Kunal prides himself on putting the user first and working to incorporate additional features that give users what they want in the most optimal way. This is why you can also find Kunal interacting with our users, serving as one of our key ambassadors in events, poster competitions and engaging with our wider external community.

Jim DeFabia
Consulting Software Engineer
LexisNexis Risk Solutions Group

Greg Panagiotatos
Software Engineer III
LexisNexis Risk Solutions Group

Continuing the 10 year anniversary podcasts, in the second installment of the HPCC Systems platform team mini-series, this episode features the documentation team responsible for producing the critical material our community needs to use the platform and its components. Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions Group sits down with Jim DeFabia, Consulting Software Engineer, and Greg Panagiotatos, Software Engineer III, both having decades of service years with the company. Jim kicks off the discussion explaining how going open source was a game changer for documentation, and the necessary shift to treating documentation as code with all the careful handling that comes with it, including version control, review cycle and merge requests. This not only allowed greater flexibility, but improved team collaboration. Greg’s background working in the legal and professional side of the company where documentation was the actual product, was a complimentary addition when he joined the team to open source HPCC Systems. His familiarity and experience working with open standards-based markup languages, like XML, helped generate the documentation to a more suitable format to accommodate an open source community.

Flavio emphasizes the fact that documentation plays a crucial role when deciding to go open source. A community cannot be formed with just releasing the code itself, proper documentation is a must-have. Over the years, both Jim and Greg have made many contributions to give our users very informative and easy to follow documentation. From tutorials, reference guides, how to videos, cook books, and best practices – (quite a few in multi-languages!) - the written words from Jim and Greg are engrained in every facet of the platform. Our community members depend, and rely heavily on documentation, and this team makes it a high priority to ensure our users can easily find the answer to their question. Throughout the forums, the website, the FAQs, and more, Jim and Greg are at the top of the BFF list with our community!

Michael Gardner
Software Engineer III
LexisNexis Risk Solutions Group

Xiaoming "Ming" Wang
Sr Consulting Software Engineer
LexisNexis Risk Solutions Group

This third installment of the HPCC Systems platform team mini-series features our host, Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions Group, interviewing two of our technologists responsible for supporting the overall infrastructure, builds and releases, Michael Gardner, Software Engineer III and Xiaoming “Ming” Wang, Sr Consulting Software Engineer, both with LexisNexis Risk Solutions Group. Flavio begins with Ming giving a background on joining the company in 2013 with his initial focus on managing the builds and related tools. Now with the move to the cloud, his role has expanded to involve integrating third party solutions such as Kubernetes and AWS with HPCC Systems. Michael joined in 2014 through the internship program and began working on the administrative tools supporting the platform builds, and like Ming, has now shifted to more cloud management tasks, like Docker, helping to expand the HPCC Systems ecosystem.

Both explain the challenges with the shift from bare metal to the cloud and give insight on how it has been beneficial in many aspects from setting up a cluster, data storage, performance improvements, to deployment. Being an open source platform, they also have to keep in mind usability, and taking an adaptive approach for providing different builds and configurations for supporting a wide variety of users. Through our own cloud initiatives, Michael and Ming share tips on deployment strategies and stress the need for testing. This is where the open source community plays a big role in contributing feedback to help fine-tune the environment. Ming and Michael are very active in the community with delivering Tech Talks, presentations and also mentoring students in our internship program. Be sure to listen to the full episode to catch all the helpful information they share to help you with your cloud journey!

Russ Whitehead
Architect
LexisNexis Risk Solutions Group

Kanghua "Kevin" Wang
Sr Consulting Software Engineer
LexisNexis Risk Solutions Group

The fourth episode of the Platform team mini-series includes interviews from Russ Whitehead, Architect, LexisNexis Risk Solutions Group and Kanghua “Kevin” Wang, Sr Consulting SWE, LexisNexis Risk Solutions Group talking about the importance of security in the HPCC Systems platform. Our host, Flavio Villanustre, VP Technology & CISO, LexisNexis Risk Solutions Group, begins with asking Russ about the early days before the platform was open source. Russ joined the company in 2008 and has always been working on the HPCC Systems platform. He explains the moment the decision was made to make the platform available to the open source community, security would become his major focus going forward. Kevin joined the company a few years earlier in 2005, and to this day, his primary responsibility is ESP services for ECL Watch.

Russ explains there was a lot of pertinent security changes that needed to happen in order to accommodate the open source community. Working together, Russ put into place a security manager framework which leverages Kevin’s user management service for robust authentication. Over time, this team has expanded the framework to accommodate pluggable security managers in which the community can customize permissions, roles, exceptions and other security features to meet their needs.

With the move to the cloud, both Russ and Kevin are working on new security models for better cloud support, which also aligns with our upcoming 8.0 release. Users will see extended ESP functionality for working with containers for improved deployment and performance. We all know that security is vital when it comes to protecting your data and environment. Our community relies heavily on our Security SMEs, Russ and Kevin, who help make HPCC Systems a mature and compelling solution!


Coming Soon

Interviewee(s) and AffiliationEpisode SummaryResources

Krishna Turlapathi
Director, Software Engineering
LexisNexis Risk Solutions Group

Ken Rowland
Consulting Software Engineer
LexisNexis Risk Solutions Group

This latest episode as part of the HPCC Systems Platform team mini-series features Krishna Turlapathi, Director Software Engineering, LexisNexis Risk Solutions Group, and Ken Rowland, Consulting Software Engineer, LexisNexis Risk Solutions Group. They share the latest they are working on with metrics, configuration tools and ECL development efforts. 

Full interview recap and recording coming soon.

Recording coming soon

Christopher Lo
Sr Software Engineer
LexisNexis Risk Solutions Group

Godson "Godji" Fortil
Software Engineer I
LexisNexis Risk Solutions Group

The final episode of the HPCC Systems Platform Team mini-series includes interviews with Christopher Lo, Sr Software Engineer, and Godson "Godji" Fortil Software Engineer I, both with LexisNexis Risk Solutions Group. They discuss with Flavio how the move to open source has changed the dynamics of testing and tech support for accommodating our open source community.

Full interview recap and recording coming soon.

Recording coming soon





Return to top




  • No labels