Become a professional.

You probably know by now: Spark is the most popular computing engine for big data, the most maintained, and with a proven track record of performance. It’s 100 times faster than the old MapReduce paradigm, and can easily be extended with machine learning and streaming capabilities, and much more.

If you’re dealing with large amounts of data, learning Spark is a must.

The demand for Spark has skyrocketed, and companies are struggling to fill their Data Scientist positions. Scala and Spark are two of the best paying technologies in the field. Forget the reported 120k salaries on PayScale and StackOverflow. I regularly see engineers and data scientists working for 150k+ per year, or charging thousands a day for consulting.

However you take it, learning Spark will be a game changer for your career, if you choose. And this course will help you get those skills. Join this Spark Big Data online course and learn by writing code.

Work with real big data.

At the end of the course, we will dive into one of the biggest datasets publicly available and we’ll put everything that we’ve learned to the test. Unlike every other material on the web (free or paid), this Spark course is the only place where you can really practice big data.

Everyone else runs a Spark job on one million records in a 20MB dataset. Why would you need Spark for that?

We do 1.4 BILLION car trips in a 400GB dataset. You don’t fit that into most computers. That’s the definition of big data.

This will be your true exercise to practice Spark with Scala. At the end of the project, we gather data insights worth millions of dollars for the company you’re looking to help, and tens of thousands for you as a consultant and data scientist.

Take the proven path

As with the other Rock the JVM courses, this Spark and Big Data Essentials course will take you through a battle-tested path to Spark proficiency as a data scientist and engineer.

As always, I’ve:

Deconstructed the complexity of Spark in bite-sized chunks that you can practice in isolation
Selected the essential concepts and exercises with the appropriate complexity
Sequenced the topics in increasing order of difficulty so that they “click” along the way
Applied everything in live code

Skills you’ll get

In short, you’ll be a productive Spark developer and you can become a data engineer.

You’ll know how to work with DataFrames for Spark jobs of any complexity
You’ll be able to integrate any data source with Spark, and you’ll know how to configure them
You’ll be able to perform
- DataFrame transformations
- Aggregations and data statistics
- Joins
- Unions
- Arbitrary transformations with Datasets
You’ll know how to process absent values
You’ll understand what data cleanup means and how to do it
You’ll be able to deconstruct complex types like structures and arrays
You’ll be able to manage type safety with DataFrames and Datasets
You’ll understand Spark internals and run arbitrary Scala code wit hRDDs
You’ll know how Spark works in a cluster
You’ll deploy your own Spark apps on Amazon
You’ll run your own data engineering projects

After this course, you’ll be able to process data in any way you need using Spark.

But most importantly, you’ll develop timeless skills that you’ll carry with you for your entire career, regardless of which data streaming tool you’ll end up using:

You’ll deeply understand the practical benefits of big data in general
You’ll be able to work with Spark for any job
You’ll know how to pick the right tool or the right API levels

Daniel Ciocîrlan

I'm a software engineer and the founder of Rock the JVM.

I'm a software engineer and the founder of Rock the JVM. I started the Rock the JVM project out of love for Scala and the technologies it powers - they are all amazing tools and I want to share as much of my experience with them as I can.

As of February 2024, I've taught Java, Scala, Kotlin and related tech (e.g. Cats, ZIO, Spark) to 100000+ students at various levels and I've held live training sessions for some of the best companies in the industry, including Adobe and Apple. I've also taught university students who now work at Google and Facebook (among others), I've held Hour of Code for 7-year-olds and I've taught more than 35000 kids to code.

I have a Master's Degree in Computer Science and I wrote my Bachelor and Master theses on Quantum Computation. Before starting to learn programming, I won medals at international Physics competitions.