school-of-sre/courses/big_data/intro.md

Big Data

Prerequisites

• Basics of Linux File systems.
• Basic understanding of System Design.

What to expect from this course

This course covers the basics of Big Data and how it has evolved to become what it is today. We will take a look at a few realistic scenarios where Big Data would be a perfect fit. An interesting assignment on designing a Big Data system is followed by understanding the architecture of Hadoop and the tooling around it.

What is not covered under this course

Writing programs to draw analytics from data.

Course Content

Table of Contents

1. Overview of Big Data
2. Usage of Big Data techniques
3. Evolution of Hadoop
4. Architecture of hadoop
   1. HDFS
   2. Yarn
5. MapReduce framework
6. Other tooling around hadoop
   1. Hive
   2. Pig
   3. Spark
   4. Presto
7. Data Serialisation and storage