Merge branch 'main' into git-py

CONTRIBUTING.md

@@ -0,0 +1 @@
+courses/CONTRIBUTING.md

NOTICE

@@ -0,0 +1,7 @@
+Copyright 2020 LinkedIn Corporation
+All Rights Reserved.
+Licensed under the BSD 2-Clause License (the "License").
+See LICENSE in the project root for license information.
+
+This product includes:
+* N/A

README.md

@@ -0,0 +1 @@
+courses/index.md

courses/CONTRIBUTING.md

@@ -0,0 +1,5 @@
+We realise that the initial content we created is just a starting point and our hope is that the community can help in the journey refining and extending the contents.
+
+As a contributor, you represent that the content you submit is not plagiarised. By submitting the content, you (and, if applicable, your employer) are licensing the submitted content to LinkedIn and the open source community subject to the BSD 2-Clause license.
+
+We suggest to open an issue first and seek advice for your changes before submitting a pull request.

courses/big_data/intro.md

@@ -1,35 +1,32 @@
-# School of SRE: Big Data
+# Big Data
 
-## Pre - Reads
+## Prerequisites
 
 - Basics of Linux File systems.
 - Basic understanding of System Design.
 
-## Target Audience
-
-The concept of Big Data has been around for years; most organizations now understand that if they capture all the data that streams into their businesses, they can apply analytics and get significant value from it. 
-This training material covers the basics of Big Data(using Hadoop) for beginners, who would like to quickly get started and get their hands dirty in this domain.
-
-## What to expect from this training 
+## What to expect from this course
 
 This course covers the basics of Big Data and how it has evolved to become what it is today. We will take a look at a few realistic scenarios where Big Data would be a perfect fit. An interesting assignment on designing a Big Data system is followed by understanding the architecture of Hadoop and the tooling around it. 
 
-## What is not covered under this training
+## What is not covered under this course
 
 Writing programs to draw analytics from data.
 
-## TOC:
+## Course Content
 
-1. Overview of Big Data
-2. Usage of Big Data techniques
-3. Evolution of Hadoop
-4. Architecture of hadoop
+### Table of Contents
+
+1. [Overview of Big Data](https://linkedin.github.io/school-of-sre/big_data/overview/)
+2. [Usage of Big Data techniques](https://linkedin.github.io/school-of-sre/big_data/overview/)
+3. [Evolution of Hadoop](https://linkedin.github.io/school-of-sre/big_data/evolution/)
+4. [Architecture of hadoop](https://linkedin.github.io/school-of-sre/big_data/architecture/)
     1. HDFS
     2. Yarn
-5. MapReduce framework
-6. Other tooling around hadoop
+5. [MapReduce framework](https://linkedin.github.io/school-of-sre/big_data/architecture/#mapreduce-framework)
+6. [Other tooling around hadoop](https://linkedin.github.io/school-of-sre/big_data/architecture/#other-tooling-around-hadoop)
     1. Hive
     2. Pig
     3. Spark
     4. Presto
-7. Data Serialisation and storage
+7. [Data Serialisation and storage](https://linkedin.github.io/school-of-sre/big_data/architecture/#data-serialisation-and-storage)

courses/git/git-basics.md

@@ -1,6 +1,6 @@
-# School Of SRE: Git
+# Git
 
-## Prerequisite
+## Prerequisites
 
 1. Have Git installed [https://git-scm.com/downloads](https://git-scm.com/downloads)
 2. Have taken any git high level tutorial or following LinkedIn learning courses
@@ -8,22 +8,22 @@
       - [https://www.linkedin.com/learning/git-branches-merges-and-remotes/](https://www.linkedin.com/learning/git-branches-merges-and-remotes/)
       - [The Official Git Docs](https://git-scm.com/doc)
 
-## What to expect from this training
+## What to expect from this course
 
 As an engineer in the field of computer science, having knowledge of version control tools becomes almost a requirement. While there are a lot of version control tools that exist today like SVN, Mercurial, etc, Git perhaps is the most used one and this course we will be working with Git. While this course does not start with Git 101 and expects basic knowledge of git as a prerequisite, it will reintroduce the git concepts known by you with details covering what is happening under the hood as you execute various git commands. So that next time you run a git command, you will be able to press enter more confidently!
 
-## What is not covered under this training
+## What is not covered under this course
 
 Advanced usage and specifics of internal implementation details of Git.
 
-## Training Content
+## Course Content
 
 ### Table of Contents
 
- 1. Git Basics
- 2. Working with Branches
- 3. Git with Github
- 4. Hooks
+ 1. [Git Basics](https://linkedin.github.io/school-of-sre/git/git-basics/#git-basics)
+ 2. [Working with Branches](https://linkedin.github.io/school-of-sre/git/branches/)
+ 3. [Git with Github](https://linkedin.github.io/school-of-sre/git/github-hooks/#git-with-github)
+ 4. [Hooks](https://linkedin.github.io/school-of-sre/git/github-hooks/#hooks)
 
 ## Git Basics
 

courses/git/github-hooks.md

@@ -1,4 +1,4 @@
-## Git with Github
+# Git with Github
 
 Till now all the operations we did were in our local repo while git also helps us in a collaborative environment. GitHub is one place on the internet where you can centrally host your git repos and collaborate with other developers.
 

courses/index.md

@@ -1 +1,25 @@
-Hello, World!!!
+# School of SRE
+![School of SRE](img/sos.png)
+Early 2019, we started visiting campuses to recruit the brightest minds to ensure LinkedIn and all the services that it is composed of is always available for everyone. This function at Linkedin falls in the purview of the Site Reliability Engineering team and Site Reliability Engineers ( SRE ) who are Software Engineers who specialize in reliability. SREs apply the principles of computer science and engineering to the design and development of computer systems: generally, large distributed ones.  
+
+As we continued on this journey we started getting a lot of questions from these campuses on what exactly site engineering roll entails? and, how could someone learn the skills and the disciplines involved to become a successful site engineer?   Fast forward a few months, and a few of these campus students had joined LinkedIn either as Interns or as full time engineers to become a part of the Site Engineering team, we also had a few lateral hires who joined our organization who were not from a traditional SRE background.  That's when a few of us got together and started to think about how we can on board new new graduate engineers to the site engineering team.
+
+There is a vast amount of resources scattered throughout the web on what are the roles and responsibilities of an SREs, how to monitor site health, handling incidents, maintain SLO/SLI etc. But there are very few resources out there guiding someone on what all basic skill sets one has to acquire as a beginner. Because of the lack of these resources we felt that individuals are having a tough time getting into open positions in the industry. We created School Of SRE as a starting point for anyone wanting to build their career in the role of SRE. 
+
+In this course we are focusing on building strong foundational skills. The course is  structured in a way to provide more real life examples and how learning each of the topics can play a bigger role in your day to day SRE life. Currently we are covering the following topics under the School Of SRE:
+
+- Fundamentals Series
+    - [Linux Basics](https://linkedin.github.io/school-of-sre/linux_basics/intro/)
+    - [Git](https://linkedin.github.io/school-of-sre/git/git-basics/)  
+    - [Linux Networking](https://linkedin.github.io/school-of-sre/linux_networking/intro/)
+- [Python and Web](https://linkedin.github.io/school-of-sre/python_web/intro/)
+- Data
+    - Relational databases (MySQL)
+    - NoSQL concepts 
+    - [Big Data](https://linkedin.github.io/school-of-sre/big_data/intro/)
+- [Systems Design](https://linkedin.github.io/school-of-sre/systems_design/intro/)
+- [Security](https://linkedin.github.io/school-of-sre/security/intro/)
+
+We believe continuous learning will help in acquiring deeper knowledge and competencies in order to expand your skill sets, every module has added reference which could be a guide for further learning.  Our hope is that by going through these modules we should be able build the essential skills required for a Site Reliability Engineer.
+
+At Linkedin we are using this curriculum for onboarding our non-traditional hires and new college grads to the SRE role. We had multiple rounds of successful onboarding experience with the new members and helped them to be productive in a very short period of time. This motivated us to opensource these contents for helping other organisations onboarding new engineers to the role and individuals to get into the role.  We realise that the initial content we created is just a starting point and our hope is that the community can help in the journey refining and extending the contents.

courses/linux_basics/command_line_basics.md

@@ -0,0 +1,459 @@
+# Command Line Basics
+
+## What is a command ?
+
+
+A command is a program that tells the operating system to perform
+specific work. Programs are stored as files in linux. Therefore, a
+command is also a file which is stored somewhere on the disk.
+
+Commands may also take additional arguments as input from the user.
+These arguments are called command line arguments. Knowing how to use
+the commands is important and there are many ways to get help in Linux,
+especially for commands. Almost every command will have some form of
+documentation, most commands will have a command-line argument -h or
+\--help that will display a reasonable amount of documentation. But the
+most popular documentation system in Linux is called man pages - short
+for manual pages.
+
+Using \--help to show the documentation for ls command.
+
+![](images/linux/commands/image19.png)
+
+## File System Organization
+
+The linux file system has a hierarchical (or tree-like) structure with
+its highest level directory called root ( denoted by / ). Directories
+present inside the root directory stores file related to the system.
+These directories in turn can either store system files or application
+files or user related files.
+
+![](images/linux/commands/image17.png)
+  
+  bin   | The executable program of most commonly used commands reside in bin directory  
+  sbin  | This directory contains programs used for system administration.
+  home  | This directory contains user related files and directories.
+  lib   | This directory contains all the library files
+  etc   | This directory contains all the system configuration files
+  proc  | This directory contains files related to the running processes on the system
+  dev   | This directory contains files related to devices on the system
+  mnt   | This directory contains files related to mounted devices on the system
+  tmp   | This directory is used to store temporary files on the system
+  usr   |  This directory is used to store application programs on the system
+
+## Commands for Navigating the File System
+
+There are three basic commands which are used frequently to navigate the
+file system:
+
+- ls
+
+- pwd
+
+- cd
+
+We will now try to understand what each command does and how to use
+these commands. You should also practice the given examples on the
+online bash shell.
+
+### pwd (print working directory)
+
+At any given moment of time, we will be standing in a certain directory.
+To get the name of the directory in which we are standing, we can use
+the pwd command in linux.
+
+![](images/linux/commands/image2.png)
+
+We will now use the cd command to move to a different directory and then
+print the working directory.
+
+![](images/linux/commands/image20.png)
+
+### cd (change directory)
+
+The cd command can be used to change the working directory. Using the
+command, you can move from one directory to another.
+
+In the below example, we are initially in the root directory. we have
+then used the cd command to change the directory.
+
+![](images/linux/commands/image3.png)
+
+### ls (list files and directories)**
+
+The ls command is used to list the contents of a directory. It will list
+down all the files and folders present in the given directory.
+
+If we just type ls in the shell, it will list all the files and
+directories present in the current directory.
+
+![](images/linux/commands/image7.png)
+
+We can also provide the directory name as argument to ls command. It
+will then list all the files and directories inside the given directory.
+
+![](images/linux/commands/image4.png)
+
+## Commands for Manipulating Files
+
+There are four basic commands which are used frequently to manipulate
+files:
+
+- touch
+
+- mkdir
+
+- cp
+
+- mv
+
+- rm
+
+We will now try to understand what each command does and how to use
+these commands. You should also practice the given examples on the
+online bash shell.
+
+### touch (create new file)
+
+The touch command can be used to create an empty new file.
+This command is very useful for many other purposes but we will discuss
+the simplest use case of creating a new file.
+
+General syntax of using touch command
+
+```
+touch <file_name>
+```
+
+![](images/linux/commands/image9.png)
+
+### mkdir (create new directories)
+
+The mkdir command is used to create directories.You can use ls command
+to verify that the new directory is created.
+
+General syntax of using mkdir command
+
+```
+mkdir <directory_name>
+```
+
+![](images/linux/commands/image11.png)
+
+### rm (delete files and directories)
+
+The rm command can be used to delete files and directories. It is very
+important to note that this command permanently deletes the files and
+directories. It's almost impossible to recover these files and
+directories once you have executed rm command on them successfully. Do
+run this command with care.
+
+General syntax of using rm command:
+
+```
+rm <file_name>
+```
+
+Let's try to understand the rm command with an example. We will try to
+delete the file and directory we created using touch and mkdir command
+respectively.
+
+![](images/linux/commands/image18.png)
+
+### cp (copy files and directories)
+
+The cp command is used to copy files and directories from one location
+to another. Do note that the cp command doesn't do any change to the
+original files or directories. The original files or directories and
+their copy both co-exist after running cp command successfully.
+
+General syntax of using cp command:
+
+```
+cp <source_path> <destination_path>
+```
+
+We are currently in the '/home/runner' directory. We will use the mkdir
+command to create a new directory named "test_directory". We will now
+try to copy the "\_test_runner.py" file to the directory we created just
+now.
+
+![](images/linux/commands/image23.png)
+
+Do note that nothing happened to the original "\_test_runner.py" file.
+It's still there in the current directory. A new copy of it got created
+inside the "test_directory".
+
+![](images/linux/commands/image14.png)
+
+We can also use the cp command to copy the whole directory from one
+location to another. Let's try to understand this with an example.
+
+![](images/linux/commands/image12.png)
+
+We again used the mkdir command to create a new directory called
+"another_directory". We then used the cp command along with an
+additional argument '-r' to copy the "test_directory".
+
+**mv (move files and directories)**
+
+The mv command can either be used to move files or directories from one
+location to another or it can be used to rename files or directories. Do
+note that moving files and copying them are very different. When you
+move the files or directories, the original copy is lost.
+
+General syntax of using mv command:
+
+```
+mv <source_path> <destination_path>
+```
+
+In this example, we will use the mv command to move the
+"\_test_runner.py" file to "test_directory". In this case, this file
+already exists in "test_directory". The mv command will just replace it.
+**Do note that the original file doesn't exist in the current directory
+after mv command ran successfully.**
+
+![](images/linux/commands/image26.png)
+
+We can also use the mv command to move a directory from one location to
+another. In this case, we do not need to use the '-r' flag that we did
+while using the cp command. Do note that the original directory will not
+exist if we use mv command.
+
+One of the important uses of the mv command is to rename files and
+directories. Let's see how we can use this command for renaming.
+
+We have first changed our location to "test_directory". We then use the
+mv command to rename the ""\_test_runner.py" file to "test.py".
+
+![](images/linux/commands/image29.png)
+
+## Commands for Viewing Files
+
+There are three basic commands which are used frequently to view the
+files:
+
+- cat
+
+- head
+
+- tail
+
+We will now try to understand what each command does and how to use
+these commands. You should also practice the given examples on the
+online bash shell.
+
+We will create a new file called "numbers.txt" and insert numbers from 1
+to 100 in this file. Each number will be in a separate line.
+
+![](images/linux/commands/image21.png)
+
+Do not worry about the above command now. It's an advanced command which
+is used to generate numbers. We have then used a redirection operator to
+push these numbers to the file. We will be discussing I/O redirection in the
+later sections.
+
+
+### cat
+
+The most simplest use of cat command is to print the contents of the file on
+your output screen. This command is very useful and can be used for many
+other purposes. We will study about other use cases later.
+
+![](images/linux/commands/image1.png)
+
+You can try to run the above command and you will see numbers being
+printed from 1 to 100 on your screen. You will need to scroll up to view
+all the numbers.
+
+### head
+
+The head command displays the first 10 lines of the file by default. We
+can include additional arguments to display as many lines as we want
+from the top.
+
+In this example, we are only able to see the first 10 lines from the
+file when we use the head command.
+
+![](images/linux/commands/image15.png)
+
+By default, head command will only display the first 10 lines. If we
+want to specify the number of lines we want to see from start, use the
+'-n' argument to provide the input.
+
+![](images/linux/commands/image16.png)
+
+### tail
+
+The tail command displays the last 10 lines of the file by default. We
+can include additional arguments to display as many lines as we want
+from the end of the file.
+
+![](images/linux/commands/image22.png)
+
+By default, the tail command will only display the last 10 lines. If we
+want to specify the number of lines we want to see from the end, use '-n'
+argument to provide the input.
+
+![](images/linux/commands/image10.png)
+
+In this example, we are only able to see the last 5 lines from the file
+when we use the tail command with explicit -n option.
+
+
+## Echo Command in Linux
+
+The echo command is one of the simplest commands that is used in the
+shell. This command is equivalent to what we have <print> in other
+programming languages.
+
+The echo command prints the given input string on the screen.
+
+![](images/linux/commands/image24.png)
+
+## Text Processing Commands
+
+In the previous section, we learned how to view the content of a file.
+In many cases, we will be interested in performing the below operations:
+
+- Print only the lines which contain a particular word(s)
+
+- Replace a particular word with another word in a file
+
+- Sort the lines in a particular order
+
+There are three basic commands which are used frequently to process
+texts:
+
+- grep
+
+- sed
+
+- sort
+
+We will now try to understand what each command does and how to use
+these commands. You should also practice the given examples on the
+online bash shell.
+
+We will create a new file called "numbers.txt" and insert numbers from 1
+to 10 in this file. Each number will be in a separate line.
+
+![](images/linux/commands/image8.png)
+
+### grep
+
+The grep command in its simplest form can be used to search particular
+words in a text file. It will display all the lines in a file that
+contains a particular input. The word we want to search is provided as
+an input to the grep command.
+
+General syntax of using grep command:
+
+```
+grep <word_to_search> <file_name>
+```
+
+In this example, we are trying to search for a string "1" in this file.
+The grep command outputs the lines where it found this string.
+
+![](images/linux/commands/image5.png)
+
+### sed
+
+The sed command in its simplest form can be used to replace a text in a
+file.
+
+General syntax of using the sed command for replacement:
+
+```
+sed 's/<text_to_replace>/<replacement_text>/' <file_name>
+```
+
+Let's try to replace each occurrence of "1" in the file with "3" using
+sed command.
+
+![](images/linux/commands/image31.png)
+
+The content of the file will not change in the above
+example. To do so, we have to use an extra argument '-i' so that the
+changes are reflected back in the file.
+
+### sort
+
+The sort command can be used to sort the input provided to it as an
+argument. By default, it will sort in increasing order.
+
+Let's first see the content of the file before trying to sort it.
+
+![](images/linux/commands/image27.png)
+
+Now, we will try to sort the file using the sort command. The sort
+command sorts the content in lexicographical order.
+
+![](images/linux/commands/image32.png)
+
+The content of the file will not change in the above
+example.
+
+## I/O Redirection
+
+Each open file gets assigned a file descriptor. A file descriptor is an
+unique identifier for open files in the system. There are always three
+default files open, stdin (the keyboard), stdout (the screen), and
+stderr (error messages output to the screen). These files can be
+redirected.
+
+Everything is a file in linux -
+[https://unix.stackexchange.com/questions/225537/everything-is-a-file](https://unix.stackexchange.com/questions/225537/everything-is-a-file)
+
+Till now, we have displayed all the output on the screen which is the
+standard output. We can use some special operators to redirect the
+output of the command to files or even to the input of other commands.
+I/O redirection is a very powerful feature.
+
+In the below example, we have used the '>' operator to redirect the
+output of ls command to output.txt file.
+
+![](images/linux/commands/image30.png)
+
+In the below example, we have redirected the output from echo command to
+a file.
+
+![](images/linux/commands/image13.png)
+
+We can also redirect the output of a command as an input to another
+command. This is possible with the help of pipes.
+
+In the below example, we have passed the output of cat command as an
+input to grep command using pipe(\|) operator.
+
+![](images/linux/commands/image6.png)
+
+In the below example, we have passed the output of sort command as an
+input to uniq command using pipe(\|) operator. The uniq command only
+prints the unique numbers from the input.
+
+![](images/linux/commands/image28.png)
+
+I/O redirection -
+[https://tldp.org/LDP/abs/html/io-redirection.html](https://tldp.org/LDP/abs/html/io-redirection.html)
+
+## Applications in SRE Role
+
+- As a SRE, you will be required to perform some general tasks on these linux servers. You will also be using the command line when you are troubleshooting issues.
+
+- Moving from one location to another in the filesystem will require the help of ls, pwd and cd commands
+
+- You may need to search some specific information in the log files. Grep command would be very useful here. I/O redirection will become handy if you want to store the output in a file or pass it as an input to another command.
+
+- Tail command is very useful to view the latest data in the log file.
+
+## Useful courses and tutorials
+
+
+- [Edx linuxcourse](https://courses.edx.org/courses/course-v1:LinuxFoundationX+LFS101x+1T2020/course/) -
+   This video course can be very helpful in developing the basics of linux command line. This course is provided
+   in both free and paidmodes by edX. If you take the free course, you will not be able to access the assignments.
+
+- [https://linuxcommand.org/lc3_learning_the_shell.php](https://linuxcommand.org/lc3_learning_the_shell.php)