intermediate linux course SOS Level102 (#119)

* intermediate linux course SOS Level102

* adding absolute URLs

* Removing unwanted .md extension from absoulute URLs

* adding the entry in index.md

courses/index.md

@@ -28,6 +28,7 @@ In this course, we are focusing on building strong foundational skills. The cour
     -   [Security](https://linkedin.github.io/school-of-sre/level101/security/intro/)
 
 -   Level 102
+    -   [Linux Intermediate](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/introduction/)
     -   Linux Advanced
         -   [Containers and orchestration](https://linkedin.github.io/school-of-sre/level102/containerization_and_orchestration/intro/)
         -   [System Calls and Signals](https://linkedin.github.io/school-of-sre/level102/system_calls_and_signals/intro/)

courses/level102/linux_intermediate/archiving_backup.md

@@ -0,0 +1,80 @@
+
+# Archiving and Backup
+
+## Introduction
+One of the things SREs make sure of is the services are up all the time (at least 99.99% of the time), but the amount of data generated at each server running those services are immense. This data could be logs, user data in the database, or any other kind of metadata. Hence we need to compress, archive, rotate, and Backup the data in a timely manner for data safety and to make sure we don’t run out of space.
+
+## Archiving
+
+We usually archive the data that are no longer needed but are kept mostly for compliance purposes. This helps in storing the data into compressed format saving a lot of space. Below section is to familiarize with the archiving tools and commands.
+
+## gzip
+
+gzip is a program used to [<u>compress</u>](https://en.wikipedia.org/wiki/Data_compression) one or more files, it replaces the original file with a compressed version of the original file.
+
+![](images/image14.png)
+
+Here we can see that the *messages* log file is compressed to almost one-fifth of the original size and replaced with messages.gz. We can uncompress this file using [*<u>gunzip</u>*](https://linux.die.net/man/1/gunzip) command.
+
+## tar
+
+*tar* program is a tool for archiving files and directories into a single file (often called tarball). This tool is usually used to prepare archives of files before it is transferred to a long term backup server. *tar* doesn’t replace the existing files and folders but creates a new file with extension *.tar* . It provides lot of flag to choose from for archiving
+
+| Flags  | Description |
+| --- | --- |
+| -c  | Creates archive |
+| -x  | Extracts the archive |
+| -f  | Creates archive with the given filename |
+| -t  | Displays or lists files in archived file |
+| -u  | Archives and adds to an existing archive file |
+| -v  | Displays verbose information |
+| -A  | Concatenates the archived file |
+| -z  | Compresses the tar file using gzip |
+| -j  | Compresses the tar file using bzip2 |
+| -W  | Verifies an archive file |
+| -r  | Updates or adds file or directory in already existing .tar file |
+
+### Create an archive with files and folder
+
+Flag `c` is used for creating the archive where `f` is the filename.
+
+![](images/image24.png)
+
+### Listing files in the archive
+
+We can use flag `t` for listing out what an archive contains.
+
+![](images/image7.png)
+
+### Extract files from the archive
+
+We can use flag `x` to unarchive the archive.
+
+![](images/image26.png)
+
+## Backup
+
+Backup is a process of copying/duplicating the existing data, This backup can be used to restore the dataset in case of data loss. Data backup also becomes critical when the data is not needed in a day to day job but can be referred to as a source of truth and for compliance reasons in future. Different types of backup are :
+
+### Incremental backup
+
+Incremental backup is the backup of data since the last backup, this reduces data redundancy and storage efficiency.
+
+### Differential backup
+
+Sometimes our data keeps on modifying/updating. In that case we take backup of changes that occurred since the last backup called differential backup.
+
+### Network backup
+
+Network backup refers to sending out data over the network from the source to a backup destination in a client-server model. This backup destination can be centralized or decentralized. Decentralized backups are useful for disaster recovery scenarios.
+
+`rsync` is one of the linux command which sync up file from one server to the destination server over the network.
+
+![](images/image11.png)
+
+The syntax for *rsync* goes like `rsync \[options\] <source> <destination>`. We can locate the file on the path specified after `:` (colon) in the “*destination”*. If nothing is specified the default path is the home directory of the user used for backup. `/home/azureuser` in this case. You can always look for different options for rsync using the `man rsync` command.
+
+### Cloud Backup
+
+There are various third parties which provide the backup of data to the cloud. These cloud backups are much more reliable than stored backups on local machines or any server without RAID configuration as these providers manage redundancy of data, data recovery along with the data security. Two most widely used cloud backup options are Azure backup (from Microsoft) and Amazon Glacier backup (from AWS).
+

courses/level102/linux_intermediate/bashscripting.md

@@ -0,0 +1,278 @@
+# Bash Scripting
+
+## Introduction
+As an SRE, the Linux system sits at the core of our day to day work and so is bash scripting. It’s a scripting language that is run by Linux Bash Interpreter. Until now we have covered a lot of features mostly on a command line, now we will use this command line as an interpreter to write programs that will ease our day to day job as an SRE.
+
+## Writing the first bash script:
+
+We will start with a simple program, we will use Vim as the editor during the whole journey.
+
+```
+#!/bin/bash
+
+# This if my first bash script
+# Line starting with # is commented
+
+echo "Hello world!"
+```
+
+The first line of the script starting with “#!” is called she-bang. This is simply to let the system which interpreter to use while executing the script.
+
+Any Line starting with “#” (other than #!) is referred to as comments in script and is ignored by the interpreter while executing the script. Line 6 shows the “echo” command that we would be running.
+
+We will save this script as “firstscript.sh” and make the script executable using `chmod`.
+
+![](images/image16.png)
+
+Next thing is to run the script with the explicit path. We can see the desired “Hello World!” as output.
+
+## Taking user input and working with variables:
+
+Taking standard input using the `read` command and working with variables in bash.
+
+```
+#!/bin/bash
+
+#We will take standard input
+#Will list all files at the path
+#We will concate variable and string
+
+echo "Enter the path"
+read path
+
+echo "How deep in directory you want to go:"
+read depth
+
+echo "All files at path " $path
+
+du -d $depth -all -h $path
+```
+
+We are reading path in variable “*path*” and variable “*depth*” to list files and directories up to that depth. We concatenated strings with variables. We always use `$` (dollar-sign) to reference the value it contains.
+
+![](images/image28.png)
+
+We pass these variables to the `du` command to list out all the files and directories in that path upto the desired depth.
+
+## Exit status:
+
+Every command and script when it completes executing, returns an integer in the range from 0 to 255 to the system, this is called exit status. “0” denotes success of the command while non-zero return code usually indicates various kinds of errors.
+
+![](images/image6.png)
+
+We use `$?` special shell variable to get exit status of the last executed script or command.
+
+## Command line arguments and understanding If … else branching:
+
+Another way to pass some values to the script is using command line arguments. Usually command line arguments in bash are accessed by $ followed by the index. The 0th index refers to the file itself, `$1` to the first argument and so on. We use `$#` to check the count of arguments passed to the script.
+
+Making decisions in the programming language is it’s integral part, and to tackle different conditions we use if … else statements or some more nested variant of it.
+
+The below script uses multiple concepts in one script. The aim of the script is to get some properties of the file.
+
+Line 4 to 7 is the standard example of "if statement" in bash. Syntax is as explained below:
+```
+If [ condition ]; then
+
+If_block_to_execute
+
+else
+
+else_block_to_execute
+
+fi
+```
+
+fi is to close the if … else block. We are comparing count of argument($#) if it is equal to 1 or not. If not we prompt for only one argument and exit the script with status code 1(not a success). One or more if statements can exist without else statement but vice versa doesn’t make any sense.
+
+Operator -ne is used to compare two integers, read as “integer1 *not equal to* integer 2”. Other comparison operators are:
+
+| Operations | Description |
+| --- | --- |
+| **num1 -eq num2** | check if 1st number is equal to 2nd number |
+| **num1 -ge num2** | checks if 1st number is greater than or equal to 2nd number |
+| **num1 -gt num2** | checks if 1st number is greater than 2nd number |
+| **num1 -le num2** | checks if 1st number is less than or equal to 2nd number |
+| **num1 -lt num2** | checks if 1st number is less than 2nd number |
+
+```
+#!/bin/bash
+# This script evaluate the status of a file
+
+if [ $# -ne 1 ]; then
+    echo "Please pass one file name as argument"
+    exit 1
+fi
+
+FILE=$1
+if [ -e "$FILE" ]; then
+    if [ -f "$FILE" ]; then
+        echo "$FILE is a regular file."
+    fi
+    if [ -d "$FILE" ]; then
+        echo "$FILE is a directory."
+    fi
+    if [ -r "$FILE" ]; then
+        echo "$FILE is readable."
+    fi
+    if [ -w "$FILE" ]; then
+        echo "$FILE is writable."
+    fi
+    if [ -x "$FILE" ]; then
+        echo "$FILE is executable/searchable."
+    fi
+else
+    echo "$FILE does not exist"
+    exit 2
+fi
+
+exit 0
+```
+
+There are lots of file expressions to evaluate file,like in bash script “-e” in line 10 returns true if the file passed as argument exist, false otherwise. Below are the some widely used file expressions:
+
+| File Operations | Description |
+| --- | --- |
+| **-e file** | File exists |
+| **-d file** | File exists and is directory |
+| **-f file** | File exists and is regular file |
+| **-L file** | File exists and is symbolic link |
+| **-r file** | File exists and has readable permission |
+| **-w file** | File exists and has writable permission |
+| **-x file** | File exists and has executable permission |
+| **-s file** | File exists and size is greater than zero |
+| **-S file** | File exists and is a network socket. |
+
+![](images/image17.png)
+
+Exit status is 2 when the file is not found. And if the file is found it prints out the properties it holds with exit status 0(success).
+
+## Looping over to do a repeated task.
+
+We usually come up with tasks that are mostly repetitive, looping helps us to code those repetitive tasks in a more formal manner. There are different types of loop statement we can use in bash:
+
+| Loop | Syntax |
+| --- | --- |
+| while | while \[ expression \]<br><br>do <br><br>    \[ while\_block\_to_execute \]<br><br>done |
+| for | for variable in 1,2,3 .. n<br><br>do <br><br>    \[ for\_block\_to_execute \]<br><br>done |
+| until | until \[ expression \] <br><br>do <br><br>    \[ until\_block\_to_execute \]<br><br>done |
+
+```
+!/bin/bash
+#Script to monitor the server
+
+hosts=`cat host_list`
+
+while true
+do
+    for i in $hosts
+    do
+    	h="$i"
+    	ping -c 1 -q "$h" &>/dev/null
+    	if [ $? -eq 0 ]
+    	then
+        	echo `date` "server $h alive"
+    	else
+        	echo `date` "server $h is dead"
+    	fi
+    done
+    sleep 60
+done
+```
+
+Monitoring a server is an important part of being an SRE. The file “host_list” contains the list of host which we want to monitor.
+
+We used an infinite “while” loop that will sleep every 60seconds. And for each host in the host_list we want to ping that host and check if that ping was successful with its exit status, if it’s successful we say server is live or it’s dead.
+
+![](images/image13.png)
+
+The output of the script shows it is running every minute with the timestamp.
+
+## Function
+
+Developers always try to make their applications/programs in modular fashion so that they don’t have to write the same code every time and everywhere to carry out similar tasks. Functions help us achieve this.
+
+We usually call functions with some arguments and expect result based on that argument.
+
+The backup process we discussed in earlier section, we will try to automate that process using the below script and also get familiar with some more concepts like string comparison, functions and logical AND and OR operations.
+
+In the below code “log_backup” is a function which won’t be executed until it is called.
+
+Line37 will be executed first where we will check the no. of arguments passed to the script.
+
+There are many logical operators like AND,OR, XOR etc.
+
+| Logical Operator | Symbol |
+| --- | --- |
+| AND | &&  |
+| OR  | \|  |
+| NOT | !   |
+
+Passing the wrong argument to script “backup.sh” will prompt for correct usage. We have to pass whether we want to have incremental backup of the directory or the full backup along with the path of the directory we want to backup. If we want the incremental backup we will an additional argument as a meta file which is used to store the information of previous backed up files.(usually a metafile is .snar extension).
+
+```
+#!/bin/bash
+#Scripts to take incremental and full backup
+
+backup_dir="/mnt/backup/"
+time_stamp="`date +%d-%m-%Y-%Hh-%Mm-%Ss`"
+
+log_backup(){
+	if [ $# -lt 2 ]; then
+		echo "Usage: ./backup.sh [backup_type] [log_path]"
+		exit 1;
+	fi
+	if [ $1 == "incremental" ]; then
+		if [ $# -ne 3 ]; then
+			echo "Usage: ./backup.sh [backup_type] [log_path] [meta_file]"
+			exit 3;
+		fi
+		tar --create --listed-incremental=$3 --verbose --verbose --file="${backup_dir}incremental-${time_stamp}.tar" $2
+		if [ $? -eq 0 ]; then
+			echo "Incremental backup succesful at '${backup_dir}incremental-${time_stamp}.tar'"
+		else
+			echo "Incremental Backup Failure"
+		fi
+
+	elif [ $1 == "full" ];then
+		tar cf "${backup_dir}fullbackup-${time_stamp}.tar" $2
+		if [ $? -eq 0 ];then
+			echo "Full backup successful at '${backup_dir}fullbackup-${time_stamp}.tar'"
+		else
+			echo "Full Backup Failure"
+		fi
+	else
+		echo "Unknown parameter passed"
+		echo "Usage: ./backup.sh [incremental|full] [log_path]"
+		exit 2;
+	fi
+}
+
+if [ $# -lt 2 ] || [ $# -gt 3 ];then
+	echo "Usage: ./backup.sh [incremental|full] [log_path]"
+	exit 1
+elif [ $# -eq 2 ];then
+	log_backup $1 $2
+elif [ $# -eq 3 ];then
+	log_backup $1 $2 $3
+fi
+exit 0
+```
+
+Passing all 3 arguments for incremental backup will take incremental backup at “/mnt/backup/” with each archive having timestamp concatenated to each file.
+
+![](images/image20.png)
+
+The arguments passed inside the function can be accessed via `$` followed by the index. The 0th index refers to the function itself,
+
+`$1` to the first argument and so on. We use `#$` to check the count of arguments passed to the function.
+
+Once we pass the string “incremental” or “full” it gets compared inside the function and the specific block is executed. Below are some more operations that can be performed over strings.
+
+| String Operations | Description |
+| --- | --- |
+| string1 == string2 | Returns true if string1 equals string 2 otherwise false. |
+| string1 != string2 | Returns true if string NOT equal string 2 otherwise false. |
+| string1 ~= regex | Returns true if string1 matches the extended regular expression. |
+| -z string | Returns true if string length is zero otherwise false. |
+| -n string | Returns true if string length is non-zero otherwise false. |

courses/level102/linux_intermediate/conclusion.md

@@ -0,0 +1,10 @@
+# Conclusion
+
+Understanding package management is very crucial as an SRE, we always want the right set of software with their compatible versions to work in harmony to drive the big infrastructure and organization. 
+
+We also saw how we can configure and use storage drives and how we can have redundancy of data using RAID to avoid the data loss, how data is placed over disk and use of file systems.
+
+Archiving and Backup is also a crucial part of being an SRE, It’s our responsibility to keep the data safe and in a more efficient manner. 
+
+Bash is very useful to automate the day to day toil that an SRE stumbles into. The above walkthrough of bash gives us an idea to get started, but mere reading through it won’t take you much further. I believe “taking action and practicing the topic” would give you confidence and will help you become a better SRE. 
+

courses/level102/linux_intermediate/introduction.md

@@ -0,0 +1,124 @@
+
+# Linux-Intermediate
+
+## Prerequisites 
+
+- Expect to have gone through the School Of SRE [<u>Linux Basics</u>](https://linkedin.github.io/school-of-sre/level101/linux_basics/intro/).
+
+## What to expect from this course
+
+ This course is divided into two sections. In the first section we will cover where we left off the Linux Basics, earlier in the School of SRE curriculum, we will deep dive into some of the more advanced linux commands and concepts.
+ 
+ In this second section we will discuss how we use Bash scripting in day to day work, automation and toil reduction as an SRE with the help of real life examples of any SRE.
+
+## What is not covered under this course
+
+This course aims to make you familiar with the intersection of Linux commands, shell scripting and how SRE uses it. We would not be covering Linux internals.
+
+
+## Lab Environment Setup
+
+- Install docker on your system. [<u>https://docs.docker.com/engine/install/</u>](https://docs.docker.com/engine/install/)
+    
+- We would be using RedHat Enterprise Linux (RHEL) 8.
+
+![](images/image1.png)
+
+- We would be running most of the commands in the above docker container.
+
+\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
+
+## Course Content 
+
+[<u>Package Management</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/package_management/)
+
+- [<u>Package:</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/package_management/#package)
+    
+- [<u>Dependencies</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/package_management/#dependencies)
+    
+- [<u>Repository</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/package_management/#repository)
+    
+- [<u>High Level and Low-Level Package management tools</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/package_management/#high-level-and-low-level-package-management-tools)
+    
+
+[<u>Storage Media</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/)
+
+- [<u>Listing the mounted storage devices</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#listing-the-mounted-storage-devices)
+    
+- [<u>Creating a FileSystem</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#creating-a-filesystem)
+    
+- [<u>Mounting the device</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#mounting-the-device)
+    
+- [<u>Unmounting the device</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#unmounting-the-device)
+    
+- [<u>Making it easier with /etc/fstab file?</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#making-it-easier-with-etcfstab-file)
+    
+- [<u>Checking and Repairing FS</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#checking-and-repairing-fs)
+    
+- [<u>RAID</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#raid)
+    
+    - [<u>RAID levels</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#raid-levels)
+        
+    - [<u>RAID 0 (Striping)</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#raid-0-striping)
+        
+    - [<u>RAID 1(Mirroring)</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#raid-1mirroring)
+        
+    - [<u>RAID 5(Striping with distributed parity)</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#raid-5striping-with-distributed-parity)
+        
+    - [<u>RAID 6(Striping with double parity)</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#raid-6striping-with-double-parity)
+        
+    - [<u>RAID 10(RAID 1+0 : Mirroring and Striping)</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#raid-10raid-10-mirroring-and-striping)
+        
+    - [<u>Commands to monitor RAID</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#commands-to-monitor-raid)
+        
+- [<u>LVM</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/storage_media/#lvm)
+    
+
+[<u>Archiving and Backup</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/)
+
+- [<u>Archiving</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#archiving)
+    
+    - [<u>gzip</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#gzip)
+        
+    - [<u>tar</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#tar)
+        
+    - [<u>Create an archive with files and folder</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#create-an-archive-with-files-and-folder)
+        
+    - [<u>Listing files in the archive</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#listing-files-in-the-archive)
+        
+    - [<u>Extract files from the archive</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#extract-files-from-the-archive)
+        
+- [<u>Backup</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#backup)
+    
+    - [<u>Incremental backup</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#incremental-backup)
+        
+    - [<u>Differential backup</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#differential-backup)
+        
+    - [<u>Network backup</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#network-backup)
+        
+    - [<u>Cloud Backup</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/archiving_backup/#cloud-backup)
+        
+[<u>Introduction to Vim</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/introvim/)
+ 
+- [<u>Opening a file and using insert mode</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/introvim/#opening-a-file-and-using-insert-mode)
+    
+- [<u>Saving a file</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/introvim/#saving-a-file)
+    
+- [<u>Exiting the VIM editor</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/introvim/#exiting-the-vim-editor)
+    
+[<u>Bash Scripting</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/bashscripting/)
+
+- [<u>Writing the first bash script</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/bashscripting/#writing-the-first-bash-script)
+    
+- [<u>Taking user input and working with variables</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/bashscripting/#taking-user-input-and-working-with-variables)
+    
+- [<u>Exit status</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/bashscripting/#exit-status)
+    
+- [<u>Command line arguments and understanding If … else branching</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/bashscripting/#command-line-arguments-and-understanding-if-..-else-branching)
+    
+- [<u>Looping over to do a repeated task</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/bashscripting/#looping-over-to-do-a-repeated-task.)
+    
+- [<u>Function</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/bashscripting/#function)
+    
+[<u>Conclusion</u>](https://linkedin.github.io/school-of-sre/level102/linux_intermediate/conclusion/)
+        

courses/level102/linux_intermediate/introvim.md

@@ -0,0 +1,36 @@
+
+# Introduction to Vim
+
+## Introduction
+As an SRE we several times log into into the servers and make changes to the config file, edit and modify scripts and the editor which comes handy and available in almost all linux distribution is Vim. Vim is an open-source and free command line editor, widely accepted and used. We will see some basics of how to use vim for creating and editing files. This knowledge will help us in understanding the next section, Scripting.
+
+## Opening a file and using insert mode
+
+We use the command *`vim filename`* to open a file *`filename`*. The terminal will open an editor but once you start writing, it won’t work. It’s because we are not in "INSERT" mode in vim.
+
+Press ***`i`*** and get into insert mode and start writing.
+
+![](images/image2.png)
+
+You will see on the bottom left “INSERT” after pressing “***i***” . You can use *`ESC`” key to get back to normal mode.
+
+## Saving a file
+
+After you insert your text in INSERT mode press ESC(escape) key on your keyboard to get out of it. Press `:`(colon shift +;) and press ***`w`*** and hit enter, the text you entered will get written in the file.
+
+![](images/image19.png)
+
+## Exiting the VIM editor
+
+Exiting vim can get real challenging for the beginners. There are various ways you can exit the Vim like exit without saving the work, exit with saving the work.
+
+Try below commands after exiting insert mode and pressing ***`:`***(colon).
+
+| Vim Commands | Description |
+| --- | --- |
+| **:q** | Exit the file but won’t exit if file has unsaved changes |
+| **:wq** | Write(save) and exit the file. |
+| **:q!** | Exit without saving the changes. |
+
+This is basic we would be needing in bash scripting in the next section. You can always visit tutorial for learning more. For quick practice of vim commands visit: [<u>https://www.openvim.com/</u>](https://www.openvim.com/)
+

courses/level102/linux_intermediate/package_management.md

@@ -0,0 +1,38 @@
+# Package Management
+## Introduction 
+
+One of the main features of any operating system is the ability to run other programs and softwares, and hence Package management comes into picture. Package management is a method of installing and maintaining software programs on any operating system.
+
+## Package
+
+In the early days of Linux, one had to download source code of any software and compile it to install and run the software. As the Linux space became more mature, it is  understood the software landscape is very dynamic and started distributing software in the form of packages. Package file is a compressed collection of files that contains software, its dependencies, installation instructions and metadata about the package.
+
+## Dependencies
+
+It is rare that a software package is stand-alone, it depends on the different software, libraries and modules. These subroutines are stored and made available in the form of shared libraries which may serve more than one program. These shared resources are called dependencies. Package management does this hard job of resolving dependencies and installing them for the user along with the software.
+
+## Repository
+
+Repository is a storage location where all the packages, updates, dependencies are stored. Each repository can contain thousands of software packages hosted on a remote server intended to be installed and updated on linux systems. We usually update the package information ( *often referred to as metadata*) by running “*sudo dnf update”.*
+
+![](images/image29.png)
+
+Try out *`sudo dnf repolist all`* to list all the repositories.
+
+We usually add repositories for installing packages from third party vendors.
+
+> dnf config-manager --add-repo http://www.example.com/example.repo
+
+## High Level and Low-Level Package management tools
+
+There are mainly two types of packages management tools:
+
+> 1\. *Low-level tools*: This is mostly used for installing, removing and upgrading package files.
+> 
+> 2\. *High-Level tools*: In addition to Low-level tools, High-level tools do metadata searching and dependency resolution as well.
+
+| Linux Distribution | Low-Level Tools | High-Level tools |
+| --- | --- | --- |
+| Debian | dpkg | apt-get |
+| Fedora, RedHat | dnf | dnf |
+

courses/level102/linux_intermediate/storage_media.md

@@ -0,0 +1,328 @@
+# Storage Media
+## Introduction
+
+Storage media are devices which are used to store data and information. Linux has amazing capabilities when it comes to handling external devices including storage devices. There are many kinds of storage devices physical storage devices like hard drives, virtual storage devices like RAID or LVM, network storage and so on.
+
+In this section we will learn to work with any storage device and configure it to our needs.
+
+## Listing the mounted storage devices:
+
+We can use command **`mount`** to list all the storage devices mounted to your computer.
+
+![](images/image30.png)
+
+The format in which we see above output is:
+
+*`device`* on *`mount_point`* type *`file\_system\_type (options)`*
+
+For example in the first line the device virtual *sysfs* is mounted at */sys* path and has a *sysfs* file system. Now let’s see what and how a filesystem is created.
+
+## Creating a FileSystem
+
+Imagine a disk where all the data stored in the disk is in the form of one large chunk, there is nothing to figure out where one piece of data starts and ends, which piece of data is located at which place of the whole chunk of data and hence the File System comes into picture. File System(fs) is responsible for data storage, indexing and retrieval on any storage device.
+
+Below are the most popularly used file systems:
+
+| FS Type | Description |
+| --- | --- |
+| FAT | File Allocation Table, initially used on DOS and Microsoft Windows and now widely used for portable USB storage |
+| NTFS | (New Technology File System) Used on Microsoft’s Windows based operating systems |
+| ext | Extended file system, designed for Linux systems. |
+| ext4 | Fourth extended filesystem, is a journaled file system that is commonly used by the Linux kernel. |
+| HFS | Hierarchical File System, in use until HFS+ was introduced on Mac OS 8.1. |
+| HFS+ | Supports file system journaling, enabling recovery of data after a system crash. |
+| NFS | Network File System originally from Sun Microsystems is the standard in UNIX-based networks. |
+
+We will try to create an *ext4* file system which is linux native fs using [*<u>mkfs</u>*](https://man7.org/linux/man-pages/man8/mkfs.8.html).
+
+**Discalimer: Run this command on empty disk as this will wipe out the existing data.**
+
+![](images/image10.png)
+
+Here the device */dev/sdb1* is formatted and it’s filesystem is changed to *ext4*.
+
+## Mounting the device:
+
+In Linux systems all files are arranged in a tree structure with (/) as root. Mounting a fs simply means making that fs accessible to a certain point in the Linux directory tree.
+
+We need a mount point(location) where we want to mount the above formatted device.
+
+![](images/image12.png)
+
+We created a mount point */mount* and used the *[<u>mount</u>](https://man7.org/linux/man-pages/man8/mount.8.html)* command to attach the filesystem. Here *-t* flag specifies what is the fs type and after that the */dev/sdb1* (device name) and /mount (mount point we created earlier).
+
+## Unmounting the device:
+
+Now let’s see how we can unmount the device, which is equally important if we have removable storage media and want to mount on another host. We use [***<u>umount</u>***](https://man7.org/linux/man-pages/man8/umount.8.html) for unmounting the device.
+
+![](images/image15.png)
+
+Our first attempt did not unmount the /sdb1 because we were inside the storage device and it was being used. Once we jumped back to the home directory we were successfully able to unmount the device.
+
+## Making it easier with /etc/fstab file?
+
+In our production environment, we can have servers with many storage devices that need to be mounted, and it is not feasible to mount each device using the command every time we reboot the system. To ease this burden, we can make use of configuration table called “fstab” usually found in ***`/etc/fstab`*** on Linux systems.
+
+![](images/image22.png)
+
+Here on the first line we have */dev/mapper/rootvg-rootlv (storage device*) mounted on */ (root mount point) which has the xfs filesystem type* followed by options.
+
+We can run *`mount -a`* to reload this file after making changes.
+
+## Checking and Repairing FS
+
+Filesystems encounter issues in case of any hardware failure, power failure and sometimes due to improper shutdown. Linux usually checks and repairs the corrupted disk if any during startup. We can also manually check for filesystem corruption using the command [***<u>fsck</u>***](https://man7.org/linux/man-pages/man8/fsck.8.html).
+
+![](images/image25.png)
+
+We can repair the same filesystem using *`fsck -y /dev/sdb1`*.
+
+There are error codes attached to each kind of file system error ,and A sum of active errors is returned.
+
+| Error Codes | Description |
+| --- | --- |
+| 0   | No errors |
+| 1   | Filesystem errors corrected |
+| 2   | System should be rebooted |
+| 4   | Filesystem errors left uncorrected |
+| 8   | Operational error |
+| 16  | Usage or syntax error |
+| 32  | Checking canceled by user request |
+| 128 | Shared-library error |
+
+In the above fs check we got return code as 12 which is the sum of error code 8(operational error) and 4(uncorrected FS error).
+
+## RAID
+
+RAID or “Redundant Arrays of Independent Disks” is a technique that distributes I/O across multiple disks to achieve increased performance and data redundancy. RAID has the ability to increase overall disk performance and survive disk failures. Software RAID uses the computer’s CPU to carry out RAID operations whereas hardware RAID uses specialized processors, on disk controllers, to manage the disks. Three essential features of RAID are mirroring, striping and parity.
+
+## RAID levels
+
+The below section discusses the RAID levels that are commonly used. For information on all RAID levels, please refer to [<u>here</u>](https://en.wikipedia.org/wiki/RAID) .
+
+### RAID 0 (Striping)
+
+Striping is the method by which data is split up into “blocks” and written across all the disks present in the array. By spreading data across multiple drives, it means multiple disks can access the file, resulting in faster read/write speeds. The first disk in the array is not reused until an equal amount of data is written to each of the other disks in the array.
+
+![](images/image9.png)
+
+Advantages
+
+- It can be easily implemented.
+    
+- Bottlenecks caused due to I/O operations from the same disk are avoided, increasing the performance of such operations.
+    
+
+Disadvantages
+
+- It does not offer any kind of redundancy. If any one of the disks fails, then the data of the entire disk is lost and cannot be recovered.
+
+Use cases
+
+RAID 0 can be used for systems with non-critical data that has to be read at high speed, such as a video/audio editing station or gaming environments.
+
+### RAID 1(Mirroring)
+
+Mirroring writes a copy of data to each disk which is part of the array. This means that the data is written as many times as disks in the array . It stores an exact replica of all data on a separate disk or disks. As expected, this would result in a slow write performance compared to that of a single disk. On the other hand, read operations can be done parallelly improving read performance.
+
+![](images/image21.png)
+
+Advantages
+
+- RAID 1 offers a better read performance than RAID 0 or single disk.
+    
+- It can survive multiple disk failures without the need for special data recovery algorithms
+    
+
+Disadvantages
+
+- It is costly since the effective storage capacity is only half of the number of disks due to replication of data.
+
+Use cases
+
+Applications that require low downtime but can have a slight hit on write performance.
+
+### RAID 4(Striping with dedicated parity)
+
+RAID 4 works uses block-level striping (data can be striped in blocks of a variety of sizes depending on the applications and data to be stored) and a dedicated drive used to store parity information.The parity information is generated by an algorithm every time data is written to an array disk. The use of a parity bit is a way of adding checksums into data that can enable the target device to determine whether the data has been received correctly. In the event of a drive failure , the algorithm can be reversed and missing data can be generated based on the remaining data and parity information.
+
+![](images/image4.png)
+
+Advantages
+
+- Each drive in a RAID 4 array operates independently so I/O requests take place in parallel, speeding up performance over previous RAID levels.
+    
+- It can survive multiple disk failures without the need for special data recovery algorithms
+    
+
+Disadvantages
+
+- A minimum of 3 disks is required for setup.
+    
+- It needs hardware support for parity calculation.
+    
+- Write speeds are slow since parity relies on a single disk drive and carry out modifications of parity blocks for each I/O session.
+    
+
+Use cases
+
+Operations dealing with really large files – when sequential read and write data process is used
+
+### RAID 5(Striping with distributed parity)
+
+RAID 5 is similar to RAID 4, except that the parity information is spread across all drives in the array. This helps reduce the bottleneck inherent in writing parity information to a single drive during each write operation. RAID 5 is the most common secure RAID level.
+
+![](images/image23.png)
+
+Advantages
+
+- Read data transactions are fast as compared to write data transactions that are somewhat slow due to the calculation of parity.
+    
+- Data remains accessible even after drive failure and during replacement of a failed hard drive because the storage controller rebuilds the data on the new drive.
+    
+
+Disadvantages
+
+- RAID 5 requires a minimum of 3 drives and can work up to a maximum of 16 drives
+    
+- It needs hardware support for parity calculation.
+    
+- More than two drive failures can cause data loss.
+    
+
+Use cases
+
+File storage and application servers, such as email, general storage servers, etc.
+
+### RAID 6(Striping with double parity)
+
+RAID 6 is similar to RAID 5 with an added advantage of double distributed parity, which provides fault tolerance up to two failed drives.
+
+![](images/image5.png)
+
+Advantages
+
+- Read data transactions are fast.
+    
+- This provides a fault tolerance up to 2 failed drives.
+    
+- RAID 6 is more resilient than RAID 5.
+    
+
+Disadvantages
+
+- Write data transactions are slow due to double parity.
+    
+- Rebuilding the RAID array takes a longer time because of complex structure.
+    
+
+Use cases
+
+Office automation, online customer service, and applications that require very high availability.
+
+### RAID 10(RAID 1+0 : Mirroring and Striping)
+
+RAID 10 is a combination of RAID 0 and RAID 1. It means that both mirroring and striping in one single RAID array.
+
+![](images/image3.png)
+
+Advantages
+
+- Rebuilding the RAID array is fast.
+    
+- Read and write operations performance are good.
+    
+
+Disadvantages
+
+- Just like RAID 1, only half the drive capacity is available.
+    
+- It can be expensive to implement RAID 10.
+    
+
+Use cases
+
+Transactional databases with sensitive information that require high performance and high data security.
+
+## Commands to monitor RAID
+
+The command `cat /proc/mdstat` will give the status of a software RAID. Let us examine the output of the command:
+
+```
+Personalities : [raid1]
+
+md0 : active raid1 sdb1[2] sda1[0]
+
+10476544 blocks super 1.1 [2/2] [UU]
+
+bitmap: 0/1 pages [0KB], 65536KB chunk
+
+md1 : active raid1 sdb2[2] sda2[0]
+
+10476544 blocks super 1.1 [2/2] [UU]
+
+bitmap: 1/1 pages [4KB], 65536KB chunk
+
+md2 : active raid1 sdb3[2]
+
+41909248 blocks super 1.1 [2/1] [_U]
+
+bitmap: 1/1 pages [4KB], 65536KB chunk
+```
+
+The “personalities” gives us the raid level that the raid is configured. In the above example, the raid is configured with `RAID 1. md0 : active raid1 sdb1[2] sda1[0]` tells us that there is an active raid of RAID 1 between sdb1(which is device 2) and sda1(which is device 0).An inactive array generally means that one of the disks are faulty. Md2 in the above example shows that we have `41909248 blocks super 1.1 [2/1] [_U]` , this means that one disk is down in this particular raid.
+
+The command `mdadm --detail /dev/<raid-array>` gives detailed information about that particular array.
+```
+sudo mdadm --detail /dev/md0
+
+/dev/md0:
+
+Version : 1.1
+
+Creation Time : Fri Nov 17 11:49:20 2019
+
+Raid Level : raid1
+
+Array Size : 10476544 (9.99 GiB 10.32 GB)
+
+Used Dev Size : 10476544 (9.99 GiB 10.32 GB)
+
+Raid Devices : 2
+
+Total Devices : 2
+
+Persistence : Superblock is persistent
+
+Intent Bitmap : Internal
+
+Update Time : Sun Dec 2 01:00:53 2019
+
+State : clean
+
+Active Devices : 2
+
+Working Devices : 2
+
+Failed Devices : 0
+
+Spare Devices : 0
+
+UUID : xxxxxxx:yyyyyy:zzzzzz:ffffff
+
+Events : 987
+
+Number Major Minor RaidDevice State
+
+0 8 1 0 active sync /dev/sda1
+
+1 8 49 1 active sync /dev/sdb1
+```
+Incase of a missing disk in the above example, the State of the raid would be ‘dirty’ and Active Devices and Working Devices would be reduced to one. One of the entries(either /dev/sda1 or /dev/sdb1 depending on the missing disk) would have their RaidDevice changed to faulty.
+
+## LVM
+
+LVM stands for Logical Volume Management. In the above section we saw how we can create FS and use individual disks according to our need the traditional way but using LVM we can achieve more flexibility in storage allocation like we can stitch three 2TB disks to make one single partition of 6TB, or we can attach another physical disk of 4TB to the server and add that disk to the logical volume group to make it 10TB in total.
+
+Refer to know more about LVM: [<u>https://www.redhat.com/sysadmin/lvm-vs-partitioning</u>](https://www.redhat.com/sysadmin/lvm-vs-partitioning)
+

mkdocs.yml

@@ -79,6 +79,14 @@ nav:
         - Writing Secure code: level101/security/writing_secure_code.md
         - Conclusion: level101/security/conclusion.md
 - Level 102:
+  - Linux Intermediate:
+      - Introduction: level102/linux_intermediate/introduction.md
+      - Package Management: level102/linux_intermediate/package_management.md
+      - Storage Media: level102/linux_intermediate/storage_media.md
+      - Archiving and Backup: level102/linux_intermediate/archiving_backup.md
+      - Introduction to Vim: level102/linux_intermediate/introvim.md
+      - Bash Scripting: level102/linux_intermediate/bashscripting.md
+      - Conclusion: level102/linux_intermediate/conclusion.md
   - Linux Advanced:
     - Containerization And Orchestration:
       - Introduction: level102/containerization_and_orchestration/intro.md