school-of-sre/courses/big_data/usage.md

Usage of Big Data techniques

1. Take the example of the traffic lights problem.
   1. There are more than 300,000 traffic lights in the US as of 2018.
   2. Let us assume that we placed a device on each of them to collect metrics and send it to a central metrics collection system.
   3. If each of the IOT devices sends 10 events per minute, we have 300000x10x60x24 = 432x10^7 events per day.
   4. How would you go about processing that and telling me how many of the signals were “green” at 10:45 am on a particular day?
2. Consider the next example on Unified Payments Interface (UPI) transactions:
   1. We had about 1.15 billion UPI transactions in the month of October, 2019 in India.
   2. If we try to extrapolate this data to about a year and try to find out some common payments that were happening through a particular UPI ID, how do you suggest we go about that?