Hack Your Big Data Skills

Lambda Architecture Lambda architecture is a generic architecture for the distributed data processing system, introduced by Nathan Marz on his great experience at Backtype and Twitter. The architecture is considered for fault tolerant against both hardware failure and human mistakes. By using this architecture, we can achieve the use cases like low-latency reads and updates. As the system suggests a distributed data processing system, it should be linearly scalable and it should be scale out rather than scale up. 1. Data The data can be of high volume, high velocity, and even different varieties of data. In the IoT world, the data may be some sort of sensor data, machine logs etc. All these data will be dispatched to the Batch Layer and the Speed Layer for further processing. 2. Batch Layer The Batch Layer stores the master copy of the dataset and precomputes batch views on that master dataset. The master dataset can be thought of as a very large list of records. ...