Lambda Architecture : Mixing Real-time processing with Batch Processing

Authors

  • Manisha Sethi Department of Computer Science and Applications, RNCET, Kurukshetra University

Keywords:

Real-time, bigdata, lambda, streaming process

Abstract

Lambda Architecture is a design principle in Big Data systems where dealing with throughput and latency in real-time is the most important. This mixture of batch processing with real-time streaming process provides the benefits of both the approaches, thus making the system having the precomputed views which enables high throughput and fresh calculations are done on online data to provide end result most accurate with high throughput, decent accuracy and low latency. The lambda architecture is inspired by the rise in bigdata architectures striving for accuracy as well as speed.

References

Schuster, Werner. "Nathan Marz on Storm, Immutability in the Lambda Architecture, Clojure". www.infoq.com. Interview with Nathan Marz, 6 April 2014

Bijnens, Nathan. "A real-time architecture using Hadoop and Storm". 11 December 2013.

Marz, Nathan; Warren, James. Big Data: Principles and best practices of scalable realtime data systems. Manning Publications, 2013.

Kar, Saroj. "Hadoop Sector will Have Annual Growth of 58% for 2013-2020", 28 May 2014. Cloud Times.

Kinley, James. "The Lambda architecture: principles for architecting realtime Big Data systems", retrieved 26 August 2014.

Ferrera Bertran, Pere. "Lambda Architecture: A state-of-the-art". 17 January 2014, Datasalt.

Yang, Fangjin, and Merlino, Gian. "Real-time Analytics with Open Source Technologies". 30 July 2014.

Ray, Nelson. "The Art of Approximating Distributions: Histograms and Quantiles at Scale". 12 September 2013. Metamarkets.

Rao, Supreeth; Gupta, Sunil. "Interactive Analytics in Human Time". 17 June 2014

Bae, Jae Hyeon; Yuan, Danny; Tonse, Sudhir. "Announcing Suro: Backbone of Netflix's Data Pipeline", Netflix, 9 December 2013

Kreps, Jay. "Questioning the Lambda Architecure". radar.oreilly.com. Oreilly. Retrieved 15 August 2014.

"Altior's AltraSTAR – Hadoop Storage Accelerator and Optimizer Now Certified on CDH4 (Cloudera's Distribution Including Apache Hadoop Version 4)" (Press release). Eatontown, NJ: Altior Inc. 2012-12-18. Retrieved 2013-10-30.

"Hadoop". Azure.microsoft.com. Retrieved 2014-07-22.

"HDInsight | Cloud Hadoop". Azure.microsoft.com. Retrieved 2014-07-22.

Varia, Jinesh (@jinman). "Taking Massive Distributed Computing to the Common Man – Hadoop on Amazon EC2/S3". Amazon Web Services Blog. Amazon.com. Retrieved 9 June 2012.

Gottfrid, Derek (1 November 2007). "Self-service, Prorated Super Computing Fun!". The New York Times. Retrieved 4 May 2010.

Downloads

Published

30-06-2015

How to Cite

Manisha Sethi. (2015). Lambda Architecture : Mixing Real-time processing with Batch Processing. International Journal for Research Publication and Seminar, 6(2), 1–8. Retrieved from https://jrps.shodhsagar.com/index.php/j/article/view/726

Issue

Vol. 6 No. 2 (2015): Volume 6 | Issue 2 | May-Jun 2015

Section

Original Research Article

License

Copyright (c) 2015 International Journal for Research Publication and Seminar

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Re-users must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. This license allows for redistribution, commercial and non-commercial, as long as the original work is properly credited.