Cover Image

An architecture for fast and general data processing on large clusters /

Bibliographic Details
Main Author: Zaharia, Matei (Author)
Format: eBook
Language:English
Published: [New York] : [San Rafael, California] : Association for Computing Machinery ; Morgan & Claypool, 2016.
Edition:First edition.
Series:ACM books ; #11.
Subjects:
Big data.
Distributed databases.
Electronic data processing > Distributed processing.
Electronic data.
Online Access:Connect to the full text of this electronic book
Table of Contents:
  • 1. Introduction
  • 1.1 Problems with specialized systems
  • 1.2 Resilient distributed datasets (RDDs)
  • 1.3 Models implemented over RDDs
  • 1.4 Summary of results
  • 1.5 Book overview
  • 2. Resilient distributed datasets
  • 2.1 Introduction
  • 2.2 RDD abstraction
  • 2.3 Spark programming interface
  • 2.4 Representing RDDs
  • 2.5 Implementation
  • 2.6 Evaluation
  • 2.7 Discussion
  • 2.8 Related work
  • 2.9 Summary
  • 3. Models built over RDDs
  • 3.1 Introduction
  • 3.2 Techniques for implementing other models on RDDs
  • 3.3 Shark: SQL on RDDs
  • 3.4 Implementation
  • 3.5 Performance
  • 3.6 Combining SQL with complex analytics
  • 3.7 Summary
  • 4. Discretized streams
  • 4.1 Introduction
  • 4.2 Goals and background
  • 4.3 Discretized streams (D-streams)
  • 4.4 System architecture
  • 4.5 Fault and straggler recovery
  • 4.6 Evaluation
  • 4.7 Discussion
  • 4.8 Related work
  • 4.9 Summary
  • 5. Generality of RDDs
  • 5.1 Introduction
  • 5.2 Expressiveness perspective
  • 5.3 Systems perspective
  • 5.4 Limitations and extensions
  • 5.5 Related work
  • 5.6 Summary
  • 6. Conclusion
  • 6.1 Lessons learned
  • 6.2 Evolution of spark in industry
  • 6.3 Future work
  • References
  • Author's biography.