Data Engineering with Python : Work with Massive Datasets to Design Data Models and Automate Data Pipelines Using Python.
This book is a comprehensive introduction to building data pipelines, that will have you moving and transforming data in no time. You'll learn how to build data pipelines, transform and clean data, and deliver it to provide value to users. You will learn to deploy production data pipelines that...
| Main Author: | |
|---|---|
| Corporate Author: | |
| Format: | eBook |
| Language: | English |
| Published: |
Birmingham :
Packt Publishing, Limited,
2020.
|
| Subjects: | |
| Online Access: | Connect to the full text of this electronic book |
Table of Contents:
- What is data engineering?
- Building our data engineering infrastructure
- Reading and writing files
- Working with databases
- Cleaning, transforming, and enriching data
- Building a 311 data pipeline
- Features of a production pipeline
- Version control with the NiFi registry
- Monitoring data pipelines
- Deploying data pipelines
- Building a production data pipeline
- Buildings Kafka cluster
- Streaming data with Apache Kafka
- Data processing with Apache Spark
- Real-time edge data with MiNiFi, Kafka, and Spark.