Lynda – Apache Spark Essential Training: Big Data Engineering-0daytown

Lynda – Apache Spark Essential Training: Big Data Engineering
Size: 372 MB | Duration: 1h 40m | Video: AVC (.mp4) 1280×720 15&30fps | Audio: AAC 48KHz 2ch
Genre: eLearning | Level: Advanced | Language: English
In order to construct data pipelines and networks that stream, process, and store data, data engineers and data-science DevOps specialists must understand how to combine multiple big data technologies. In this course, discover how to build big data pipelines around Apache Spark. Join Kumaran Ponnambalam as he takes you through how to make Apache Spark work with other big data technologies. He covers the basics of Apache Kafka Connect and how to integrate it with Spark for real-time streaming. In addition, he demonstrates how to use the various technologies to construct an end-to-end project that solves a real-world business problem.

Topics include:

* What is data engineering?
* Spark and Kafka for data engineering
* Moving data with Kafka and Kafka Connect
* Kafka integration with Apache Spark
* How Spark works
* Optimizing for lazy evaluation
* Complex accumulators