Data Engineering Streaming

• Key differences between batch & streaming
• Streaming Data Management use cases
• Streaming architecture
• Kappa architecture
• End-to-end Streaming Data Management
• Types of streaming data
• Benefits of streaming

Lab: 

• Getting Started

• Data Engineering Streaming overview
• Stream Data Processing with Spark streaming
• DES component architecture
• DES key features

• Kafka Concepts
• Kafka core APIs
• Topics in Kafka
• Kafka models
• Kafka Use cases

Lab:

• Install & Configure Kafka
• Create a Kafka connection

• Sources in a streaming mapping
• Targets in streaming Mapping
• Lookup sources
• DQ Transformations in streaming mappings
• Kafka Data Object Properties
• DES Transformations

Lab:

•  Create a Mapping with Kafka Source & HDFS Target
•  Create a Mapping with Kafka Source & Kafka Target
•  Enhance Mapping Using Filter & Expression Transformations
•  Enhance Mapping Using Window & Aggregator Transformations
• Create a Mapping Using Kafka Source & Kudu Target
• Create a Mapping to write standardized data using Kafka Source & Hive Target
• Create a Mapping Using Parser Transformation
• Create a Mapping Using Classifier Transformation

• Spark Monitoring
• Viewing Logs
• Troubleshooting

Lab: 

• · Monitor a DES Mapping

• Tune the performance of Spark jobs
• List some best practices while working with streaming data

A: Streaming is a technique used for transferring data so that it can be processed as a continuous & steady stream. Streaming technologies are becoming crucial with the growth of the internet.

A: In this training, you will learn how to prepare, maintain, enrich & process a stream of data in real-time using Informatica, Kafka, Spark & Edge.

A: Prime videos, Netflix, Hulu, and some video share websites like youtube and Twitch used for live video game streaming & Spotify & apple music are used for live streaming music are some of the examples of streaming.

A: Radiant has highly intensive selection criteria for Technology Trainers & Consultants who deliver training programs. Our trainers & consultants undergo rigorous technical & behavioral interviews & assessment processes before they are onboarded in the company.

Our Technology experts/trainers & consultants carry deep-dive knowledge in the technical subject & are certified by the OEM.

Our training programs are practically oriented with 70% – 80% hands-on training technology tools. Our training program focuses on one-on-one interaction with each participant, the latest content in the curriculum, real-time projects & case studies during the training program.

Our faculty will provide you the knowledge of each training from a fundamental level in an easy way & you are free to ask your doubts any time from your respective faculty.

Our trainers have the patience & ability to explain difficult concepts in a simplistic way with depth & width of knowledge.

To ensure quality learning, we provide support sessions even after the training program.

A: Radiant Techlearning offers a training program on weekdays, weekends & a combination of weekdays & weekends. You can always choose the schedule that best suits your need.

A: We would always recommend you attend the live session to practice & clarify the doubts instantly & get more value from your investment. However, due to some contingency, if you have to skip the class, Radiant Techlearning will help you with the recorded session of that particular day. However, those recorded sessions are not meant only for personal consumption & NOT for distribution or any commercial use.

A: Radiant Techlearning has a data center containing the Virtual Training environment for the purpose of participant hand-on-practice. 

Participants can easily access these labs over Cloud with the help of a remote desktop connection. 

Radiant virtual labs provide you the flexibility to learn from anywhere in the world & in any time zone.