Build Server Less data pipeline using AWS Glue
What you'll learn
- Learn AWS Glue in 2 hours
- Setup AWS Glue crawlers and cron jobs to make them run according to your business requirements.
- How to setup an AWS Glue Pipeline
- Use AWS Glue Transformations
Requirements
- Basic Python.
Description
In this course student will learn what is AWS Glue ,Components, Preparation for AWS Glue ,Glue Architecture, Benefits And Limitations Of AWS Glue & AWS Glue Terminology.In Section 2 Student will learn what is crawler, data catalog, Data base, tables and Practical demo of S3 Crawler, MYSQL Crawler, JSON Crawler & Build Custom Classifier.
You’ll learn how to set up a Glue data crawler, then how to crawl the data in a S3 folder to populate the Glue Data Catalog with metadata about the S3 data.
In Section 3 : You will learn about Development Endpoint, Setup endpoint, Glue Context & Dynamic Frame, How to create dynamic frame using RDD.
In Section 4 : You will learn about Transformation, Resolve Choice, Split Rows, Map ,Filter, Select & Rename, Spigot, Flatten using JSON, Drop.
In Section 5 : you will learn about JOB & create trigger. How you can setup JOB on regular interval.
Future topics : In Future I add add more content on Workflow, How to build incremental data pipeline using Bookmark, What is Glue Studio & What is AWS Data brew.
This course is more focused on Practical example & Student will be able to work on AWS Glue project.
Student will learn how to design Data pipeline.
Who this course is for:
- Big data Engineer.
