DATA PIPELINE USING AWS GLUE

WORLD HAPPINESS DATA 2018 AND 2019 UNIONED AND EXPORTED TO S3 BUCKET

This demo shows the completed Data Pipeline project using Union transformation for World Happiness Report data for 2018 and 2019. AWS Glue has a convenient Visual tool to choose from Sources, Transform Methods, and Targets. The python script is also auto-generated by AWS Glue.

PROCESS: Extract data and upload to s3 bucket -> Define Source, Transform Methods and Targets -> Run ETL job

Screenshots:

1. Final step confirmed
2. Final visual diagram of the ETL process in AWS Glue
3. Confirmation of successful ETL run in the AWS Glue Job runs page
4. Successfully loaded unioned data in s3 bucket for staging
5. Auto-generated script by AWS Glue.