Python-ETL-pipeline-using-Airflow-on-AWS

Data-Engineering Open Source

This project demonstrates how to build and automate an ETL pipeline written in Python and schedule it using open source Apache Airflow orchestration tool on AWS EC2 instance.

★ 24 Stars ⑂ 7 Forks License: Open Source

Download Latest Release Official Website ↗ GitHub Repository ↗

About Python-ETL-pipeline-using-Airflow-on-AWS

This project demonstrates how to build and automate an ETL pipeline written in Python and schedule it using open source Apache Airflow orchestration tool on AWS EC2 instance.

Supported Platforms

Open Source Web Cross-Platform

Related in Data-Engineering

data-engineering-zoomcamp

Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼

prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

airbyte

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

taipy

Turns Data and AI algorithms into production-ready web applications in no time.

dagster

An orchestration platform for the development, production, and observation of data assets.

Cookbook

The Data Engineering Cookbook