Data-Pipeline-with-dbt-using-Airflow-on-GCP

Data-Engineering Open Source

This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such

★ 28 Stars ⑂ 7 Forks License: Open Source

Download Latest Release Official Website ↗ GitHub Repository ↗

About Data-Pipeline-with-dbt-using-Airflow-on-GCP

This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.

Supported Platforms

Open Source Web Cross-Platform

Related in Data-Engineering

data-engineering-zoomcamp

Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼

prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

airbyte

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

taipy

Turns Data and AI algorithms into production-ready web applications in no time.

dagster

An orchestration platform for the development, production, and observation of data assets.

Cookbook

The Data Engineering Cookbook