Tìm thấy 653 ứng dụng & công cụ phù hợp
Code examples and experiments from https://engineeringfordatascience.com
:arrows_counterclockwise: :running: EtLT of my own Strava data using the Strava API, MySQL, Python, S3, Redshift, and Airflow
Arcane Insight is a data analytics project designed to harness the power of SQLMesh & DuckDB to collect, transform, and analyze data from Blizzard’s Hearthstone API. Focused on card statistics and att
A software engineering framework to jump start your machine learning projects
Airflow DAGs for https://github.com/blockchain-etl/bitcoin-etl
Less-Resilient MapReduce framework for Go
Lightweight Python library and CLI that generates Databricks Workflows definition from your dbt project — each dbt model, test, seed, and snapshot runs as a separate task.
Write data & AI pipelines in (SQL, Spark, Pandas) and deploy to the cloud, simplified
An end-to-end data pipeline which extracts divvy bikeshare data from web loads it into data lake and datawarehouse transforms it using dbt and finally , a dashboard to visualize the data using looker
Kaggle Data Explorer look-alike: an easy tool to browse, filter, and analyze datasets with interactive tables and charts.
A Python toolkit for extracting and transforming Bitcoin blockchain data into structured formats.
FinOps repository: An open-source framework for multi-cloud cost visibility. Extendable with dlt.
OpenSnowcat Enricher (Apache 2.0 License)
Deploy a Prefect flow to serverless AWS Lambda function
Data Engineering Bootcamp
Production-style real-time e-commerce lakehouse with Kafka, Airflow, Databricks, Medallion architecture, data quality, quarantine, Terraform, and Dash analytics.
Frontend & BFF (Backend for frontend) for Olake. This includes the UI code and backend code for storing the configuration of sync and orchestrating it.
Real-time fraud detection engine for M-Pesa mobile money — Kafka/Flink streaming, ML-based risk scoring, and explainable multi-signal checks (velocity, SIM swap correlation, mule networks) feeding an
Apache Airflow Guide
The goal of this project is to illustrate Extract Transform Load (ETL) using Python and SQL. ETL is a process commonly done in computing, which takes raw data, cleans it and stores it for later use. T
A pipeline orchestration tool
This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.
Portfolio of projects and studies conducted in data engineering.
Data Engineering - Metropolitan Transportation Authority (MTA) Subway Data Analysis