OpenHub — The Ultimate Curated Software & Tool Directory

postcard-company-dataform

example BQ data model using Dataform

Data-Engineering 17

prefect-saturn

Python client for using Prefect Cloud with Saturn Cloud

Data-Engineering 17

open_auto_loader

OpenAutoLoader: A lightweight, open-source alternative to Databricks Auto Loader. Built with Polars and SQLite for efficient, incremental file ingestion.

Data-Engineering 17

sc2-data-pipeline

StarCraft 2 Data Pipeline with Airflow, DuckDB and Streamlit

Data-Engineering 17

postcard-company-datamart

learning-by-doing data model built with dbt-core

Data-Engineering 17

data

🎲 Repositório para armazenar todos os componentes referentes a Data Science / Data Engineering do projeto

Data-Engineering 17

dolphin-mcp-pilot

Production-ready MCP server for Apache DolphinScheduler (小海豚) — lets AI agents build, schedule, run and recover workflows

Data-Engineering 17

dbmazz

CDC, radically simplified. One Rust binary replaces Kafka + Debezium + ZooKeeper + Flink. Any source, any target. 300K+ events/sec · <1s latency · ~5 MB RAM. Fewer components = fewer failures + low

Data-Engineering 17

youtube-channel-analytics

📊 ETL pipeline for YouTube competitor analytics. Orchestrated with Airflow, Docker, and AWS. Features sentiment analysis and a Power BI dashboard.

Data-Engineering 17

foxtrot

A store abstraction and analytics system for real-time event data.

Data-Engineering 17

Flink-Example

Flink Example

Data-Engineering 17

Airflow-StudyClub

Grupo de estudio Apache Airflow organizado por la comunidad Data Engineering Latam

Data-Engineering 17

data-engineering-roadmap

Complete data engineering roadmap with technologies, certifications (AWS, GCP, Azure, Snowflake, Databricks), and learning resources.

Data-Engineering 17

rypipe

Format- and source-agnostic ingestion framework: a common execution runtime for turning record-oriented data sources into typed columnar data. Rust core, Python bindings; adapters live in separate pac

Data-Engineering 17

shelftxt

Open-source FastAPI recommendation engine for books • backend architecture • data handling • scalable systems

Data-Engineering 17

apache-airflow-intro

Verified Open Source software & tools ready for production.

Data-Engineering 17

dagger

Define sophisticated data pipelines with Python and run them on different distributed systems (such as Argo Workflows).

Data-Engineering 17

Feature-Engineering-Technique

📊 A practical toolkit for feature engineering and selection in Python. Explore EDA, handle missing data and outliers, scale/encode/transform features, discretize, and select via filter, wrapper, embed

Data-Engineering 17

SQL-DataAnalytics-Projects

A collection of SQL-based data analytics projects demonstrating techniques for data extraction, transformation, analysis, and visualization. These projects showcase practical SQL queries and insights

Data-Engineering 17

data-engineer-project

Data Engineering Capstone

Data-Engineering 17

apache-nifi-helm

Helm chart deploys Latest Apache NiFi in a Kubernetes cluster

Data-Engineering 17

pai-aws

Data Engineering: Chapter 5 aws chapter for pragmatic ai. Creates an "real world" Data Engineering API using Flask,Click, Pandas and Swagger docs

Data-Engineering 17

aws_end_to_end_streaming_pipeline

An AWS Data Engineering End-to-End Project (Glue, Lambda, Kinesis, Redshift, QuickSight, Athena, EC2, S3)

Data-Engineering 17

datatalk-cli

Query CSV, Excel & Parquet files with natural language. Fast, local, DuckDB-powered.

Data-Engineering 17