OpenHub — The Ultimate Curated Software & Tool Directory

data-forge

Data Forge — a modern data stack playground to practice flows and best practices, not just tools. Spark, Trino, Kafka, Iceberg, ClickHouse, Airflow, MinIO, Superset — all wired together locally with D

Data-Engineering 183

dcs-core

Open Source Data Quality Monitoring.

Data-Engineering 179

jayvee

Jayvee is a domain-specific language and runtime for automated processing of data pipelines

Data-Engineering 178

wimsey

Easy and flexible data contracts

Data-Engineering 172

Data-Science-For-EveryOne

Data Science boot camp aims to make the field of data science accessible and understandable to a wide range of individuals, regardless of their background or expertise.

Data-Engineering 171

Movalytics-Data-Warehouse

Data pipeline performing ETL to AWS Redshift using Spark, orchestrated with Apache Airflow

Data-Engineering 167

big-data-modeling

Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University

Data-Engineering 165

raptor

Transform your pythonic research to an artifact that engineers can deploy easily.

Data-Engineering 164

FMD_FRAMEWORK

The Fabric Metadata-Driven (FMD) Framework is a community-driven accelerator for Microsoft Fabric that provides the foundation for automated, scalable, and enterprise-grade data platforms. By treating

Data-Engineering 163

didact

The open core .NET job orchestrator that we've been missing

Data-Engineering 162

job-board-aggregator

Job board aggregator indexing 1,000,000+ active positions from 20,000+ companies across Greenhouse, Lever, Ashby, Workday, and other major ATS platforms. Multithreaded Python ETL pipeline, daily autom

Data-Engineering 160

khaos

Kafka data generator and load testing tool - generate fake messages, simulate producers and consumers, test broker failures, and run chaos engineering scenarios

Data-Engineering 159

Databricks-Certified-Data-Engineer-Professional-Questions

This repo contains "Databricks Certified Data Engineer Professional" Questions and related docs.

Data-Engineering 156

pyspark-tutorial

PySpark Tutorial for Beginners - Practical Examples in Jupyter Notebook with Spark version 3.4.1. The tutorial covers various topics like Spark Introduction, Spark Installation, Spark RDD Transformati

Data-Engineering 156

dbt-sugar

dbt-sugar is a CLI tool that allows users of dbt to have fun and ease performing actions around dbt models

Data-Engineering 154

Meerschaum

Create and manage data pipes with Meerschaum.

Data-Engineering 154

data-engineer-portfolio

This is a repository to demonstrate my details, skills, projects and to keep track of my progression in Data Analytics and Data Science topics.

Data-Engineering 153

opendatadiscovery-specification

ODD Specification is a universal open standard for collecting metadata.

Data-Engineering 152

ProDa

📖 Data Engineering from Raw Corpora

Data-Engineering 150

dbmask

Discover, mask, and verify sensitive data in SQL databases — an auditable scan → mask → validate workflow for safe database copies.

Data-Engineering 148

accelerator

The Accelerator is a tool for fast and reproducible processing of large amounts of data.

Data-Engineering 147

docglow

Modern documentation site generator for dbt Core — lineage explorer, health scoring, full-text search. Live demo: https://demo.docglow.com

Data-Engineering 147

awesome-olap

🧊 A curated list of OLAP databases, data lake tools, columnar engines, and analytics frameworks for data engineers.

Data-Engineering 146

football-datasets

Complete football/soccer datalake with 93000+ players from Transfermarkt. Includes player profiles, performance statistics, market values, transfer histories, injury records, national team data, and t

Data-Engineering 145