OpenHub — The Ultimate Curated Software & Tool Directory

orbital

🛰 A desktop app that allows you to search, filter, and preview video files on your computer - like YouTube for your local file system.

Data-Engineering 104

datadocs

Documentation for data enthusiasts

Data-Engineering 104

Fraud_Detection_SQL

Fraud Detection on credit card transations

Data-Engineering 104

chile-hub

Datos públicos de Chile curados, normalizados y validados: geografía, demografía, economía, salud, educación y distritos electorales. Listos para consumir en una línea de código con Polars, DuckDB, SQ

Data-Engineering 104

cortex

Cortex | Data Framework—a cutting-edge SDK that simplifies real-time data processing with intuitive operators, robust state management, and seamless telemetry for efficient, scalable pipelines.

Data-Engineering 101

minarrow

Fast, minimal, and ergonomic zero-copy Apache Arrow tabular data implementation in Rust, with PyO3, Python bindings, NdArray, XArray, shared-memory and low-latency SIMD for data-engineering and column

Data-Engineering 100

flowman

Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.

Data-Engineering 98

zksync-era-ETL

Best zkSync-era ETL ever 😜

Data-Engineering 98

delta-sharing-rs

A Minimalistic Rust Implementation of Delta Sharing Server.

Data-Engineering 97

arnio

Data trust for Python. Validate, clean, and profile DataFrames before everything else.

Data-Engineering 96

vscode-dataform-tools

Dataform Tools - VS Code extension to run and visualise Dataform data pipelines and much more

Data-Engineering 94

mpc-DL-controller

Deep Neural Network architecture as a predictive optimal controller for {HVAC+Solar cell + battery} disturbance afflicted system vs classic Model Predictive Control

Data-Engineering 94

SimTradeData

SimTradeData is a utility library supporting SimTradeDesk, SimTradeLab and simtradeML with reliable, high-quality simulated trading data for model training, backtesting, and performance evaluation.

Data-Engineering 93

mloda

mloda.ai - Open Data Access for AI and ML. Plugin-based. Traceable. Framework-agnostic.

Data-Engineering 92

Validoopsie

A simple and easy to use Data Validation library for Python.

Data-Engineering 91

sequor

Build complete API integrations with YAML and SQL. Rapid development without vendor lock-in and per-row costs.

Data-Engineering 91

AIDE-unipi

Students' material for the course in Artificial Intelligence and Data Engineering at University of Pisa.

Data-Engineering 90

vietlott-data

Automation fetching data for Vietlott. Just for fun.

Data-Engineering 89

gallia-core

A schema-aware Scala library for data transformation

Data-Engineering 89

batch-data-pipeline

🦆 Batch data pipeline with Airflow, DuckDB, Delta Lake, Trino, MinIO, and Metabase. Full observability and data quality.

Data-Engineering 89

airflow-testing-ci-workflow

(project & tutorial) dag pipeline tests + ci/cd setup

Data-Engineering 88

prism

Prism is the easiest way to develop, orchestrate, and execute data pipelines in Python.

Data-Engineering 88

odbc-scanner-duckdb-extension

A DuckDB extension to read data directly from databases supporting the ODBC interface

Data-Engineering 88

jev-curate

High-throughput synthetic and pretraining dataset sifter for TypeSafe Jev. Rust streaming core, Parquet and JSONL I/O, typed Choice/Score/Noul judgments, speculative fan-out, 24.0 rows/sec measured.

Data-Engineering 87