DataFlow-Engine
Data-Engineering Open Source数据流引擎是一款面向数据集成、数据同步、数据交换、数据共享、任务配置、任务调度的底层数据驱动引擎。数据流引擎采用管执分离、多流层、插件库等体系应对大规模数据任务、数据高频上报、数据高频采集、异构数据兼容的实际数据问题。
★ 697 Stars
⑂ 114 Forks
License: GNU General Public License v3.0
About DataFlow-Engine
数据流引擎是一款面向数据集成、数据同步、数据交换、数据共享、任务配置、任务调度的底层数据驱动引擎。数据流引擎采用管执分离、多流层、插件库等体系应对大规模数据任务、数据高频上报、数据高频采集、异构数据兼容的实际数据问题。
Supported Platforms
Open Source Web Cross-Platform
Related in Data-Engineering
data-engineering-zoomcamp
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼
prefect
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
airbyte
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
taipy
Turns Data and AI algorithms into production-ready web applications in no time.
dagster
An orchestration platform for the development, production, and observation of data assets.
Cookbook
The Data Engineering Cookbook