data-engineering topic

List data-engineering repositories

viewflow

121
Stars
10
Forks
Watchers

Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

verified publisher icon datacamp

setl

177
Stars
31
Forks
Watchers

A simple Spark-powered ETL framework that just works 🍺

verified publisher icon SETL-Framework

data-science-on-gcp

1.3k
Stars
709
Forks
Watchers

Source code accompanying book: Data Science on the Google Cloud Platform, Valliappa Lakshmanan, O'Reilly 2017

verified publisher icon GoogleCloudPlatform

yuniql

413
Stars
62
Forks
Watchers

Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!

verified publisher icon rdagumampan

eyes

117
Stars
18
Forks
Watchers

Public Opinion Mining System of Taiwanese Forums

verified publisher icon r05323028

procfwk

176
Stars
113
Forks
Watchers

A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.

verified publisher icon mrpaulandrew

mage-ai

7.2k
Stars
660
Forks
Watchers

🧙 Build, run, and manage data pipelines for integrating and transforming data.

verified publisher icon mage-ai

active_workflow

805
Stars
67
Forks
Watchers

Polyglot workflows without leaving the comfort of your technology stack.

verified publisher icon automaticmode

memphis

3.2k
Stars
211
Forks
Watchers

Memphis.dev is a highly scalable and effortless data streaming platform

verified publisher icon superstreamlabs

Data-Science-Roadmap

3.3k
Stars
450
Forks
Watchers

Data Science Roadmap from A to Z

verified publisher icon Moataz-Elmesmary