feast icon indicating copy to clipboard operation
feast copied to clipboard

It is possible to materialize in S3 from multiple data sources?

Open andre-yai opened this issue 2 years ago • 1 comments

Problem Statement:

The current implementation lacks support for materializing data in S3 from multiple sources, including S3 file source, RDS, Athena, and Redshift. This capability is crucial for scenarios where data needs to be consolidated from diverse sources into a single destination in S3.

Expected Behavior:

The Materialize process should be enhanced to seamlessly handle data extraction and consolidation from various sources such as S3 file source, RDS, Athena, and Redshift. The goal is to empower users to efficiently and flexibly manage data flows across these different platforms.

andre-yai avatar Jan 16 '24 18:01 andre-yai

The current implementation lacks support for materializing data in S3 from multiple sources

Hi, can you clarify what you mean here? In feast, materialize refers to ingesting data from offline sources to online stores (redis for example). There's no concept of materialization to S3.

tokoko avatar Apr 03 '24 08:04 tokoko