Sean Loiselle

There is a very rough prototype of this feature in #13849 There is a design doc for the proposed approach in Notion under the name "Scalable reads from persist"

We ended up using #13849 to merge the code for sharded reads from persist. However, the code is currently unused. Dan + Aljoscha realized there is an issue with how...

another instance here https://buildkite.com/materialize/tests/builds/40641#_

The specification of this implementation is in https://github.com/MaterializeInc/materialize/blob/main/doc/developer/platform/ux.md; search for `websocket`. A few random notes: - This implementation will likely need to live in `src/environmentd/src/http.rs` - You can repurpose the...

@JLDLaughlin The idea behind labeling this as a platform blocker was codifying the shape of these queries to prevent easy-to-avoid breaking changes, i.e. the release of platform should have a...

@heeringa asks: > For developers that will use this programmatically via an API, does it make sense to have a single command version? We could add an inline definition of...

@andrioni Not yet--we haven't removed the ability to e.g. specify Kafka connections inline. You can find some more details from @benesch here https://github.com/MaterializeInc/cloud/issues/2837#issuecomment-1186250863

@JLDLaughlin No, not yet. There is a lot of work (i.e. typing) remaining to make this happen. I'll spin up some issues to atomize this and make it a little...

@JLDLaughlin @chaas updated this to make surveying the remaining work a little easier. modulo SSH connections, the remaining work is not particularly hard, but is a phenomenal amount of typing.

cc @uce @nmeagan11 as I created some tracking issues for the storage team, as well