Aljoscha Krettek

storage: shard write work in `persist_sink`

1

### Motivation Opening this PR early to showcase the approach/gather feedback. Epic that somewhat tracks this: https://github.com/MaterializeInc/materialize/issues/13884 ### Tips for reviewer Especially the changes that were required in `persist`, around...

### Tips for reviewer The comments should explain it all, but please yell at me if they don't. Also: I should probably downgrade the logging to `trace!` before merging, but...

WIP: suss out problems arising from storaged parallelism

4

For running CI.

### Motivation This is one of the issues that @philip-stoev discovered when trying to run CI with `workers > 1` (see #14273) ### Notes for reviewers It's very important to...

adapter: factor CollectionsController out of StorageController

1

### Motivation First step of https://github.com/MaterializeInc/materialize/pull/24488. For now, largely for getting a CI run and for socializing the changes. ### Tips for reviewer Individual commits tell a story! ### Checklist...

[Epic] persist-txn: abstraction for scalable and isolated tables

1

## Background Use-case isolation is meant to provide customers with the tools for managing incidental unavailability. Milestones Overview: https://www.notion.so/materialize/Use-case-Isolation-Milestones-b64e53900c0c498eae4bd38df6adb7c8 --- Currently, table writes are performed by separate persist operations _per_...

## Background Use-case isolation is meant to provide customers with the tools for managing incidental unavailability. Milestones Overview: https://www.notion.so/materialize/Use-case-Isolation-Milestones-b64e53900c0c498eae4bd38df6adb7c8 --- Currently the coordinator/adapter takes the following actions in response to...

## Background Use-case isolation is meant to provide customers with the tools for managing incidental unavailability. Milestones Overview: https://www.notion.so/materialize/Use-case-Isolation-Milestones-b64e53900c0c498eae4bd38df6adb7c8 --- ```[tasklist] ### Subtasks - [ ] #22174 - [ ]...

## Background Use-case isolation is meant to provide customers with the tools for managing incidental unavailability. Milestones Overview: https://www.notion.so/materialize/Use-case-Isolation-Milestones-b64e53900c0c498eae4bd38df6adb7c8 --- Currently, there is a (process-global) lock for table writes. Mainly...

adapter: use STORAGE ReadHold instead of custom hold tracking in adapter

5

Preparatory work for https://github.com/MaterializeInc/materialize/issues/24845, where we want to introduce more concurrency to the Coordinator and Controllers. Before, adapter would track outstanding `ReadHolds` in its own state and continually update the...