synapse
synapse copied to clipboard
`compute_state_delta` pulls out the full state when doing gappy syncs
This causes spikes in memory usage when handling /sync requests.
The callstack:
File "/home/erikj/synapse/synapse/handlers/sync.py", line 1562, in handle_room_entries
await self._generate_room_entry(
File "/home/erikj/synapse/synapse/handlers/sync.py", line 2072, in _generate_room_entry
state = await self.compute_state_delta(
File "/home/erikj/synapse/synapse/handlers/sync.py", line 966, in compute_state_delta
state_at_previous_sync = await self.get_state_at(
File "/home/erikj/synapse/synapse/handlers/sync.py", line 680, in get_state_at
state = await self.get_state_after_event(
File "/home/erikj/synapse/synapse/handlers/sync.py", line 639, in get_state_after_event
state_ids = await self._state_storage_controller.get_state_ids_for_event(
File "/home/erikj/synapse/synapse/storage/controllers/state.py", line 303, in get_state_ids_for_event
state_map = await self.get_state_ids_for_events(
File "/home/erikj/synapse/synapse/storage/controllers/state.py", line 253, in get_state_ids_for_events
group_to_state = await self.stores.state._get_state_for_groups(
File "/home/erikj/synapse/synapse/storage/databases/state/store.py", line 288, in _get_state_for_groups
group_to_state_dict = await self._get_state_groups_from_groups(
File "/home/erikj/synapse/synapse/storage/databases/state/store.py", line 173, in _get_state_groups_from_groups
It looks like we did this to handle a bug where membership changes in the gap get missed: https://github.com/matrix-org/synapse/blob/845732be450b3f9c991df35b2f07d600a0eca6dd/synapse/handlers/sync.py#L948-L951
See the comment here: https://github.com/vector-im/element-web/issues/7211#issuecomment-419668549
I don't think there's an easy fix for this one.