Support MSC3916 by adding a federation `/download` endpoint

[H-Shay]

MSC3916 outlines the addition of a new federation endpoint to serve media downloads to other servers. This PR implements that endpoint and the new multipart/form-data response format that it returns. Future PRs will implement the rest of MSC3916, including altering the new client-server /download endpoint to use the new federation endpoint for remote download requests.

Should be reviewable commit-by-commit

[H-Shay]

Does this require more tests? If so, what should be tested?

[H-Shay]

complement run failing with CreateDirtyDeployment failed: CreateDirtyServer: Failed to deploy image complement-synapse : Error response from daemon: Conflict. The container name "/complement_fed_dirty_hs1" is already in use by container "08dc1e60340dfb37e078f29731cb4d102e1149a6afb2f5f62fc41564fd9caeac". You have to remove (or rename) that container to be able to reuse that name. - pretty sure that's not this PR? Normally I would just hit the re-run button but I no longer have that power...

[anoadragon453]

complement run failing with CreateDirtyDeployment failed: CreateDirtyServer: Failed to deploy image complement-synapse : Error response from daemon: Conflict. The container name "/complement_fed_dirty_hs1" is already in use by container "08dc1e60340dfb37e078f29731cb4d102e1149a6afb2f5f62fc41564fd9caeac". You have to remove (or rename) that container to be able to reuse that name. - pretty sure that's not this PR? Normally I would just hit the re-run button but I no longer have that power...

I think that error is a red herring. In the Synapse logs, I'm seeing the following exception:

  Error during startup:
  Traceback (most recent call last):
  pusher1 | 2024-05-09 12:22:43,513 - synapse.replication.tcp.redis - 292 - INFO - sentinel - Connecting to redis server localhost:6379
    File "/usr/local/lib/python3.11/site-packages/synapse/app/_base.py", line 258, in wrapper
      await cb(*args, **kwargs)
    File "/usr/local/lib/python3.11/site-packages/synapse/app/_base.py", line 594, in start
      hs.start_listening()
    File "/usr/local/lib/python3.11/site-packages/synapse/app/generic_worker.py", line 247, in start_listening
      self._listen_http(listener)
    File "/usr/local/lib/python3.11/site-packages/synapse/app/generic_worker.py", line 187, in _listen_http
      resources[FEDERATION_PREFIX] = TransportLayerServer(self)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/synapse/federation/transport/server/__init__.py", line 75, in __init__
      self.register_servlets()
    File "/usr/local/lib/python3.11/site-packages/synapse/federation/transport/server/__init__.py", line 78, in register_servlets
      register_servlets(
    File "/usr/local/lib/python3.11/site-packages/synapse/federation/transport/server/__init__.py", line 318, in register_servlets
      servletclass(
    File "/usr/local/lib/python3.11/site-packages/synapse/federation/transport/server/federation.py", line 812, in __init__
      self.media_repo = self.hs.get_media_repository()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/synapse/server.py", line 220, in _get
      dep = builder(self)
            ^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/synapse/server.py", line 686, in get_media_repository
      return MediaRepository(self)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.11/site-packages/synapse/media/media_repository.py", line 92, in __init__
      self.max_upload_size = hs.config.media.max_upload_size
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  AttributeError: 'ContentRepositoryConfig' object has no attribute 'max_upload_size'

The error about conflicting containers is certainly unhelpful though...

[H-Shay]

Weird! If it's an attribute error why did only one of the three complement tests fail? Don't they use the same image? I'm gonna dig deeper but I'm surprised...

Edit: looks like this was due to workers trying to instantiate the federation download servlet (which relies on the media repo) when the media repo has been disabled via config. I wondered why this wasn't picked up by the Sytest worker runs but I suspect that it is due to the fact that sytest doesn't appear to use the configure_workers_and_start script which explicitly disables the media repo on most workers. Anyhow I think the error is resolved now :)

[H-Shay]

It seems very unlikely to me that the sytest failure is a result of the last round of changes as I only added a test, newlines, and a comment...

[H-Shay]

Updated to reflect that new version of MSC removes server_name from the endpoint.