Add MediaRecorder component

[zboyles]

Hey everyone! 👋

I'm taking a crack at adding audio recording and playback as discussed in issue #172. Since this is still pretty exploratory, I figured I'd open up this draft PR to get some feedback and collaborate with any interested parties. Hopefully, once we have the Pydantic classes and schema files for this MediaRecorder sorted, the audio recording functionality will start to work.

I decided to call the component MediaRecorder instead of AudioRecorder to keep our options open for video recording down the line, since the browser API supports both. There is even a chance video recording functions alongside the audio recording, assuming the audio recording reaches a functional state. But like everything else at this state, we can change it based on how things shape up and the feedback received.

I'm also thinking about adding an AudioPlayer component that can handle streams and URLs, to add bi-directional audio communication functionality to FastUI.

I'd really appreciate any thoughts, ideas, or contributions you have, as these features are tackled. Whether it's code, suggestions, or pointing out potential issues, all kinds of collaboration is welcome. 🤙

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.88%. Comparing base (16abe6a) to head (0dedbbc). Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #244      +/-   ##
==========================================
+ Coverage   95.68%   95.88%   +0.19%     
==========================================
  Files          14       14              
  Lines         950      996      +46     
==========================================
+ Hits          909      955      +46     
  Misses         41       41              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[zboyles]

After renaming MediaRecorder to Recorder as to avoid conflicts with the browser API, the component is rendering correctly, requesting/gaining permission to access the microphone successfully, and starting/stopping the recording as expected. What's left is arguably the most import part, returning the Blob data. These are the potential solutions I can think of:

1. Anytime the python Recorder is used, similar to a form's submit_url, it would also require the declaration of an endpoint to receive the data, and this endpoint URL would be added to the component instance. When the recording is stopped, the data is submitted to the endpoint URL.

I can't think of another way to pass the data back without a larger change.

🧠🌪️🌧️ Brainstorm Advanced Options

A "larger change" would involve an optional setup function where you would register your app with FastUI allowing dynamic endpoint creation or a single endpoint accepting a list of UploadFile with dynamic request routing. FastUI could then have a parameterized callable type tooling for component developers to use. I image they would provide an endpoint ID, probably via forwarding an ID parameter through their typescript component, this value would be passed into function returning a hook that accepts data when called and forwards it to an endpoint constructed with the endpoint ID initially passed in. The previously mentioned setup function would have access to all the IDs used in the app's components and would forward the data to the callable value set in the component's parameterized callable type.

Instead of a setup function, it could be a FastUI helper function where the developer creates an endpoint and calls this helper function, passing the request along with it. If one is added, it would allow extended capabilities as outlined above.

I could use some feedback on this if anyone has any insights or has identified issues with how I currently understand the situation.

[zboyles]

Quick update

Changes / Current Functionality

• The Recorder functions properly when recording audio.
• Video might also be working but I don't have a webcam available to test yet.
• There are 2 options to handle the recorded output:
  1. A submit_url can be specified, pointing to an endpoint, see below for an example, that will receive the file.
  2. If save_recording field is set to True, the browser client will be prompted to save the recording once stopped.
  • Both submit_url and save_recording can be used simultaneously.
• A default theme is specified in npm-fastui-bootstrap.
• The component consists of a button for start and another for stop, each using the same theme as the standard Button.
• There is a display_style option defaulting to 'standard' and can be set to 'toggle' which renders a single button, again using the same standard Button theme.
• As with text and stop_text, there is an image_url and stop_image_url.
  • Unlike text and stop_text, stop_image_url defaults to the image, if specified, at image_url.
  • Images can be positioned with image_position which determines if an image is placed on the left or right of the text.
  • The class/theme uses a sub element for left-image, right-image, and container.
  • Image size can be set with image_width and image_height, both defaulting to 24px.
  • Visibility can be toggled with hide_image and functions similar to hide_text.

Endpoint Example

The following code displays a basic endpoint example capable of receiving a recording and saving it to disk:

@router.post("/media_upload")
async def media_upload(recording: UploadFile):
    save_path = Path("some/path")
    recording_path = save_path / (recording.filename or "recording.webm")
    try:
        contents = await recording.read()
        recording_path.write_bytes(contents)
    except Exception as e:
        print(e)
    return PlainTextResponse(f"File {recording_path.as_posix()} saved")

Once I have time, I'll finish cleaning/refactoring the code and add the changes to this PR.