USB serial (CDC-ACM) to host is stingy with buffers, leads to blocking & bursting

[egnor]

Observed symptoms:

• When attempting to write from e.g. a Linux host to the (super adorable) Seeeduino XIAO, there is basically 0 buffer. The host write will block until the Arduino code calls Serial.read(). This creates problems such as https://github.com/pyserial/pyserial/issues/653. Having 0 buffer in a serial port system is generally bad for performance.
• When receiving data, even if the host is attempting to send a significant quantity of data, Serial.read() will return -1 frequently. For example if the host sends "Hello World!", Serial.read() might return 'H', 'e', 'l', -1, 'l', 'o', ' ', 'W', -1, ...
• Other USB-serial device implementations don't do this.

Discussion:

This code in USBDeviceClass::recv (called from _Serial::read) directly reads data from the USB buffer and only acknowledges the transfer once the code has been read by the app:

https://github.com/Seeed-Studio/ArduinoCore-samd/blob/991c4b032270cc7e53fdd70e31e75eeca2bdedc8/cores/arduino/USB/USBCore.cpp#L649

I believe this is why there is a zero-slack channel here, and why Serial.read() gives spurious -1 values as it reaches the end of one buffer and then returns it to the host for more data. Almost every other USB-serial implementation I've seen places some sort of ring buffer in between; USB transfers deposit data into the ring, and the application pulls out of it. That way there is a reasonable amount of "flex" and data can flow smoothly. Is that not something that can be done in this case?