Get all partial packages received instead of discarding them?

[araldit]

Hi,

I'm sending a large message from an embedded client. If I only eg. get, lets say 9 out of 10 messages through, all packages will get lost due to a retransmission timeout. What will be the best way to catch a timeout exception and retrieve the received bytes before the message gets released from memory?

[chrysn]

Just to clarify: The embedded client sends a block-wise request with a large request body that gets fragmented. The CoAP server is implemented in aiocoap, and is supposed to process what has been received. Is that right?

If so, there are two avenues:

• Notifying the application on partial requests being evicted is currently not implemented. Might be feasible, though, as an extension; will need good understanding of use cases.
• The handler can opt out of the server-side block-wise processing. For that, it'd implement async def needs_blockwise_assembly(self, request) to return False, or implement render_to_pipe right away. The server will then be fed the blocks individually. Then it can either act on the requests as they come in, or use some mechanisms otherwise implemented (currently) in aiocoap.interfaces.Resource._redner_to_pipe. Note that the helpers that takes from aiocoap.blockwise are not exactly super stable API, so if you use them, consider pinning your aiocoap version precisely. The change log will tell when things happen there.

[araldit]

The use case you describe is exactly right.

I am investigating solution number 2. If a block wise transfer is ongoing from client -> server, when does the server stop listening for the new messages - what part of the code schedules this?

[chrysn]

If a block wise transfer is ongoing from client -> server, when does the server stop listening for the new messages - what part of the code schedules this?

The timeouts are implemented in aiocoap.blockwise.Block1Spool, whose assemblies is a aiocoap.util.asyncio.timeoutdict with a timeout of MAX_TRANSMIT_WAIT.

If you want to manage the timeouts yourself, implementing a custom Block1Spool-ish type may be practical.

It'd likely not be based on TimeoutDict, but rather on a custom managed structure that produces an incomplete request message rather than dropping it; if it is combined with a custom render_to_pipe implementation, it can still send the partial message on to the handler. I recommend that that message retain some Block1 option value, by which the handler can distinguish it from fully assembled messages. The Block1 value may be technically invalid (eg. 0/1/1024 when the payload is 9KiB long), but no part in aiocoap will take offense, and the handler can then correctly grasp that this message represents an incomplete block-wise request that has been partially reassembled.