Support context cancellation in Gateway when serving car files

[iand]

Checklist

• [X] My issue is specific & actionable.
• [X] I am not suggesting a protocol enhancement.
• [X] I have searched on the issue tracker for my issue.

Description

This is a low priority general enhancement to conserve resources when dealing with slow clients. Car files may be very large and take a long time to transfer and we should respect cancellation of the request context (due to timeout or explicit).

As of commit 862ce6bb8f6ce24b91ed3dc59e7406faed34f583, car files are served in a single non-cancelable call https://github.com/ipfs/go-ipfs/blob/862ce6bb8f6ce24b91ed3dc59e7406faed34f583/core/corehttp/gateway_handler_car.go#L76

Supporting context cancellation would require go-car to support cancellation when writing and traversing a dag. Currently go-car v1 is being used. The top level function WriteCar accepts a context but it is not used for cancellation. The bulk of the work is performed by merkledag.Walk which only respects context cancellation in the parallel case, which requires an explicit concurrency option to be passed. Changing merkledag.sequentialWalkDepth to check for context cancellation would work here.

Alternatively switching to go-car v2 is an option, using NewSelectiveWriter but this would also require cancellation behavior to be pushed down into go-ipld-prime's traversal package. But see https://github.com/ipld/go-ipld-prime/issues/154 and https://github.com/ipld/go-ipld-prime/issues/303

[lidel]

@schomatis mind adding this to your queue?

Only ask from my end is to keep returning CAR v1.

[schomatis]

@lidel Can I have some context on https://github.com/ipfs/kubo/pull/8758, please? Contrary to what I'm reading from Ian we seem to be in IPLD land and not merkledag (I'm only familiar with the second). I'm looking at traverseBlocks()/walkAdv_iterateSelective() but have no idea where is the traversal actually taking place and where would I plumb the context.

[iand]

See https://github.com/ipfs-shipyard/gateway-prime (here) for a version of the gateway that uses IPLD and go-car v2

[schomatis]

Thanks for the reference @iand. I'm still not sure if that is an answer to my request above or some more general context.

[iand]

Thanks for the reference @iand. I'm still not sure if that is an answer to my request above or some more general context.

Just general context