History service: use a separate context when IO
Is your feature request related to a problem? Please describe. History shard will override the context to ensure minimal timeout, however, this checking / overriding logic does not prevent caller from cancelling the call, potentially causing shard instability (assert shard whenever see non shard ownership lost error)
thanks @nagl-temporal for finding this potential issue
To repro: Insert a short (I used 10ms) sleep in UpdateWorkflowExecution to increase the chances that context cancellation occurs during the persistence operation. start a reasonably chatty (parallelism=4) maru workload against temporal running locally. Ctrl-c your bench-go worker - you should see rangeID updates in the log from reliable-shard. You may have to do this a few times to get unlucky (or increase the sleep).
It's weird that a badly behaved client can trigger rangeID updates.