foundationdb icon indicating copy to clipboard operation
foundationdb copied to clipboard

Use multiple proxies when committing large transactions

Open etschannen opened this issue 6 years ago • 0 comments

If mutations and conflict ranges for a large transaction were divided across multiple proxies, we could reduce the latency of the commit.

The mutations need to be committed in order, so the client will include a part number, the number of parts, and a unique ID with each large commit it divides up.

The proxy will not do any batching with commits thats have been divided into parts.

The master will remember the version it assigned to the unique ID, so that it gives out the same version to each proxy.

The resolvers and tlogs will wait for all parts before processing a version.

The proxy will add the part number to the version for mutation logging related to backup and DR. Backup and DR will need to know how to correctly interpret this part number, so we will need to be careful with the protocol version.

etschannen avatar Jan 02 '20 20:01 etschannen