This PR fixes many of the "quiet database" errors in simulation tests running version vector, caused by peeks waiting for versions that never arrive during recovery.

in version vector, a locked tLog may be requested for a version beyond what the tLog has. During recovery tLogs are peeked up to the RV, but this can higher than logData->version, as tLogs in version vector advance at different rates.

To avoid deadlocking, return with the end version received in the request, which is the RV if set to a valid version during recovery. Do not do this if the version is not set or is not an valid version. For example, the SS may also be peeking tLogs during recovery, but they may not yet know the RV; they set the end version to infinity.

The PR modifies log routers instantiated for recovery to receive the RV in their initiation message, as do tLogs. This ensures they send a valid end version (RV) during recovery in peek messages.

This PR removes the previous solution, in which RPCs were sent to tLogs during recovery with the RV.

Joshua 20241008-140414-dlambrig-a9091fcbb91cc075 (commit 7)

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

[x] The PR has a description, explaining both the problem and the solution.
[x] The description mentions which forms of testing were done and the testing seems reasonable.
[x] Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

[ ] This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
[ ] There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

Sep 23 '24 15:09 dlambrig

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 16f7fb217dd100c574a9baf19563663ccc468ae4
Duration 0:06:24
Result: :x: FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Sep 23 '24 15:09 foundationdb-ci

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 16f7fb217dd100c574a9baf19563663ccc468ae4
Duration 0:11:09
Result: :x: FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Sep 23 '24 15:09 foundationdb-ci

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: 16f7fb217dd100c574a9baf19563663ccc468ae4
Duration 0:16:50
Result: :x: FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Sep 23 '24 15:09 foundationdb-ci

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: be140c5c2602c6bf7ef9c867a3ff7b19f425024d
Duration 0:07:00
Result: :x: FAILED
Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /opt/homebrew/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

Sep 23 '24 15:09 foundationdb-ci