foundationdb icon indicating copy to clipboard operation
foundationdb copied to clipboard
verified publisher icon apple

Repair system data

Open liquid-helium opened this issue 4 years ago • 16 comments

Added a tool to repair system data when systemKeys on SS is lost or corrupted.

  1. User discovers system data loss/corruption, and send repair_system_data request.
  2. A master recovery is triggered.
  3. After all roles are recruited, random storage servers are selected to host the system data.
  4. The contents in txnStateStore is copied to the new team, and serverKeys are reconstructed as well.

Test: I am still working 2 bugs, but I don't expect any major changes.

Code-Reviewer Section

The general guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

  • [ ] The PR has a description, explaining both the problem and the solution.
  • [ ] The description mentions which forms of testing were done and the testing seems reasonable.
  • [ ] Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

  • [ ] This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or master if this is the youngest branch)
  • [ ] There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

liquid-helium avatar Oct 29 '21 16:10 liquid-helium

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: 02678b0b2b08ce8759df884da0d1e367509c12a5
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Oct 29 '21 16:10 foundationdb-ci

AWS CodeBuild CI Report for macOS Catalina 10.15

  • CodeBuild project: foundationdb-pr-macos
  • Commit ID: 02678b0b2b08ce8759df884da0d1e367509c12a5
  • Result: FAILED
  • Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Oct 29 '21 16:10 foundationdb-ci

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: 8d0a10464016ce9395c4b3db068d25a40d187c14
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Oct 29 '21 23:10 foundationdb-ci

AWS CodeBuild CI Report for macOS Catalina 10.15

  • CodeBuild project: foundationdb-pr-macos
  • Commit ID: 8d0a10464016ce9395c4b3db068d25a40d187c14
  • Result: SUCCEEDED
  • Error: N/A
  • Build Logs (available for 30 days)

foundationdb-ci avatar Oct 29 '21 23:10 foundationdb-ci

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: 00656defb53e663bd0287be3669dd40a5ce18ce4
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Oct 30 '21 03:10 foundationdb-ci

AWS CodeBuild CI Report for macOS Catalina 10.15

  • CodeBuild project: foundationdb-pr-macos
  • Commit ID: 00656defb53e663bd0287be3669dd40a5ce18ce4
  • Result: SUCCEEDED
  • Error: N/A
  • Build Logs (available for 30 days)

foundationdb-ci avatar Oct 30 '21 04:10 foundationdb-ci

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: 86312742dd129804fb813a2fcd8c5c3a4dbe1c38
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 01 '21 22:11 foundationdb-ci

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: c37295ef3416558a7c8f62122daa958caa461647
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 01 '21 22:11 foundationdb-ci

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: 20f587c34a2a6232a6244dce8ba6f799b2e2a19a
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 01 '21 22:11 foundationdb-ci

AWS CodeBuild CI Report for macOS Catalina 10.15

  • CodeBuild project: foundationdb-pr-macos
  • Commit ID: 20f587c34a2a6232a6244dce8ba6f799b2e2a19a
  • Result: SUCCEEDED
  • Error: N/A
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 01 '21 23:11 foundationdb-ci

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: 290071429714cd8ed90b3f16aacd3acba6d79e22
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 02 '21 15:11 foundationdb-ci

AWS CodeBuild CI Report for macOS Catalina 10.15

  • CodeBuild project: foundationdb-pr-macos
  • Commit ID: 290071429714cd8ed90b3f16aacd3acba6d79e22
  • Result: SUCCEEDED
  • Error: N/A
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 02 '21 16:11 foundationdb-ci

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: fd111a60fe651bf4f88f84902c5b6ae05daa36c9
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 15 '21 17:11 foundationdb-ci

AWS CodeBuild CI Report for macOS Catalina 10.15

  • CodeBuild project: foundationdb-pr-macos
  • Commit ID: fd111a60fe651bf4f88f84902c5b6ae05daa36c9
  • Result: SUCCEEDED
  • Error: N/A
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 15 '21 17:11 foundationdb-ci

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: fd111a60fe651bf4f88f84902c5b6ae05daa36c9
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 17 '21 00:11 foundationdb-ci

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: fd111a60fe651bf4f88f84902c5b6ae05daa36c9
  • Result: FAILED
  • Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
  • Build Logs (available for 30 days)

foundationdb-ci avatar Nov 17 '21 00:11 foundationdb-ci