[CORE-1507] SR: Add support for `?verbose` compatibility checks

[oleiman]

Some side effects of this:

• In general, this PR keeps existing checks in place but adds reference-compatible error codes and messages to provide human-readable detail.
• On the protobuf side, this PR introduces a handful of checks that are functionally new (i.e. schemas which were previously considered compatible are no longer so):
  • ONEOF_FIELD_REMOVED
  • MULTIPLE_FIELDS_MOVED_TO_ONEOF
  • REQUIRED_FIELD_{ADDED,REMOVED}
  • FIELD_NAMED_TYPE_CHANGED
  • MESSAGE_REMOVED
    • We already detect some of these (file level) but this PR adds additional checks for nested message definitions
• Previously our compatibility checks would short circuit, returning a 👍 / 👎 as soon as a determination is made. Now we want to build up a collection of errors so that users can make all the required changes and avoid repeated checks. To do this, we need to rejigger the structure of the recursion a bit and introduce some new types for collecting errors.
• We also want to report "path" information about errors (i.e. where in the schema the error occurred), so we need to carry that information through the recursion. I used std::filesystem::path for this as a first pass, but in the end it looks fit for purpose. We could implement something similar if the cost is a concern, but I suspect most of the cost is in storing the path elements themselves, so it may be a wash. It might be nice to just not do this if verbose error tracking is not enabled.

Fixes CORE-1507

Backports Required

• [ ] none - not a bug fix
• [ ] none - this is a backport
• [ ] none - issue does not exist in previous branches
• [ ] none - papercut/not impactful enough to backport
• [x] v24.1.x
• [ ] v23.3.x
• [ ] v23.2.x

Release Notes

Improvements

• SchemaRegistry: Adds support for the ?verbose parameter on POST /compatibility/subjects/{subject}/versions/{version}

[oleiman]

force push move some compatibility code from header

[oleiman]

/dt

[vbotbuildovich]

new failures in https://buildkite.com/redpanda/redpanda/builds/48905#018f5f60-2f1f-49b5-9525-d2010401037a:

"rptest.tests.schema_registry_test.SchemaRegistryBasicAuthTest.test_post_compatibility_subject_version"

new failures in https://buildkite.com/redpanda/redpanda/builds/48905#018f5f51-1de4-47ce-b35c-e447df7d2995:

"rptest.tests.schema_registry_test.SchemaRegistryBasicAuthTest.test_post_compatibility_subject_version"

[vbotbuildovich]

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/48905#018f5f60-2f1f-49b5-9525-d2010401037a

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/48905#018f5f60-2f27-47c1-8a6a-c53f8393bb98

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/48912#018f6055-325b-4a7f-883b-d9f53a360d48

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/48962#018f65f4-a4d7-4fb0-94e8-9e3b0c3d643e

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/48962#018f65f4-a4d2-4971-aa11-9112ae7a151e

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/49069#018f757f-a9d4-405c-923d-0c9527e33c52

[oleiman]

force push typo

[oleiman]

/ci-repeat 1

[oleiman]

force push improve commits, tests, remove some junk

[oleiman]

force push remove some cruft

[oleiman]

force push rebase dev to account for breaking seastar change

[oleiman]

force push downcase enums

[oleiman]

/ci-repeat 1

[oleiman]

This project is getting deprioritized in favor of FIPS work for at least the next couple weeks. Please direct any inquiries to myself, @aanthony-rp, or @michael-redpanda

[oleiman]

subsumed by https://github.com/redpanda-data/redpanda/pull/22798, https://github.com/redpanda-data/redpanda/pull/22877, and https://github.com/redpanda-data/redpanda/pull/22958