Add mechanism for verifying that source code in documentation is valid

[andygrove]

Which issue does this PR close?

Closes https://github.com/apache/arrow-datafusion/issues/7951

Rationale for this change

Adds a mechanism for ensuring that the documentation contains valid source code. See docs README in this PR for a description of the mechanism.

What changes are included in this PR?

• Moves some source code from markdown to unit tests
• Adds include directives in the markdown
• Adds a Python script to merge the markdown and source code when building the docs

This resulted in some small improvements to the documentation for the two pages that are updated to use this mechanism:

• The udf docs no longer use let mut when creating contexts, and no longer use unwrap
• The code for building logical plans is now tested to make sure it produces the expected plan
• The published code is now formatted by cargo fmt

We can follow up with more PRs to move more code over. I suggest we create one PR per page to keep reviews short.

Are these changes tested?

Yes! Some code that was previously untested is now tested.

Are there any user-facing changes?

No

[andygrove]

@alamb I have addressed the feedback. PTAL when you have time.

[alamb]

I took the liberty of merging this branch to main.

While I was testing it it turns out there are a bunch of clippy errors in the library code now such as

error: unused imports: `SessionContext`, `create_udf`
  --> docs/src/library_udfs.rs:24:27
   |
24 | use datafusion::prelude::{create_udf, SessionContext};
   |                           ^^^^^^^^^^  ^^^^^^^^^^^^^^


error: this import is redundant
  --> docs/src/library_udfs.rs:26:1
   |
26 | use tokio;
   | ^^^^^^^^^^ help: remove it entirely
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#single_component_path_imports
   = note: `-D clippy::single-component-path-imports` implied by `-D warnings`


error: function `add_one` is never used
  --> docs/src/library_udfs.rs:29:4
   |
29 | fn add_one(args: &[ArrayRef]) -> Result<ArrayRef> {
   |    ^^^^^^^
   |
   = note: `-D dead-code` implied by `-D warnings`

However, this PR seems not to trigger the full CI checks (probably because it only runs the docs workflow): https://github.com/apache/arrow-datafusion/blob/main/.github/workflows/docs_pr.yaml

[github-actions[bot]]

Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.