Pipelines checking commits that delete submodules can leave their contents in place after merge

[nickpascucci]

We recently saw this log in a passing build for a PR pipeline (redacted):

Initialized empty Git repository in /tmp/build/get/.git/
Switched to a new branch 'master'
Submodule '<path>/<submodule>' (https://github.com/<org>/<repo>) registered for path '<path>/<submodule>'
Cloning into '/tmp/build/get/<path>/<submodule>'...
Submodule path '<path>/<submodule>': checked out '261056d0f518aa60a50be02e20ddf9e6a5a370ac'
Updating 7b8274a3..e5fedaa8
warning: unable to rmdir '<path>/<submodule>': Directory not empty
Fast-forward

The submodule directory was deleted in the commit being tested, but the contents were left in place by the pipeline. As a result, the build passed when it should not have.

The build used the dev tagged resource, which as of now has SHA sha256:0f5a63bfee64002fe621bd39fa43a593c1db1ec94cb7b7ddee0d63c425466a9e

Happy to provide any additional information needed to diagnose.