LdapSkipCertificateMockMvcTests test failures in local runs, introduced by PR 2711

[swalchemist]

Using the head of the develop branch - when I run all of the uaa unit tests on my machine like this: ./gradlew test, both tests in the LdapSkipCertificateMockMvcTests class (in the server module) consistently fail in the setup:

LdapSkipCertificateMockMvcTests > testIgnoreExpiredServerCertificate() FAILED
    java.lang.AssertionError: Status expected:<200> but was:<401>
        at org.springframework.test.util.AssertionErrors.fail(AssertionErrors.java:59)
        at org.springframework.test.util.AssertionErrors.assertEquals(AssertionErrors.java:122)
        at org.springframework.test.web.servlet.result.StatusResultMatchers.lambda$matcher$9(StatusResultMatchers.java:627)
        at org.springframework.test.web.servlet.MockMvc$1.andExpect(MockMvc.java:214)
        at org.cloudfoundry.identity.uaa.mock.util.MockMvcUtils.getClientCredentialsOAuthAccessToken(MockMvcUtils.java:1213)
        at org.cloudfoundry.identity.uaa.mock.util.MockMvcUtils.getClientCredentialsOAuthAccessToken(MockMvcUtils.java:1188)
        at org.cloudfoundry.identity.uaa.mock.util.MockMvcUtils.createOtherIdentityZoneAndReturnResult(MockMvcUtils.java:555)
        at org.cloudfoundry.identity.uaa.mock.util.MockMvcUtils.createOtherIdentityZoneAndReturnResult(MockMvcUtils.java:627)
        at org.cloudfoundry.identity.uaa.mock.util.MockMvcUtils.createOtherIdentityZoneAndReturnResult(MockMvcUtils.java:636)
        at org.cloudfoundry.identity.uaa.mock.ldap.LdapSkipCertificateMockMvcTests.setUp(LdapSkipCertificateMockMvcTests.java:70)

I have not seen these test fail on Concourse, only locally. Similar result, though, when I run the tests the way we run them in CI:

./gradlew clean
./gradlew -Dspring.profiles.active=hsqldb,default -Djava.security.egd=file:/dev/./urandom assemble --max-workers=4 --no-daemon --stacktrace --console=plain --exclude-task :cloudfoundry-identity-samples:assemble
./gradlew -Dspring.profiles.active=hsqldb,default -Djava.security.egd=file:/dev/./urandom test --no-daemon --stacktrace --console=plain --exclude-task :cloudfoundry-identity-samples:assemble

I reproduced the two failures in one run like this, but not on another, where one of the UaaTokenServicesTests flaked instead.

Running the LdapSkipCertificateMockMvcTests test class alone does not reproduce the failures. The smallest set of tests I've run that reproduced it was to run all of the unit tests in the uaa module, then concurrently run LdapSkipCertificateMockMvcTests. It seems that there is some test pollution or a race condition.

These failures were introduced in PR #2711 (which doesn't seem to be the root cause), specifically in JdbcScimUserProvisioning.getUserConfig(). If I revert the change just in that method (where the new behavior is to throw an exception), these tests pass and JdbcScimUserProvisioningTests.cannotCreateUserWithInvalidIdentityZone fails.

In one attempt at running run-unit-tests.sh, however, all the tests passed.

[strehle]

No issue in my IDE, and no running ./gradlew clean :cloudfoundry-identity-uaa:test --tests "org.cloudfoundry.identity.uaa.mock.ldap.LdapSkipCertificateMockMvcTests" same with ./gradlew -Dspring.profiles.active=hsqldb,default -Djava.security.egd=file:/dev/./urandom test --no-daemon --stacktrace --console=plain --exclude-task :cloudfoundry-identity-samples:assemble

[swalchemist]

I did some experimenting on my Mac.

1. I added two copies of the LdapSkipCertificateMockMvcTests to the tests, each with a different name. I did not reproduce the failure in the original or the copies with gradlew test.
2. I added @Diabled to the LdapSearchAndBindTest class, and then again did not reproduce the failure.
3. I renamed LdapSkipCertificateMockMvcTests as ALdapSkipCertificateMockMvcTests and then did not reproduce the failure.

This does seem to depend on which tests are run and the order they're run in.