Bootstrap Drupal correctly from inside SimpleSAMLphp

[longwave]

This was previously reported in #75 but closed as outdated, however it is still an issue.

When certain caches are cold or stale, it is possible for them to be rebuilt during a request that is wrapped by SimpleSAMLphp instead of Drupal's own index.php. However when this happens, the cache is not rebuilt correctly and are missing critical data. This then results in various errors related to missing fields on the Drupal site such as:

Attempted to create an instance of field with name field_XXX on entity type profile when the field storage does not exist.
Attempted to create an instance of field with name body on entity type block_content when the field storage does not exist.
Field layout_builder__layout is unknown.

or errors with missing plugins:

The "field_item:language" plugin does not exist. Valid plugin IDs for Drupal\Core\TypedData\TypedDataManager are: ...

This is because DrupalHelper::bootDrupal() does not fully bootstrap Drupal correctly before calling core services which may go on to initialise caches. In particular the DrupalKernel::preHandle() method is not called which is responsible for loading all modules, which causes the rebuilt caches to skip items from those modules and cause the errors later down the line.

As per the comment if you look at FunctionalTestSetupTrait::initKernel() or even /authorize.php you will see the correct way of partially bootstrapping Drupal without calling DrupalKernel::handle() is to call preHandle() (which calls loadLegacyIncludes() but also does a bit more).

[sonarqubecloud[bot]]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[RoSk0]

Hi Dave,

Thanks for this!

I have a client with a similar symptoms where we suspected Memcached to be an issue, but I haven't had a chance to look at the problem in detail.

I will try to test this sometime soon.

Do you have a steps to reproduce this? As I'm missing reliable step on the mentioned project.

[longwave]

Yeah, I don't have reliable steps to reproduce, but anecdotally it did happen more frequently when the discovery cache bin was stored in Redis instead of the database. This had been happening for several months on a production site, but was only triggered maybe once or twice a month, and did not happen in other environments.

I tracked it down by swapping out EntityFieldManager and logging a stack trace in ::cacheSet() when an entity that I knew should have fields was cached with no fields. When this happened the stack trace always started in SimpleSAMLphp's module.php. This led me to #75 and since applying the MR here we haven't seen the issue again.

I have since tried to reproduce locally by manually clearing caches at specific times but so far have had no luck.

When discussing this in Drupal Slack I was previously pointed to https://www.drupal.org/project/drupal/issues/3207813 - this could also be considered a bug in Drupal as it should not try to build caches without checking all parts are fully loaded, but I think for the most robust fix we should bootstrap Drupal correctly from within this SimpleSAMLphp module as well.

[longwave]

Further to this, this doesn't entirely solve the issue, there is a further bug in Drupal core: https://www.drupal.org/project/drupal/issues/3441833