Bulk export fails when payload offloading is configured

[lmsurpre]

Describe the bug

[8/11/22, 17:55:29:709 UTC] 00000054 FHIRPersisten E   Unexpected error while processing token value records.
                                 java.lang.NullPointerException
	at java.base/java.util.zip.InflaterInputStream.<init>(Unknown Source)
	at java.base/java.util.zip.GZIPInputStream.<init>(Unknown Source)
	at java.base/java.util.zip.GZIPInputStream.<init>(Unknown Source)
	at org.linuxforhealth.fhir.persistence.jdbc.dao.impl.FetchResourcePayloadsDAO.run(FetchResourcePayloadsDAO.java:132)
	at org.linuxforhealth.fhir.persistence.jdbc.impl.FHIRPersistenceJDBCImpl.fetchResourcePayloads(FHIRPersistenceJDBCImpl.java:3168)
	at org.linuxforhealth.fhir.bulkdata.jbatch.export.fast.ResourcePayloadReader.readItem(ResourcePayloadReader.java:316)
	at com.ibm.jbatch.container.artifact.proxy.ItemReaderProxy.readItem(ItemReaderProxy.java:77)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.readItem(ChunkStepControllerImpl.java:329)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.readAndProcess(ChunkStepControllerImpl.java:245)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeChunk(ChunkStepControllerImpl.java:626)
	at com.ibm.jbatch.container.controller.impl.ChunkStepControllerImpl.invokeCoreStep(ChunkStepControllerImpl.java:795)
	at com.ibm.jbatch.container.controller.impl.BaseStepControllerImpl.execute(BaseStepControllerImpl.java:295)
	at com.ibm.jbatch.container.controller.impl.ExecutionTransitioner.doExecutionLoop(ExecutionTransitioner.java:118)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeCoreTransitionLoop(WorkUnitThreadControllerImpl.java:96)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.executeWorkUnit(WorkUnitThreadControllerImpl.java:178)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl$AbstractControllerHelper.runExecutionOnThread(WorkUnitThreadControllerImpl.java:503)
	at com.ibm.jbatch.container.controller.impl.WorkUnitThreadControllerImpl.runExecutionOnThread(WorkUnitThreadControllerImpl.java:92)
	at com.ibm.jbatch.container.util.BatchWorkUnit.run(BatchWorkUnit.java:113)
	at com.ibm.ws.context.service.serializable.ContextualRunnable.run(ContextualRunnable.java:79)
	at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:245)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

When I tried configuring it locally, I got this one (which makes more sense to me since fhir-persistence-blob is not a dependency of our fhir-bulkdata-webapp.

An error occurred while trying to load FHIR persistence factory class 'org.linuxforhealth.fhir.persistence.blob.FHIRPersistenceJDBCBlobFactory': java.lang.ClassNotFoundException: org.linuxforhealth.fhir.persistence.blob.FHIRPersistenceJDBCBlobFactory

Environment main

To Reproduce Steps to reproduce the behavior:

1. start a fhir server that is configured with payload offloading (e.g. via build/pre-integration-test-docker.sh azurite)
2. add a resource
3. invoke export
4. note that the export fails and check the server log

Expected behavior Either: A. Bulk export should work irregardless of the payload offload configuration; B. this should be clearly documented as a limitation of the current payload offload documentation

Additional context At the very least, I think it should be possible to fall back to our legacy "search-based" export implementation.

[lmsurpre]

I took the easy way out and just had it fall back to our search-based export. I considered opening a new "tech debt" issue for properly supporting fast-export, but after talking with robin we think export should be totally different in case of offloading because all the resources are already external.

[lmsurpre]

I worked with Prasanna to verify this one. We followed the steps from the "To Reproduce" section and we successfully exported some patients to azurite. We execed onto the azurite container and saw what looked like the expected output. Seems to be working.