[BUG] DataPrepper crashes when following quickstart instructions

[jmwilkinson]

Describe the bug DataPrepper crashes when following quickstart instructions

The stack trace is quite long, but the relevant part is (not terribly decipherable, but anyway)...

Reading pipelines and data-prepper configuration files from Data Prepper home directory. /opt/java/openjdk/bin/java Found openjdk version of 17.0 2023-11-08T19:02:11,203 [main] INFO org.opensearch.dataprepper.DataPrepperArgumentConfiguration - Command line args: /usr/share/data-prepper/pipelines,/usr/share/data-prepper/config/data-prepper-config.yaml 2023-11-08T19:02:11,440 [main] INFO org.opensearch.dataprepper.parser.PipelinesDataflowModelParser - Reading pipeline configuration from pipelines.yaml 2023-11-08T19:02:11,782 [main] WARN org.springframework.context.support.AbstractApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dataPrepper' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/DataPrepper.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'pipelineParser' defined in org.opensearch.dataprepper.parser.config.PipelineParserConfiguration: Unsatisfied dependency expressed through method 'pipelineParser' parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionsApplier' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionsApplier.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionLoader' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionLoader.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionPluginConfigurationConverter' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionPluginConfigurationConverter.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionPluginConfigurationResolver' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionPluginConfigurationResolver.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataPrepperConfiguration' defined in org.opensearch.dataprepper.parser.config.DataPrepperAppConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.opensearch.dataprepper.parser.model.DataPrepperConfiguration]: Factory method 'dataPrepperConfiguration' threw exception; nested exception is java.lang.IllegalArgumentException: Invalid DataPrepper configuration file.

To Reproduce

Follow the instructions: https://opensearch.org/docs/latest/data-prepper/getting-started/

pipelines.yaml

simple-sample-pipeline:
  workers: 2
  delay: "5000"
  source:
    random:
  sink:
    - stdout:

data-prepper-config.yaml This is empty since it doesn't seem to require anything by default

docker command

docker run --name data-prepper -p 4900:4900 -v ${PWD}/pipelines.yaml:/usr/share/data-prepper/pipelines/pipelines.yaml -v ${PWD}/data-prepper-config.yaml:/usr/share/data-prepper/config/data-prepper-config.yaml opensearchproject/data-prepper:latest

Expected behavior Should have the output of UUIDs as described.

[jmwilkinson]

This seems to have something to do with mapping the empty data-prepper-config.yaml file in. I think maybe the simplest fix would be to add more documentation around the requirements for that file in the getting started instructions.

[dlvenable]

Yes, I think that the current documentation directs users to run with a data-prepper-config.yaml, but it does not tell those users how to set that up. We should update the documentation for clarity.