Flowise icon indicating copy to clipboard operation
Flowise copied to clipboard

[BUG] Crash when upserting documents via API with S3 storage and env-only config

Open mspasov opened this issue 9 months ago • 0 comments

🐛 Describe the Bug

I have a document store using the JSON File loader (though this likely applies to any loader handling multipart uploads). My deployment runs on AWS ECS with S3 configured as the storage backend, and all configuration is provided exclusively via environment variables in the container.

When attempting to upload a new file for upsertion via the API endpoint:

POST https://<HOST>/api/v1/document-store/upsert/c0a42c42-08ba-426f-9284-ffbacb67e1ca

Flowise crashes with the following error:

2025-04-23 07:36:39 [INFO]: ⬆️ POST /api/v1/document-store/upsert/c0a42c42-08ba-426f-9284-ffbacb67e1ca
--
2025-04-23 07:36:39 [ERROR]: uncaughtException: ENOENT: no such file or directory, open ''
Error: ENOENT: no such file or directory, open ''
    at Object.readFileSync (node:fs:449:20)
    at getOrgId (/usr/local/lib/node_modules/flowise/dist/utils/index.js:1529:42)
    at S3Storage.key [as getKey] (/usr/local/lib/node_modules/flowise/dist/utils/index.js:1551:33)
    ...

It appears the issue originates from getMulterStorage() invoking getOrgId(), which fails because it's unable to locate settings.json.

⚠️ This causes an unhandled exception and crashes the application.

Root Cause (Suspected)

The root cause seems to be an implicit dependency on settings.json for retrieving the organization ID. In environments like ECS, Docker, or Kubernetes—where stateless, ephemeral containers are preferred—it is not always viable to persist local files.

📌 For services meant to run in containerized or clustered environments, it is highly advisable to support configuration solely via environment variables, rather than requiring filesystem-based config (e.g., settings.json). Otherwise, users are forced to mount volumes or bake configs into custom Docker images.


🧪 Steps to Reproduce

  1. Launch a Flowise container configured via environment variables (no config files).
  2. Create a data source.
  3. Create a new JSON Loader.
  4. Upload and upsert the initial file — this should succeed and show up in the S3 bucket.
  5. Attempt to upload a second file via POST /api/v1/document-store/upsert/<Store UUID>.

✅ Expected Behavior

The file should be successfully upserted without requiring a settings.json file or crashing the application.


🛠️ Environment

  • Platform: AWS ECS (Docker)
  • Configuration: Environment variables only, S3 storage.
  • Docker Image: flowiseai/flowise:2.2.7-patch.1
  • Flowise Version: 2.2.7-patch.1

mspasov avatar Apr 23 '25 15:04 mspasov