Queue mode - redis filling over time - workers became inoperable until main was restarted

[Zeouterlimits]

Describe the bug

flowise-main's redis data seems to be growing over time flowise-worker's redis filled up and continued to log:

ReplyError: OOM command not allowed when used memory > 'maxmemory'. script: ef9feb35a83fa9a8c571510f20e2d57c75f68fd3, on @user_script:212.
    at parseError (/usr/src/node_modules/.pnpm/[email protected]/node_modules/redis-parser/lib/parser.js:179:12)
    at parseType (/usr/src/node_modules/.pnpm/[email protected]/node_modules/redis-parser/lib/parser.js:302:14) {
  command: {
    name: 'evalsha',
    args: [
      'ef9feb35a83fa9a8c571510f20e2d57c75f68fd3',
      '11',
      'bull:flowise-queue-upsertion:wait',
      'bull:flowise-queue-upsertion:active',
      'bull:flowise-queue-upsertion:prioritized',
      'bull:flowise-queue-upsertion:events',
      'bull:flowise-queue-upsertion:stalled',
      'bull:flowise-queue-upsertion:limiter',
      'bull:flowise-queue-upsertion:delayed',
      'bull:flowise-queue-upsertion:paused',
      'bull:flowise-queue-upsertion:meta',
      'bull:flowise-queue-upsertion:pc',
      'bull:flowise-queue-upsertion:marker',
      'bull:flowise-queue-upsertion:',
      '1757510336381',
      <Buffer de 00 04 a5 74 6f 6b 65 6e d9 29 65 34 39 65 34 64 61 30 2d 31 62 64 39 2d 34 37 35 35 2d 39 62 35 66 2d 63 38 63 31 32 38 36 35 38 66 39 30 3a 33 31 ... 33 more bytes>
    ]
  }
}

Resolved when we restarted flowise-main

To Reproduce

Unclear

Expected behavior

1. Clearer error
2. Rather than filling up, a eviction of oldest jobs etc would be done

Screenshots

flowise gui:

flowise-worker redis:

Flow

No response

Use Method

pnpm start

Flowise Version

3.0.2

Operating System

Linux

Browser

None

Additional context

No response

[HenryHengZJ]

any luck modifying REMOVE_ON_AGE and REMOVE_ON_COUNT https://docs.flowiseai.com/configuration/running-flowise-using-queue

[Zeouterlimits]

We do have:

QUEUE_REDIS_EVENT_STREAM_MAX_LEN:     2000
REMOVE_ON_COUNT:                      2000
REMOVE_ON_AGE:                        86400

I guess we can try lower them, it's hard to measure the impact though.