[BUG] scan doesn't work when using only flash storage

[fvigotti]

Describe the bug scan doesn't work when using only flash storage

To reproduce ( single instance, no cluster , docker.io/eqalpha/keydb:latest ( as of today ) )

save "" appendonly no

storage-provider flash /opt/dbstorage/flash storage-provider-options "use_direct_reads=true;allow_mmap_reads=false;use_direct_writes=true;allow_mmap_writes=false" ( also storage-provider-options are not documented but I've found those on the web )

enter some keys ( 10000+ ) in the database , restart the db or enter more keys that exceed the maxmemory

Expected behavior keydb-cli scan command once interated should return the iterator and should allow to iterate all keys, with ~3gb db ( 10k keys) KEYS * # return all keys albeit being superslow ( reading from disk ) keydb-cli scan # iterated return only 114 keys ( I've tried everything, from count 10000 , to count 10 and iterating pointers manually ) even after KEYS * command the scan doesn't work ( return always the same amount of keys )

[msotheeswaran-sc]

Currently KeyDB does not implement scan with FLASH, but we will add it to our list of features to add before official FLASH release.

[keithchew]

Hi

I can also confirm this affects modules that call RedisModule_Scan. Using flash with a dataset larger than memory, only the records within memory will be scanned.

[MrBlaise]

Hey!

I am facing this issue right now. I have a housekeeping service that regularly checks all the keys in redis against a metadata server which relies on SCAN. Anyone has any recommendation how to scan all they keys when using FLASH mode accurately?

[keithchew]

hi @MrBlaise, you will need to use the KEYS command in the meantime...