lua-nginx-module icon indicating copy to clipboard operation
lua-nginx-module copied to clipboard

time filter for shdict get_keys

Open Rockybilly opened this issue 1 year ago • 2 comments

I have stumbled on a use case where I need to get to most recent items from the shared dict on the get_keys method. For example the ones added in the last 60 seconds. I didn't find a way yet, probably has to be implemented. I think of two ways.

  1. Add a creation_time to the ngx_http_lua_shdict_node_t struct, but I didn't go through all the code to say that this is really needed. This doesn't seem favorable as it increases the memory usage,
  2. add a parameter to get_keys function like ttl_remaining_above? so that this value can be used as a condition while getting the keys (expiry_time - now > param). This seems easier to implement and less taxing.

I think this would be a nice addition, what do you think?

Rockybilly avatar Apr 18 '24 09:04 Rockybilly

in your case, you could perhaps add a shared dictionary named recent_keys. when writing data to other shared dictionaries, also write a key with a TTL of 60 seconds to recent_keys. this way, when you need to retrieve keys written in the past 60 seconds, you can use recent_keys:get_keys(). just an idea.

tzssangglass avatar Apr 19 '24 07:04 tzssangglass

in your case, you could perhaps add a shared dictionary named recent_keys. when writing data to other shared dictionaries, also write a key with a TTL of 60 seconds to recent_keys. this way, when you need to retrieve keys written in the past 60 seconds, you can use recent_keys:get_keys(). just an idea.

Yes I thought of this. Also looping through all keys and getting all the ttl values and do like ttl_init - ttl type of filtering, however first one is not flexible if I want to filter by a parameter, and second one is just inefficient.

As I go through the docs I feel more like ngx.shared.DICT is lacking very trivial features although it was implemented a long time ago :) Just a comment and not a criticism.

Rockybilly avatar Apr 19 '24 11:04 Rockybilly