When caching adapters, cache the adapter ID + the API token pair

[noah-yoshida]

Feature request

When we cache adapters, we should cache the adapter ID + the API token pair. Even if the adapter is already on GPU memory, we should ensure that the caller has access to it by maintaining a cache of adapter ID + api token pairs.

Motivation

Otherwise, we could get situations where one users calls prompt w/ a private HF hub adapter and HF key, it works and is cached, then another user could call prompt w/ the same adapter without setting a HF api token in the request. Since the adapter is cached, the request works.

Your contribution

I can try to implement it, but I am quite busy so not sure when I can get to it.

[safimuhammad]

hello, I would like to work on this.

[magdyksaleh]

Hey @safimuhammad - wanna chat on discord for next steps on this?

[safimuhammad]

@magdyksaleh Sure, here's my discord user name msafi38

[safimuhammad]

hey @magdyksaleh , reaching you out on discord, lets discuss next steps on this.