rebuff Explicitly set limits for prompt length

Hi, I'm getting this error 502 Server Error: Bad Gateway for url: https://alpha.rebuff.ai/api/detect when I use detect_injection() with long user inputs. A couple of sentences is fine, but a longer DAN prompt doesn't work. Appreciate any help I could get with this, thanks!

Jul 12 '23 22:07 QUANGLEA

Hey @QUANGLEA, thanks for raising!

Jul 22 '23 13:07 woop

I am also getting same issue, is there any way something like calculate the tokens or string size and pass into chunks like tiktoken or something? Thanks.

Aug 04 '23 09:08 cis-muzahid

@cis-muzahid what would the idea API look like for you if we were to break up the prompt into chunks? That seems like something the caller should do, not the PI library. What do you think?

Aug 04 '23 14:08 woop

Hi @QUANGLEA and @cis-muzahid. Thanks for reporting! I've edited the issue title so that Rebuff will explicitly set what the maximum prompt length is for this endpoint. After we've achieved that we can discuss if we should support more optimal handling or if that's on the caller to do.

Do you have a reproducible example we can use to debug?

Oct 04 '23 01:10 seanpmorgan