garak
garak copied to clipboard
NVIDIA
the LLM vulnerability scanner
``` PS C:\Users\akncsjnas\Desktop\PT\LLM\garak> pip install -e . Obtaining file:///C:/Users/acsascassc/Desktop/PT/LLM/garak Installing build dependencies ... done Checking if build backend supports build_editable ... done Getting requirements to build editable ... done Preparing...
step-by-step guide * [ ] soft intro * [ ] structure * [ ] talk about how we manage the project - on github; get in touch!; open license only...
* [ ] add buffs.rst * [ ] add buffs section * [ ] add buffs docs * [ ] add buffs to doc tests
maybe this data could be filtered for a min. prefix length
e.g. for parallel_requests: ``` File "/home/lderczynski/dev_ext/garak/garak/harnesses/base.py", line 92, in run attempt_results = probe.probe(model) ^^^^^^^^^^^^^^^^^^ File "/home/lderczynski/dev_ext/garak/garak/probes/base.py", line 143, in probe attempts_completed.append(self._execute_attempt(this_attempt)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/lderczynski/dev_ext/garak/garak/probes/base.py", line 96, in _execute_attempt this_attempt.outputs =...
`garak --list_generators` skips a bunch of Hugging Face generators ``` ... generators: ggml 🌟 generators: ggml.GgmlGenerator generators: guardrails 🌟 generators: guardrails.NeMoGuardrails generators: huggingface 🌟 generators: huggingface.LLaVA generators: langchain 🌟 generators:...
no api access for me, so implement a clewd/slaude workaround * https://rentry.org/clewd / https://gitgud.io/ahsk/clewd/ * https://rentry.org/slaude2, at a pinch...
OpenAIGenerator throws pickling exception after single-attempt probes when `parallel_attempts` > 1
OpenAIGenerator throws `"TypeError: cannot pickle '_thread.RLock' object"` when invoked with `--parallel_attempts` > 1. The exception appears in a probe executed after a probe that involves only attempt. debugging journey follows:...