Azure
FEAT Reporting feature for PyRIT
Is your feature request related to a problem? Please describe.
I am looking for a way to have a report generated for all the attacks I generate using different Orchestrators.
Describe the solution you'd like
A report either a csv etc. that gives me insights of what attacks were successful and what were not. Also, attacks breakdown would certainly help that would inform me about the different areas of LLM breakdowns.
Tldr:
I am looking for a module that could generate a single/multiple reports based on different attack strategies.
Yes, this is some time we think about a lot. The tricky part is making it generic with a CSV.
The raw conversations: Multi-turn/multi-piece conversations need to be grouped together, in order, and with one row per piece, probably followed by all related scores (?) or alternatively scores on the same line as the corresponding piece.
Aggregates: Then there would be a set of summary stats by orchestrator, by harm category, etc. I could imagine a set of plots to be added as well.
Notably, all of this would be pulled from the DB, so some filters (by time period, by other metadata, by user, etc) would be useful.
If anyone is interested in contributing any of these parts please reach out.
print_conversation_async prints the whole conversations that an orchestrator produced, but no aggregate results. A nice overview might look like this:
=======================
Attack success rate (ASR): 55/100 = 0.550
=======================
This would need to be functionality that lives on the result object, currently called OrchestratorResult but subject to change with #945.
A great first step would be to just print out the overall attack success rate. Follow-up steps could include:
- [ ] also printing the ASR by harm category (prompts should have
harm_categoriesset, e.g., here) - [x] printing ASR by orchestrator type (e.g., prompt sending orchestrator vs. Crescendo vs. TAP vs. ...)
- [ ] printing ASR by converter type (e.g., translation converter, base64, etc.)
- [ ] printing ASR by other metadata such as labels (e.g., to differentiate between different models that you're attacking to understand which one is more susceptible to attacks; or languages) or dataset names (since people often want to compare ASR on datasets)
- [ ] print examples that resulted in high scores (let's say
> 0.9for float scale, orTruefor True-False) to give an idea of what the worst cases look like - [ ] export summary stats to a JSON / CSV file similar to how we're exporting prompts in this notebook
- [ ] compare multiple types of scores (objective achieved, various harm specific scorers, refusals, etc.) in addition to ASR
- [ ] generate charts (using matplotlib) for some of the above including
- [ ] bar chart with samples per harm category/dataset
- [ ] bar chart with ASR per harm category/dataset/orchestrator/converter/orchestrator+converter/labels
- [ ] heatmap chart showing which combination of orchestrator (one axis) and converters (other axis) worked best including basic prompt sending without converters as baseline
- [ ] do all of the above and dump the results into a PDF report
Example of existing report generation code (that I wouldn't necessarily recommend but it may serve as inspiration). Part of the issue there was that harm categories didn't propagate from seed prompts (think: the input dataset) to the prompt memory entries (think: the resulting sent prompts in the DB). This should be addressed soon. Until then, you can't really leverage harm categories. Everything else is very much still relevant, though. It's also fairly custom tailored to the scorers we were using in that particular operation but reporting code should be generic and work for any kind of scorer.
# this code creates a CSV file with a table that has a few categories in either dimension and lists the corresponding ASR
from pyrit.common.csv_helper import write_csv
from collections import defaultdict
for model_name in model_names:
# these are memory labels set on the orchestrator, so the orchestrator attaches them to each memory record in the DB
# that makes these attributes by which we can query
memory_labels = {
"op_name": op_name,
"user_name": "romanlutz",
"model_name": model_name,
}
datasets_list = list(datasets.keys())
converters_set = set()
stats = defaultdict(lambda: {"total": 0, "refusal": 0, "Hate > 0": 0, "Hate > 5": 0, "Sexual > 0": 0, "Sexual > 5": 0, "Violence > 0": 0, "Violence > 5": 0, "SelfHarm > 0": 0, "SelfHarm > 5": 0})
pieces = memory.get_prompt_request_pieces(labels=memory_labels) # type: ignore
stats["overall"]
for piece in pieces:
stats["overall"]["total"] += 1
stats[piece.labels["test_name"]]["total"] += 1
stats[piece.labels["converters"]]["total"] += 1
converters_set.add(piece.labels["converters"])
for score in piece.scores:
if score.score_type == "true_false":
if score.score_category == "refusal":
if score.get_value():
stats["overall"][score.score_category] += 1
stats[piece.labels["test_name"]][score.score_category] += 1
stats[piece.labels["converters"]][score.score_category] += 1
elif score.score_type == "float_scale":
# this particular scorer has a scale up to 7, so scale from 0-1 to 0-7 and highlight how many items were at least 1 and at least 6
severity_value = score.get_value() * 7
if severity_value > 0.5: # at least 1
stats["overall"][f"{score.score_category} > 0"] += 1
stats[piece.labels["test_name"]][f"{score.score_category} > 0"] += 1
stats[piece.labels["converters"]][f"{score.score_category} > 0"] += 1
if severity_value > 5.5: # at least 6
stats["overall"][f"{score.score_category} > 5"] += 1
stats[piece.labels["test_name"]][f"{score.score_category} > 5"] += 1
stats[piece.labels["converters"]][f"{score.score_category} > 5"] += 1
else:
raise ValueError(f"Unknown score type: {score.score_type}")
reformatted_stats = []
for sub_category, values in stats.items():
reformatted_stats.append({
"Dataset Slice": sub_category,
"Total": values["total"],
"Refusal": values["refusal"],
"Hate > 0": values["Hate > 0"],
"Hate > 5": values["Hate > 5"],
"Sexual > 0": values["Sexual > 0"],
"Sexual > 5": values["Sexual > 5"],
"Violence > 0": values["Violence > 0"],
"Violence > 5": values["Violence > 5"],
"SelfHarm > 0": values["SelfHarm > 0"],
"SelfHarm > 5": values["SelfHarm > 5"]
})
# sorting so that overall is at the top, then all the datasets, finally converters
reformatted_stats = [
r for r in reformatted_stats if r["Dataset Slice"] == "overall"
] + [
r for r in reformatted_stats if r["Dataset Slice"] in datasets_list
] + [
r for r in reformatted_stats if r["Dataset Slice"] in converters_set
]
file_name = f"{op_name}-{model_name}.csv"
with open(file_name, "w", newline='') as file:
write_csv(file, examples=reformatted_stats)
I want to plus one the request for statistics about each campaign. In addition, I'd like to suggest:
Currently we have limited output formats (JSON & CSV) that dump all output from a run. By adding output formats that are better suited for inclusion in reports we can make it easier for users to quickly get understandable results to their customers. This would need to be a combination of deciding on what data to include and what formats to use.
Likely output formats: HTML, RTF, JSON Possible configuration options:
- Limited export: only show successful tests and only include the prompts that gave success.
- Moderate export: Same as Limited but also include statistics on how many attempts were made and how many were successful vs. failed)
- Detailed export: Same as Moderate but also include the failed goals and associated prompts
- Repeatable export: produce a notebook that can be replayed to run all the same attempts
