jbolor21

MAINT Making Prompt Converters Async

1

## Description Making all prompt converters async and fixing unit tests accordingly ## Tests and Documentation Ran and created new unit tests to test async functionality

## Description Added Exception Handling to OpenAIChatInterface ## Tests and Documentation Added unit tests in the test_openai_chat_target file to ensure exception handling works

## Description Decreased the retries from 5 to 2 for unit tests to speed up runtime from

FEAT string to JSON converter

2

#### Describe the solution you'd like Add a converter using json.dumps() to make the string json formatted

## Description Adding a new target for endpoints without APIs (like Bing Image Creator). This is updated from three closed PRs: https://github.com/Azure/PyRIT/pull/350 https://github.com/Azure/PyRIT/pull/351 https://github.com/Azure/PyRIT/pull/392 Also to make this work the...

## Description Adding dataset of prompts used for testing medical bias in AI systems. Dataset is from huggingface (https://huggingface.co/datasets/katielink/EquityMedQA) ## Tests and Documentation Added new unit tests and tested with...

MAINT: Adding DB Schema Diagram

1

## Description Improving our docs on memory and adding a diagram for our db schema ## Tests and Documentation NA

FEAT: CBT-Bench Dataset

3

#### Adding this dataset of psychotherapy benchmark questions: https://huggingface.co/datasets/Psychotherapy-LLM/CBT-Bench #### Describe the solution you'd like: This dataset can be fetched and used in PyRIT!

FEAT: Add SorryBench Dataset

5

This dataset should be available within PyRIT: https://huggingface.co/datasets/sorry-bench/sorry-bench-202406) Associated paper: https://arxiv.org/pdf/2502.04322v1 _**[[Content Warning: Prompts are aimed at provoking the model, and may contain offensive content.]]**_ _Additional Disclaimer: Given the content...

## Description Adding evaluation datasets for likert scale datasets + scoring scales used + demo in notebook