tool_calling_api
tool_calling_api copied to clipboard
Add captum, inspect, and llm as judges
Feature Request
Please add captum, inspect, and llm as judge components in the project. This will allow for broader evaluation capabilities and integration with advanced model analysis tools.
Requested features to add:
- captum for model interpretability of a encoder-decoder model
- inspect review PII detection, toxicity and prompt injection
- llm as a judge to review the logs incase of any issues
Motivation
Adding these judges will enhance model interpretability, debugging, and assessment by leveraging specialized tools.
Expected Outcome
- All three judges (
captum,inspect,llm) are available and can be called as part of the evaluation pipeline. - Documentation is updated to reflect their usage.