🚀 Feature: Log prompts while preserving user PII with Presidio

[clemlesne]

Which component is this feature for?

All instrumentations backend

🔖 Feature description

Remove PII from prompts before sending it, with Presidio.

🎤 Why is this feature needed?

Today, either we log prompts to monitoring backend or nothing. Monitor those details are important for prompt quality and A/B testing. But, this includes sending PII to the monitoring backend, which is in most companies not conceived to handle this type of data (GDPR if you hear me).

[nirga]

Thanks @clemlesne that's a great idea! We can maybe add it at the SDK level on the span processor. I wonder if there's any latency or performance impact.

[romeroej-inch]

Presidio works very badly on languages other than english. just a heads up. we ended up using another PaaS MSFT service (Cogservices). But perhaps it would make sense to have a plugin interface process PII before loggin it.

[nirga]

Or potentially this might be something that should be done on the ingestion side? I worry about creating a significant overhead on production devices.