openllmetry icon indicating copy to clipboard operation
openllmetry copied to clipboard

🚀 Feature: allow disabling prompt sending as an argument to Traceloop.init()

Open nirga opened this issue 2 years ago • 15 comments

Which component is this feature for?

Traceloop SDK

🔖 Feature description

Add a setting called trace_content (which will default to true) and will control whether all the instrumentations are sending sensitive content (like prompts and completions). This should override the env var behavior (similar to other flags we have).

🎤 Why is this feature needed ?

More control on this sensitive feature

✌️ How do you aim to achieve this?

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

  • [X] I checked and didn't find similar issue

Are you willing to submit PR?

None

nirga avatar Oct 31 '23 19:10 nirga

is this still available to work on?

randolfuy01 avatar Apr 10 '24 17:04 randolfuy01

Yes @randolfuy01 :)

nirga avatar Apr 10 '24 17:04 nirga

perfect I'll look into it!

randolfuy01 avatar Apr 10 '24 19:04 randolfuy01

@nirga @randolfuy01 Hi there! Is it still WIP? If not, I would love to start looking around this issue:)

sanspareilsmyn avatar May 17 '24 06:05 sanspareilsmyn

Not sure what's @randolfuy01's status, but there are other issues that you can also look at that are unassigned @sanspareilsmyn!

nirga avatar May 21 '24 20:05 nirga

@nirga Hi! Is this issue still open? If so, I would be happy to look at it :)

liamjdavis avatar Jul 16 '24 22:07 liamjdavis

Sounds good @liamjdavis!

nirga avatar Jul 17 '24 01:07 nirga

@nirga To clarify in greater detail,

  1. The setting is the flag in the sdk's config init.py 'is_content_tracing_enabled

https://github.com/traceloop/openllmetry/blob/9ad18b7067f3036a13268b50366ff4207e32ffa2/packages/traceloop-sdk/traceloop/sdk/config/init.py#L8-L9

  1. Then the setting is called in the global sdk init.py

https://github.com/traceloop/openllmetry/blob/9ad18b7067f3036a13268b50366ff4207e32ffa2/packages/traceloop-sdk/traceloop/sdk/init.py#L79

  1. And I need to write the functionality that for each instrument,

https://github.com/traceloop/openllmetry/blob/9ad18b7067f3036a13268b50366ff4207e32ffa2/packages/traceloop-sdk/traceloop/sdk/init.py#L55

the sensitive content being shared for each individual instrument is controlled.

liamjdavis avatar Jul 17 '24 20:07 liamjdavis

Yes!

nirga avatar Jul 19 '24 17:07 nirga

@nirga I've opened #1620 for this issue.

liamjdavis avatar Jul 21 '24 20:07 liamjdavis