openllmetry icon indicating copy to clipboard operation
openllmetry copied to clipboard

🚀 Feature: support emitting events for prompts in addition to current behavior

Open nirga opened this issue 1 year ago • 19 comments

Which component is this feature for?

All Packages

🔖 Feature description

Fix the following packages and their tests:

  • [ ] AlephAlpha
  • [ ] Anthropic
  • [ ] Bedrock
  • [ ] Cohere
  • [ ] Google Generative AI
  • [ ] Groq
  • [ ] Langchain
  • [ ] LlamaIndex
  • [ ] Mistral AI
  • [ ] Ollama
  • [ ] OpenAI
  • [ ] Replicate
  • [ ] SageMaker
  • [ ] Together
  • [ ] Transformers
  • [ ] VertexAI
  • [ ] WatsonX

🎤 Why is this feature needed ?

Full compliance with new semantic conventions

✌️ How do you aim to achieve this?

See new OpenAI instrumentation for inspiration.

  • [ ] Make sure to keep the current way of emitting events and the new (event-based) way with the official semantic conventions.
  • [ ] Add a config to each instrumentation called use_legacy_attributes which defaults to true, and if set to false emit events instead of the prompt / completion attributes.
  • [ ] Propagate a new initialization parameter of the SDK called use_legacy_attributes that sets this config in all instrumentations.

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

  • [X] I checked and didn't find similar issue

Are you willing to submit PR?

None

nirga avatar Jan 02 '25 15:01 nirga

/bounty $1200

nirga avatar Jan 02 '25 15:01 nirga

💎 $1,500 bounty • Traceloop (YC W23)

Steps to solve:

  1. Start working: Comment /attempt #2456 with your implementation plan
  2. Submit work: Create a pull request including /claim #2456 in the PR body to claim the bounty
  3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

❗ Important guidelines:

  • To claim a bounty, you need to provide a short demo video of your changes in your pull request
  • If anything is unclear, ask for clarification before starting as this will help avoid potential rework
  • Low quality AI PRs will not receive review and will be closed
  • Do not ask to be assigned unless you've contributed before

Thank you for contributing to traceloop/openllmetry!

Attempt Started (UTC) Solution Actions
🔴 @akv2011 Jan 02, 2025, 04:06:44 PM WIP
🟢 @itsparser Jan 02, 2025, 06:10:34 PM WIP
🟢 @maliniv1530 Jan 02, 2025, 07:31:59 PM WIP
🟢 @gerred Jan 02, 2025, 11:35:19 PM WIP
🟢 @sanjay7178 Jan 03, 2025, 03:35:43 AM WIP
🔴 @SAIKIRANSURAPALLI Jan 03, 2025, 05:57:40 AM WIP
🟢 @aayushdhiman01 Jan 05, 2025, 09:11:09 AM WIP
🟢 @MrKrishnabhagat Apr 09, 2025, 09:55:22 AM #2809 Reward
🟢 @LuizDMM Apr 10, 2025, 12:52:42 AM #2810 Reward
🟢 @GauravSanMod Jan 12, 2025, 10:09:03 AM WIP
🟢 @naaa760 Jan 18, 2025, 07:45:56 AM WIP
🟢 @onyedikachi-david Jan 22, 2025, 03:23:44 PM #2541 Reward

algora-pbc[bot] avatar Jan 02 '25 15:01 algora-pbc[bot]

@nirga is there a hard deadline to generate pr?

akv2011 avatar Jan 02 '25 16:01 akv2011

@akv2011 ideally by Monday 6/1

nirga avatar Jan 02 '25 16:01 nirga

/attempt #2456

Options

0xvasanth avatar Jan 02 '25 18:01 0xvasanth

/attempt #2456

akv2011 avatar Jan 02 '25 19:01 akv2011

/attempt #2456

Options

maliniv1530 avatar Jan 02 '25 19:01 maliniv1530

/attempt #2456

Options

gerred avatar Jan 02 '25 23:01 gerred

/attempt #2456

Options

sanjay7178 avatar Jan 03 '25 03:01 sanjay7178

/attempt #2456

Options

SAIKIRANSURAPALLI avatar Jan 03 '25 05:01 SAIKIRANSURAPALLI

@nirga wanted to update progress Have implemented event logs and sdk implementation Working on tests. will raise pr by 8/1

maliniv1530 avatar Jan 05 '25 19:01 maliniv1530

/attempt #2456

Options

naaa760 avatar Jan 18 '25 07:01 naaa760

Add an additional configuration option to control which types of events are emitted:

emit_prompt_events: Emit events for prompts (default: true).
emit_completion_events: Emit events for completions (default: true).

naaa760 avatar Jan 18 '25 07:01 naaa760

/bounty $1500

nirga avatar Jan 20 '25 18:01 nirga

/attempt #2456

Algora profile Completed bounties Tech Active attempts Options
@onyedikachi-david 14 bounties from 7 projects
TypeScript, Python,
JavaScript & more
Cancel attempt

onyedikachi-david avatar Jan 22 '25 15:01 onyedikachi-david

💡 @onyedikachi-david submitted a pull request that claims the bounty. You can visit your bounty board to reward.

algora-pbc[bot] avatar Jan 22 '25 15:01 algora-pbc[bot]

/attempt #2456

MrKrishnabhagat avatar Apr 09 '25 09:04 MrKrishnabhagat

/attempt #2456

LuizDMM avatar Apr 10 '25 00:04 LuizDMM

@nirga Is the issue still open? Would like to contribute.

srikary12 avatar Apr 27 '25 13:04 srikary12

🎉🎈 @LuizDMM has been awarded $750 by Traceloop (YC W23)! 🎈🎊

algora-pbc[bot] avatar May 24 '25 14:05 algora-pbc[bot]