Add support for collecting usage data from plugin responses

[willsmanley]

I need to track token usage on a per-user and per-session basis as I'm sure many others do as well.

It would be nice to configure llm like this:

def token_usage_callback(usage_data):
    print(usage_data)

llm = openai.llm(
   model="gpt-4o", 
   token_usage_callback=token_usage_callback
)

and then you can store all of that usage data as needed for analytics and attribution

[willsmanley]

hi @theomonnom what do you think about enabling usage attribution1 from plugins like this? is this already on your roadmap?

[yuyuma]

On Slack https://livekit-users.slack.com/archives/C07FY8WHGPM/p1729050553562499 David pointed me to this issue.

I just wanted to add that I'm also looking for usage data from plugin responses, specifically for realtime agent framework. The information is being sent on the wire through OpenAI websockets but is just not currently exposed by LiveKit.

[davidzhao]

feature added in #984, will be released in 0.11.0

[yuyuma]

@davidzhao perhaps I've missed something but I took a brief look through #984 and didn't see anything related to exposing token usage for openai realtime plugin. was this issue only intended to bring usage data for voice pipeline agents for now?

[davidzhao]

realtime plugin isn't exposing it yet.. tracking this effort in #1010