langfuse-python icon indicating copy to clipboard operation
langfuse-python copied to clipboard

fix: ensure raw_stream.response is httpx.Response in langfuse streaming adapter

Open recursingfeynman opened this issue 3 months ago • 3 comments

When Langfuse is enabled, the object passed to OpenAI’s stream manager as raw_stream is adapter LangfuseResponseGeneratorAsync. The OpenAI manager sets self._response = raw_stream.response and then calls await self._response.aclose(). LangfuseResponseGeneratorAsync previously set response to the AsyncStream itself (not to httpx.Response), so the manager tried to call aclose() on an AsyncStream, causing:

AttributeError: 'AsyncStream' object has no attribute 'aclose'

Steps to reproduce

  • openai == 2.4.0
  • langfuse == 3.7.0
from langfuse.openai import AsyncOpenAI

request_options = {
    "model": "...",
    "messages": [{"role": "user", "content": "Hello"}],
    "n": 1,
}

async with AsyncOpenAI().chat.completions.stream(**request_options) as stream:
    async for event in stream:
        print(event)

[!IMPORTANT] Ensure raw_stream.response is httpx.Response by renaming response to stream and setting self.response to stream.response in relevant classes and functions.

  • Behavior:
    • Ensure raw_stream.response is httpx.Response in async adapter by renaming response to stream in _wrap and _wrap_async functions.
    • Set self.response to stream.response in LangfuseResponseGeneratorSync and LangfuseResponseGeneratorAsync classes.
  • Classes:
    • LangfuseResponseGeneratorSync: Renamed response to stream and set self.response to stream.response.
    • LangfuseResponseGeneratorAsync: Renamed response to stream and set self.response to stream.response.
  • Functions:
    • _wrap: Renamed response to stream.
    • _wrap_async: Renamed response to stream.

This description was created by Ellipsis for 816c68a7d57abb8363f929dcb0aded68c8e4c3e2. You can customize this summary. It will automatically update as commits are pushed.

recursingfeynman avatar Oct 17 '25 20:10 recursingfeynman

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Oct 17 '25 20:10 CLAassistant

Greptile encountered an error while reviewing this PR. Please reach out to [email protected] for assistance.

greptile-apps[bot] avatar Oct 17 '25 20:10 greptile-apps[bot]

@hassiebp Hi! Maybe add some tests?

recursingfeynman avatar Oct 29 '25 07:10 recursingfeynman