NeMo-Guardrails icon indicating copy to clipboard operation
NeMo-Guardrails copied to clipboard

Fix/nvidia ai endpoints streaming

Open Pouyanpi opened this issue 1 year ago • 0 comments

This PR fixes the streaming issue introduced with langchain_nvidia_ai_endpoints package.

  • It creates a patched version of ChatNVIDIA based on the version of the langchain_nvidia_ai_endpoints package. Two patched versions are created for the version ranges 0.1. - 0.2.0 and 0.2.0 - 0.3.0. The ChatNVIDIAFactory is responsible for checking the package version and returning the corresponding patched class. The patched class is then assigned to ChatNVIDIA, which will be used in providers.

  • Imodifies the import statement in the providers module to import the patched version of ChatNVIDIA from the _langchain_nvidia_ai_endpoints_patch module.

  • it is tested on langchain_nvidia_ai_endpoints versions 0.1.3.0 - 0.1.4 - 0.1.5 - 0.1.6 - 0.1.7 and 0.2.0

Note: Only the streaming support is tested

Pouyanpi avatar Jul 26 '24 13:07 Pouyanpi