Naren Gogineni

Integrate Portkey LLM Provider

3

## Description Add support for Portkey Notes: - Refactored to move initialisation of the LLM Config object inside the LLM Providers. This is to allow extensibility depending on Provider, the...

### 🚀 The feature Integrate Portkey ### Motivation, pitch Portkey exposes a standardized OpenAI like interface and allows users to switch between LLM providers with ease, this reduces code redundancy...

**Description:** (optional) - Add support for transcriptions and translations routes **Related Issues:** (optional) - #488 **TODO** - [ ] Refactor the code **Refactoring to do** - [ ] Think of...

**Description:** - Added new route and handler for /audio/speech **Related Issues:** - #488 **Testing Done** - [X] Generated audio in all the supported formats for /audio/speech - [X] Error Response...

Feat: Provider Tests

1

**Title:** - Tests for providers to make sure they are OpenAI compliant Initial idea for this sprung from reading the very descriptive GitHub issue raised by @raulraja https://github.com/Portkey-AI/gateway/issues/116 **Motivation:** (optional)...

**Title:** - Make anthropic usage object in streaming mode compliant with openAI response **Description:** - Moved prompt_tokens to streamState object - used the streamState object to access the prompt_tokens **Related...

### What Happened? Anthropic transform currently sends completion_tokens and prompt_tokens in separate chunks ### What Should Have Happened? Openai format is to send them together in the last chunk ###...

**Title:** - Support Llama hosted API Model on Vertex https://console.cloud.google.com/vertex-ai/publishers/meta/model-garden/llama3-405b-instruct-maas **Description:** (optional) - Created transform and config for llama inside vertex - Handled JSON streaming response **Related Issues:** (optional) -...

### What Would You Like to See with the Gateway? Add support for https://console.cloud.google.com/vertex-ai/publishers/meta/model-garden/llama3-405b-instruct-maas ### Context for your Request _No response_ ### Your Twitter/LinkedIn _No response_

### What Would You Like to See with the Gateway? add functional/unit tests to make the gateway robust ### Context for your Request _No response_ ### Your Twitter/LinkedIn _No response_