Anıl Berk Altuner

Transformers model as Judge

5

#153 Transformers library added to llm as judge. Only need is when the JudgeLLM class called, change judge model with transformers. Ex. ``` JudgeLLM( judge_model_name="microsoft/Phi-3-mini-128k-instruct", template_path="src/lighteval/tasks/extended/mt_bench/judge_prompts.jsonl", multi_turn=True, ) ```

Title: Enhancement: Update Signal Function to Accept Vector Input Issue Body: Been working with the signal function in our library and the current version of it only accepts content IDs...

Initialization problem on custom client

6

Hi, I'm working on custom client and using with stdio. ` server_params = StdioServerParameters( command=self.server_config.command, args=self.server_config.args, env=self.server_config.env, ) stdio_transport = await self.exit_stack.enter_async_context( stdio_client(server_params) ) self.stdio, self.write = stdio_transport session =...

Contribution doc for additional language model provider

1

Hello! Looking for contributors to help implement a new language model provider in Concordia, [Dria](https://github.com/firstbatchxyz/dria-sdk). So, I looking to language_model class and adapted it. But other than that, is there...