yan-virin comments

Results 3 comments of


                                            yan-virin

Invalid response format. Expected a list of dictionaries with keys 'verdict'

Same problem with models like gpt2 and google/flan-t5-large from hugging face. My code is: from langchain_community.llms import HuggingFaceHub model = HuggingFaceHub(repo_id="google/flan-t5-large", huggingfacehub_api_token="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") print(evaluate(dataset, llm=model)) I am getting: /usr/local/google/home/yvirin/ragas_env/lib/python3.11/site-packages/ragas/evaluation.py:276: RuntimeWarning: Mean...

Invalid response format. Expected a list of dictionaries with keys 'verdict'

Is there someone who was able to run RAGAS without openai api key successfully? If so, could you please share your setup?

Invalid response format. Expected a list of dictionaries with keys 'verdict'

@jjmachan While technically using other models should work, in practice it seems like it just does not work. I am wondering if there is anyone in the community who actually...