form-extractor-prototype icon indicating copy to clipboard operation
form-extractor-prototype copied to clipboard

Add ability to compare different LLMs

Open timpaul opened this issue 1 year ago • 3 comments

The tool currently uses Claude, but other LLMs could also do this task. We need to be able to compare the effectiveness of multiple models, so we can choose the right one

timpaul avatar Apr 23 '24 06:04 timpaul

Perhaps a 'settings' page where you can both set the API and the model you want to use?

timpaul avatar May 02 '24 08:05 timpaul

You could look at OpenRouter as an option to easily switch between models. They have a Javascript/Typescript API which would make integration easy. With OpenRouter you'd also be able to automatically switch between models if there are outages, price hikes, or performance issues.

nicolasperez19 avatar May 30 '24 13:05 nicolasperez19

Thanks @nicolasperez19 - I'll take a look at that :-)

timpaul avatar May 30 '24 13:05 timpaul