Use with open source LLM model?

[Trawmoney]

Wondering if possible to run with models like llama2 or huggingface models, or Ollama or something like litellm.

[setomage]

LLM stands for Large Language Model, LLAMA2 are the LLMs that Meta released. These and image making models are normally found on Hugging Face, and sadly are in a different ballpark.

[Daniel15]

A large language model wouldn't be useful for reinforcement learning.

[RTriplett]

There was a recent paper where GPT-4 was used in conjunction with reinforcement learning, but I'm not sure how it could be adapted to this since it reads the source code to generate reward functions https://eureka-research.github.io/

[fangyuan-ksgk]

As @RTriplett pointed out, Eureka is predicated on the assumption that there is access to the environment's source code. However, the scenario differs in the case of the Pokemon environment, seemingly drawing closer parallels to Voyager https://github.com/MineDojo/Voyager. But to my understanding, Voyager takes advantage of the fact that 'observation' of any object is given explicitly to us, this means some basic maneuver like 'kill the mob', 'pick up something', 'craft something' can be coded and serve as the 'control primitives'. I wonder if the pokemon situation is the same...

[RTriplett]

It looks like Voyager relies on Mineflayer https://github.com/PrismarineJS/mineflayer to view interact with the world I'm sure something similar could be done woth GPT-4, it seems to have a good idea what should be done next in the game, but seem to not quite as much in spirit of the RL project