Alex Cheema
Alex Cheema
Anything I can help with @risingsunomi? I would really like to get this merged ASAP as a lot of people are trying to run on Linux and running into some...
> > Anything I can help with @risingsunomi? I would really like to get this merged ASAP as a lot of people are trying to run on Linux and running...
@risingsunomi have you pushed your latest changes? I still get the issue with torch not being in the setup.py script
Error on `pip install`: ``` ERROR: Could not find a version that satisfies the requirement torch==2.4.0+cu124 (from exo) (from versions: 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.3.1, 2.4.0, 2.4.1) ERROR: No matching...
Getting this output now after ``` exo --inference-engine pytorch --run-model llama-3.1-8b ```
Yes, it looks like a good fit for exo since the architecture is parallelisable and requires a lot of memory.
isnt there a double slash with the default value? @jshield other than that looks great
> @AlexCheema we good to merge this now? Yep lgtm
I really like this idea. @RayFernando1337 suggested something similar when we were running into issues with slow download for Llama 3.1 405b. Lets use this as a design discussion.
@stephanj you’re going too much into implementation details. Turn the ai tool off for now and let’s try to think about what we actually want here at a high level