JOB
JOB
Did you ever get this issue resolved? I am getting the same error
@bormanst I tried your solution but I get the following errors. What version of TensorRT/TensorRT-LLM did you build was it 0.5 or higher? Did you manage to build the engine?...
Scratch that. I had the the wrong path for the model directory and I am back to getting `assert False, "Quantized checkpoint format not supported!"`
I think I will have to try to get 0.7.0 to compile again but I am getting linker issues due to the batch manager not existing and I had to...
@bormanst Thanks for all the info. I’ll work with 0.7.0 on windows and keep trying. Did you build with cpp bindings? I’m guessing that’s why I am getting the issue...
Ah ok. That makes sense that you are using the prebuilt wheel. I was trying to compile the wheel from source. I will give it a go tomorrow with the...
@maazaahmed I believe the instructions for main/0.8 have now changed. Check the readme for changes to building examples/llama/README.md