PromtEngineer
I ran the code successfully but can't receive the answer
I experience the same issue - No response, My setup:
- LInux Debian 11 (bullseye)
- 2 Nvidia T4 GPUs with 16GB GPU memory each
- cuBLAS enabled with
CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python==0.1.83 --no-cache-dir
@Jane1702 @RobertoDonPedro @faiconda are you using the default model?
@Jane1702 @RobertoDonPedro @faiconda are you using the default model?
Yes, but I tried it with several other models as well, including the GPTQ models since I have 2 GPUs. But it just does not answer.
@Jane1702 @RobertoDonPedro @faiconda are you using the default model?
Yes , I used the default model
I just found the issue in my case. It was not the model, but this line in the run_localGPT.py file:
query = input("\nEnter a query: ")
It does not read the provided input and waits forever. If I replace this line manually in the code with e.g. query = "Summarize your content" (and add a break at the end of the while loop) everything works as expected.
I think this is caused by running localGPT with VSCode on a remote VM. However, still wondering what's the issue with VSCode in this regard.
Excuse-me , when I changed the code as you said , it finished the code but it didn't return the answer
Excuse-me , when I changed the code as you said , it finished the code but it didn't return the answer
..then the root of your issue is something else than in my case. You did ingest the data, right?
I have just run it successfully. I ran again with command prompt and it worked . Thank you very much for your help ^^
However , I don't know why it can't read some .txt files ?
Just solved my issue. It wasn't related to VS Code but to the fact that I called it from a nodejs app via shelljs and shelljs does not accept input. Now I call it natively via child_process.execFileSync with the stdio: 'inherit' parameter and now it works as expected. So in summary my issue was neither related to localGPT nor to VS Code but to they way I called localGPT. Potentially @Jane1702 you could also doublecheck the way you call it.
I don't know why but today I tried again and it worked normally . But it takes so long to have 1 answer (more than 20mins)
