PromtEngineer
Issue while running UI
Hi, While running UI, facing "Internal Server Error ". Also, terminal shows below error. * Running on http://127.0.0.1:5110 INFO:werkzeug:Press CTRL+C to quit ggml_metal_graph_compute: command buffer 0 failed with status 5 GGML_ASSERT: /private/var/folders/db/1sxlm1394c1flcjkslpxqjvm0000gn/T/pip-install-7twb1gab/llama-cpp-python_988209e5b6c341f3867dd9dfd3a1e0e4/vendor/llama.cpp/ggml-metal.m:1185: false zsh: abort python run_localgpt_api.py (rgpt) mica@Admins-MacBook-Pro localgpt_llama2 % /Users/mica/anaconda3/envs/rgpt/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '
I am running it on apple macbook pro.
Kindly guide.
Same issue here. Running it with Apple Macbook M1 (--device_type mps), after having installed llama-cpp as told for mps
@arioboo Does your optimisation work? Like, running with '--device_type mps' does it have a good and quick prompt output? Or is it slow? By, does your optimisation works, I mean do you feel in this case of running program that using M2 provide faster processing thus prompt output?
What is the average prompt output time? Not when using UI which you said you encountered problems but when using the program through CLI.
Thank you, Nitkarsh Chourasia
