Getting frequent replicate.exceptions.ModelError on small, text only queries using python API, llama-3-70b-instruct model

[gthaker]

I get this error. If I rerun it usually goes thru fine but dies again down the road. (I am trying 300 total question from a professional exam.) I had no trouble running this with llama-2-70b model, i did it many times.

Traceback (most recent call last):
  File "/export/home/gthaker/repos/radonc_inSvc_LLM/run_llm.py", line 58, in <module>
    key, answer, delta = ask_llama_2(ques)
  File "/export/home/gthaker/repos/radonc_inSvc_LLM/llama_2_api.py", line 35, in ask_llama_2
    output = replicate.run(model,
  File "/export/home/gthaker/repos/radonc_inSvc_LLM/venv/lib/python3.10/site-packages/replicate/client.py", line 157, in run
    return run(self, ref, input, **params)
  File "/export/home/gthaker/repos/radonc_inSvc_LLM/venv/lib/python3.10/site-packages/replicate/run.py", line 61, in run
    raise ModelError(prediction.error)
replicate.exceptions.ModelError