ChezzPlaya

Compiled app.getGraph() throws an exception

3

### Checked other resources - [X] I added a very descriptive title to this issue. - [X] I searched the LangChain documentation with the integrated search. - [X] I used...

Llama 3 models not stopping when using VLLM as a client

2

Steps to repoduce: ```python import dspy vllm_llama3_70b = dspy.HFClientVLLM(model="meta-llama/Meta-Llama-3-70B-Instruct", port=8000, url="http://localhost") dspy.configure(lm=vllm_llama3_70b) #Example DSPy CoT QA program qa = dspy.ChainOfThought('question -> answer') response = qa(question="What is the capital of Australia?")...

[Bug] Mixtral-8x7B-Instruct-v0.1-q4f16_1-MLC can't run

4

## 🐛 Bug FileNotFoundError: Cannot find the model library that corresponds to `None` when running mixtral ## To Reproduce I followed this example for usage: https://github.com/mlc-ai/mlc-llm/pull/1529#issue-2063018397 ``` from mlc_chat import...

[Question] Support constraining via context free grammar for dsls

2

Hi @lbeurerkellner, Do you have any plans to "natively" integrate token constraint into the lmql language, perhaps through ATLR/Lark/ENBF grammar notation? This is a feature currently supported by guidance (https://github.com/guidance-ai/guidance?tab=readme-ov-file#context-free-grammars)...

web_demo doesn't work

2

I tried to run the gradio demo as follows: `python web_demo.py --from_pretrained cogcom-base-17b --local_tokenizer lmsys/vicuna-7b-v1.5 --bf16` resulting in this error: ``` drawn_imgs = gr.Gallery( ^^^^^^^^^^^ TypeError: Gallery.__init__() got an unexpected...

How to handle sliders and swipes?

1

So, from the provided paper only various click actions accross different domains have been shown? What about sliders and swipes? These do need a from (x, y) and to (x,...

How to inference in 4bit?

1

When I load the model in 4bit: ```python tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen-VL-Chat", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("cckevinn/SeeClick", device_map="auto", trust_remote_code=True, load_in_4bit=True, do_sample=True, temperature=1e-3).eval() model.generation_config = GenerationConfig.from_pretrained("Qwen/Qwen-VL-Chat", trust_remote_code=True) ``` I get the following error...