Sung Kim

Followed the instruction - running locally. Runs once then fails afterward

5

# Install the tools pip3 install openai pip3 install ./llm-tool/. llm run TheBloke/Llama-2-13B-Ensemble-v5-GGUF 8000 python3 querylocal.py Actual Result: Works! Run python3 querylocal.py again Fails *************http://localhost:8000/v1************* Traceback (most recent call last):...

AssertionError: daemonic processes are not allowed to have children

7

I keep on getting this AssertionError when I try to use FlagEmbedding outside of the provided sample. Can you provide an example of getting around this problem?

Installed intel-extension-for-transformers and I get an error - No module named 'intel_extension_for_pytorch'"

6

I am using Arc770 GPU on Windows 11 1. I have installed WSL2 2. I have installed miniconda 3. I follow instruction - "pip install intel-extension-for-transformers" 4. Run the example...

Docker image (intelanalytics/ipex-llm-xpu): Documentation stated I would need to disable iGPU to use A770. When will you fix this issue since disabling iGPU is problematic?

6

Docker image (intelanalytics/ipex-llm-xpu): Documentation stated I would need to disable iGPU to use A770. When will you fix this issue since disabling iGPU is problematic?

Is there a docker container I can use?

2

I am looking for something like this, so I can run this on a single 3090. docker run --gpus 1\ -e HF_TOKEN=$HF_TOKEN -p 8000:8000 \ ghcr.io/mistralai/mistral-src/vllm:latest \ --host 0.0.0.0 \...

How would one go about running embedding as a service using something like vLLM?

5

I would like to run embedding as a service using something like vLLM on a Docker container on different host. How would one go about doing this?

Enhancement: Increase 1,000 non-zero elements limit to 1,200 for sparsevec when creating HNSW index

4

HNSW Index Current: sparsevec - up to 1,000 non-zero elements (added in 0.7.0) Enhancement Request: sparsevec - up to 1,200 non-zero elements

Feature: Add support for the sparse vector, such as SPLADE

6

### SDK Python ### Description Question: Do you support the sparse vector, such as SPLADE?

Feature: Support for sparse vectors for example SPLADE

4

### SDK Python ### Description LanceDB currently only support dense vector and fts. I would like add support for sparse vector.