Tanishk Bansode

it compiled for me. Debian(proot-distro) with venv clang-14.x.x (installed by `apt install clang`) export CC=clang export CXX=clang++ I tried b4 with clang-19 but didnt work for me, so used the...

maybe try 1. using `python3 -m venv /path/to/venv` `. /path/to/venv/bin/activate` ` pip install -e . ` 2. and also try manually installing setuptools by `pip install setuptools`

if it supports tinygrad or mlx, yes

I think the problem is that numpy doesn't support bf16 and in [sharded_inference_engine.py](https://github.com/exo-explore/exo/blob/main/exo/inference/mlx/sharded_inference_engine.py), the program tries to convert mlx tensors to numpy array hence causing the error. I updated the...

@lordoliver can you check if it slows down non-bf16 models too? or if it breaks the code in worst case scenario?

> 1. As numpy doesn't support bfloat16, updated code to check if it is bf16, and if yes, then convert to float32. > 2. also renamed the output_data and first_layer...

From my understanding of the code i've read, 50051 is used only for testing in [/test_udp_discovery.py](https://github.com/exo-explore/exo/blob/1a77a52d71417fa010d6d45984658bec81af6f8e/exo/networking/udp/test_udp_discovery.py#L10) and other test files with other ports(5052/3/4). But in main.py it uses [find_available_port](https://github.com/exo-explore/exo/blob/1a77a52d71417fa010d6d45984658bec81af6f8e/exo/main.py#L110) function...

you can run the models mentioned in [models.py](https://github.com/exo-explore/exo/blob/main/exo/models.py). As you mentioned that you are using linux, you can run all the models which tinygrad supports which is sadly just llama...

Correct me if I am wrong, but ig you can just edit the exo/models.py and add there. add ` "deepseek-v3-0324-8bit": { "layers": 61, "repo": { "MLXDynamicShardInferenceEngine": "mlx-community/DeepSeek-v3-0324-8bit", }, },` in...

There's already a issue for this error( #799 ), still not yet solved though 😔 ! It's related to tensor type bf16, even the one in the models.py ain't working...