John-Henry Lim

Possibility of syncing cluster adapter across physical servers

7

Would a combination of Cluster Adapter and inter-server adapters like Redis Adapter be possible? See diagram below. 

run_dev.sh: using `--gpus` instead of `--runtime nvidia`

3

https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/6d3c5c00e0e2b3fc1d75eb4286848d23b05d6dca/scripts/run_dev.sh#L195-L203 I noticed `run_dev.sh`'s Docker container works (`torch.cuda.is_available()` returns True) if I replace `--runtime nvidia` with `--gpus all`. I also noticed in the dev environment setup guide (https://nvidia-isaac-ros.github.io/getting_started/dev_env_setup.html) that `nvidia-container-runtime`...

**Is your feature request related to a problem? Please describe.** PyTorch is able to install its CUDA dependencies via the above wheels during pip-install. Adding these wheels to the dependencies...

KV Cache Quantization

1

Hey, thanks for your work. I saw https://huggingface.co/ISTA-DASLab/Meta-Llama-3-70B-Instruct-AQLM-2Bit-1x16/discussions/2 about how 8-bit KV cache quantization can be enabled on vLLM. I am not too sure of how exactly KV cache is...

Use exllamav2's smart 4-bit KV cache for memory benchmark

1

See: https://github.com/turboderp/exllamav2/blob/master/doc/qcache_eval.md exllamav2 has a 4-bit KV cache that has similar ppl to unquantized cache from turboderp's testing. In practice, I find that exllamav2 uses less VRAM than llama.cpp for...

[Suggestion] Metapackage as solution to current issues with opencv as dependency

2

### Expected behaviour Depending on opencv-python has always been an issue, depending on whether libraries used `opencv-python`, `opencv-python-headless`, `opencv-contrib-python` or `opencv-contrib-python-headless`. I do know that opencv isn't designed as a...

Use HuggingFace's Quanto library KV Cache Quantization for any Transformers-based loader

1

**Description** HuggingFace's Quanto has implemented 4 bit & 2 bit KV cache quantization compatible with Transformers. See: https://huggingface.co/blog/kv-cache-quantization I may PR when I've time to experiment.

[Feature Request] Support glob patterns for window classes to force blur

1

I am using "Blur only matching" mode to avoid the performance penalty of rendering everything as transparent. However, under my kvantum theme, most KDE applications include some transparency that should...

Yes, I have seen #7284 and want to add onto it. In my use case, I am doing a bind mount of devfs `/dev` into the container while also using...

feat: comfyui create venv as needed

1

Fixes #263, #272. As comfy-cli might be installed globally via `pipx` or `uv`, as per linked issue, it ends up installing the comfyui dependencies into the managed environment. Especially since...