ColumbusAI comments

Repositories
Issues
Comments

Results 13 comments of


                                            ColumbusAI

calling exe from ssh on WSL2 will cause `UtilAcceptVsock:244: accept4 failed 110`

To clarify, the error I receive is this: WSL (5984) ERROR: UtilAcceptVsock:248: accept4 failed 110 It is not resolved by enabling or disabling Interop as I have tried both methods.

[FEATURE] Sharded Quantization to Enable Consumer GPUs to Quantize 80B+ LLMs

Thoughts?

[FEATURE] Sharded Quantization to Enable Consumer GPUs to Quantize 80B+ LLMs

> [@ColumbusAI](https://github.com/ColumbusAI) This would be a good feature to have but has two issues. Even if model weight is split via tensor parallel for forwarding code, the gptq quantization part...