Minh-Thuc

Rviz can't be build in meta-ros1-melodic

1

Hello, I'm trying to use meta-ros on `branch zeus` for my card which run only version zeus of yocto. i used steps in [wiki].(https://github.com/ros/meta-ros/wiki/OpenEmbedded-Build-Instructions#add-meta-ros-to-an-existing-openembedded-project) I want to run the simulation...

tensor parallel by nccl + mpi

1

WIP for the feature tensor parallel. There are some points to investigate: - Make new version of converter to move forward the number heads before the appearance of weight, bias...

Convert the model (currently support opennmt-py) and save to the memory => inference without saving model. Customizing the wrapper to save the memory used. TODO: implementation for other converters

Mixtral support

2

I tried to support Mixtral by implementing MoE. The current problem is the inaccurancy in tensor after passing the MHA layer causes the wrong selection between the experts for certain...

This changes will enable the library to be integrated with conan2.

Quantzation AWQ GEMM + GEMV

3

Support quantization 4 bit with AWQ. There are 2 stable versions available: ``gemm`` and ``gemv``. Currently, I only add AWQ for Llama and Mistral converter. Other models could be added...

Currently, the Pypi hub of Ctranslate2 exceeds 20GB, the release 4.3.0 is missing for python 3.8 and 3.9. Request is sent to pypi to upgrade the limit. Some points to...