bene-ges

# What does this PR do ? Add a one line overview of what this PR aims to accomplish. **Collection**: [Note which collection this PR will affect] # Changelog -...

Hi, I was trying to launch `federate/cross_silo/cuda_rpc_fedavg_mnist_lr_example`, mapping all processes (1 server and 2 clients) to a single gpu. it ended with error ``` File "/home/myhome/.local/lib/python3.10/site-packages/torch/distributed/rpc/backend_registry.py", line 235, in _validate_device_maps...

There are some typos "salve" instead of "slave" in identifiers [1](https://github.com/FedML-AI/FedML/blob/3f3e7c7aafe9fac51343110406bc0a9e7d097f16/python/fedml/fa/cross_silo/client/client_initializer.py#L55) [2](https://github.com/FedML-AI/FedML/blob/3f3e7c7aafe9fac51343110406bc0a9e7d097f16/python/fedml/computing/scheduler/slave/client_runner.py#L1224) [3](https://github.com/FedML-AI/FedML/blob/3f3e7c7aafe9fac51343110406bc0a9e7d097f16/python/fedml/computing/scheduler/slave/client_runner.py#L1226) [4](https://github.com/FedML-AI/FedML/blob/3f3e7c7aafe9fac51343110406bc0a9e7d097f16/python/fedml/cross_cloud/client/client_initializer.py#L88) [5](https://github.com/FedML-AI/FedML/blob/3f3e7c7aafe9fac51343110406bc0a9e7d097f16/python/fedml/cross_silo/client/client_initializer.py#L88) [6](https://github.com/FedML-AI/FedML/blob/3f3e7c7aafe9fac51343110406bc0a9e7d097f16/python/fedml/serving/client/client_initializer.py#L80)

fix bug in set_device_map for trpc

1

device_list is a dictionary and we need to map to its values, not keys. Issue: https://github.com/FedML-AI/FedML/issues/2002

Во вложении пример файла, на который ctc-инференс стабильно возвращает пустую транскрипцию, проверено на двух разных машинах с разными видеокартами. [common_voice_ru_35728771.zip](https://github.com/salute-developers/GigaAM/files/15138667/common_voice_ru_35728771.zip) Это из набора mozilla common voice (cv-corpus-12.0-delta-2022-12-07) и на нем...

Eval bug: inference of 32B eats too much memory on ROCM HIP (5x AMD Radeon Instinct Mi50 (gfx906))

5

### Name and Version ``` ./llama-cli --version ROCm calling rocblas_initialize as a workaround for a rocBLAS bug ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 5 ROCm devices: Device 0:...

## ❓ General Questions While the inference speed is 2-3 times faster than llama.cpp, I observe some metrics degradation. For example, I have a simple test to do some punctuation/capitalization/correction...

[Bug] Rope doesn't work for llama-3

1

## 🐛 Bug When I pass prompt longer than 8192, e.g. 10000 tokens the model returns gibberish ## To Reproduce Steps to reproduce the behavior: 1. Run server mlc_llm serve...