Chunyu comments

Results 9 comments of


                                            Chunyu

Support FSDP worker and vLLM Ascend

transformers v4.51.4 starts to support ASCEND NPU to directly enable ```flash_attention_2```. It seems that the transformers section in README needs to be adjusted.

[RFC] [sub roadmap] [25Q2] Add Ascend NPU support for OpenRLHF

[04.22] - OpenRLHF-NPU can enable ```--packing_samples```. - Update Q2 roadmap: ring_flash_attention will be supported in OpenRLHF-NPU.

[WIP] support Ascend NPU backend

Thanks for sharing this PR! I can successfully run DPO/RM on the NPU(Atlas 800T A2 Training Server) with ease. I am verifying the [Ray](https://github.com/ray-project/ray/pull/41256) part in OpenRLHF. Could you share...

support NPU?

https://github.com/OpenRLHF/OpenRLHF/issues/914 is a roadmap of the OpenRLHF-NPU workflow for reference.

Support for Ascend NPU

> Will OpenRLHF support Huawei Ascend NPU (ShengTeng AI Processor)? https://github.com/OpenRLHF/OpenRLHF/issues/914 is a roadmap of the OpenRLHF-NPU workflow for reference.

How about adding local kernel loading to `transformers.KernelConfig()`

> Hi [@zheliuyu](https://github.com/zheliuyu) ! Yes it totally makes sense, we can have a `use_local_kernels` flag inside `KernelConfig` to do that, do you want to open a PR for that ?...

How about adding local kernel loading to `transformers.KernelConfig()`

WIP on https://github.com/zheliuyu/transformers-kernels

How about adding local kernel loading to `transformers.KernelConfig()`

https://github.com/huggingface/transformers/pull/42800 PR merged. Documentation will be updated accordingly, but this issue can be closed. Thanks to everyone who followed this issue.❤

FP16 training has an anomaly on the NPU.

> [@zheliuyu](https://github.com/zheliuyu) Can you help me take a look? Thanks for the feedback. @FightingZhen will pay attention to this issue.