Yanan Xie

https://github.com/aio-libs/aiohttp/issues/5375#issuecomment-791034670 solved the problem for me.

There are two reasons why I am looking for FasterTransformer support: 1. storywriter is used for generating long texts. This takes longer and requires more efficient inference to shorten the...

I am able to get reasonable result from FasterTransformer + MPT-7B-Storywriter with 2 changes to FasterTransformer: 1. After [src/fastertransformer/kernels/unfused_attention_kernels.cu Line 1287](https://github.com/NVIDIA/FasterTransformer/blob/eb9b81b65909cb14f582581c1ed4ee8e1e299be9/src/fastertransformer/kernels/unfused_attention_kernels.cu#L1287), add `val = max(-6.f, min(val, 6.f));`. This is for...

> I am running into this same issue. The provided script to convert storywriter HF to FT leads to this error: > > `RuntimeError: clip_qkv is enabled for this MPT...

Yes. But in your example, it's a StreamingDataset-specific pre-processing function. What I need is to provide a Stream-specific pre-processing function. Or is there a way to create a mixture with...

We are using ClearML to send tasks to GPUs. The main process spins up other processes for DDP training. Each task can use up to 8 GPUs. We are not...

Nope. I am using torch.multiprocessing to start sub processes.

Please prioritize the enhancement. I spent one hr on this because of bad documentation.