binbabou comments

Repositories
Issues
Comments

Results 5 comments of


                                            binbabou

Support masking for LSTM, RNN?

LSTM does not support mask currently, will it be supported later?

Support for `LayerNormalization` from ONNX opset 17

Same issue. Is there any plan to support rewrite LayerNormalization（opset 17）when use tf‘s savedmodel format？

[Bug]: VLLM 0.8.2 OOM error (No error in 0.7.3 version)

+1 with qwen2.5-vl-awq 0.8.2， same parameters with 0.8.1 is ok。

Support disaggregated prefill ?

So does it support multiple P nodes and multiple D nodes? If supported, please provide an example, e.g. 1P2D

[Bug] InternVL2-2B的推理速度慢，发现是视觉特征提取的耗时很长

internvl2-2B，一张(131, 259)的图片，to cpu耗时80ms，preprocess+vision model加起来才30ms 0.5几的版本开启过prefix-caching，VLM结果不正确，我理解目前只适用于LLM 现在用0.6.3，我打算直接禁掉to cpu这步操作了...