DefTruth
DefTruth
目前还没有在CUDA 12.0的host上测试过，你再容器内运行nvidia-smi是正常的吗？
暂时没有添加OCR模型
本次更新主要是升级推理库
> WTAL soon! Can you update the test? ok, I will add a test for prefix prefill kernel with alibi later.
The local test passed. Waiting for all kernel tests of CI to pass. ```bash pytest test_prefix_prefill.py ================================================================================== test session starts =================================================================================== platform linux -- Python 3.10.13, pytest-8.1.0, pluggy-1.4.0 configfile: pyproject.toml...
@rkooo567 @simon-mo hi~ Can you take a look at this PR? All CI tests have passed. I need this fix for the bloom model. many thanks ~
@rkooo567 alibi tests have been moved to test_prefix_prefill.py. Kernel tests in CI have passed.
> So the logic change itself is equivalent to non-alibi case right? > > Also, @WoosukKwon it'd be great if you can take a look at alibi slope testing. It...
@rkooo567 seems many tests interrupted by a signal
@youkaichao @rkooo567