PeiqinSun
PeiqinSun
1. 这个repo里面的确vgg16_bn算的和readme不一致. 原因是代码: https://github.com/lmbxmu/HRankPlus/blob/master/cal_flops_params.py#L36 我理解是少了最后一层的剪枝和fc计算量增加导致计算不一致.
Please provide your installation details.
Closing due to lack of response from @wslgqq277g
从输出的角度来看, 同一个beta相当于这个channel什么信息也没有学到. 故剪掉是合理的.
I think this issue is from your misunderstanding. Please check your code to verify it.
There is no reply for 7 days, so we close the issue.
we will release FQ-ViT soon.
hi, @elven2016. Do you have encounted a phenomenon that the loss value change to upward trend after one epoch(total 3-epoch) ? like: 
是可以的. 你们只要显存能放得下就没有问题. 由于推荐大家使用2080ti的原因是因为我们目前的kernel只写了关于compute capability=75(即2080系列)的实现。如果你们想使用更多系列的kernel也可以向我们提开发需求, 我们会尽快支持的.
We also try to implement 4bit-qlora, thanks to the optimized kernel implementation of back-propagation, the fine-tuning speed is similar to 8-bit lora at present. Welcome to use and issue: https://github.com/megvii-research/Sparsebit/tree/main/large_language_models/alpaca-qlora