hiyijian comments

Results 21 comments of


                                            hiyijian

Train model with apex

I had the same problem. We need a way to exclude SW layer from O2 just like BN. But I have not found a proper way

Slimming Resnet

Thanks. Do you think the sparsity will be effected if BN layers on main branch are not penalty by L1 norm. If yes, how? Thanks

ROC question

how about the finnal ROC performance on FDDB please? Is it also the same as original one ?

ibv_devinfo output "Failed to open device"

Yes. The Plugin only support for RoCE now?

ibv_devinfo output "Failed to open device"

@paravmellanox is there any update now? Thanks

ibv_devinfo output "Failed to open device"

@addcloud I am not an expert at network stuff at all. I used to stuck in enabling SRIOV for a quite long time. The reason for failing to enable it...

My train model using train_celeba.yml

These is no network initialization in this repo. Probably, this is the reason why we get totally diffrient results by using CUDA10.2 and CUDA 9.2

Significant performance drops when using fast memory efficient attention

@danthe3rd I also need alibi support. for now, I pass ```bias = LowerTriangularMaskWithTensorBias(alibi_bias)``` to ```xops.memory_efficient_attention(..., attn_bias=bias )```. The forward only is ok, but failed at backward in training mode. Is...

A mismatch found in your code and paper.

@borisfom Maybe another mismatch: wgrad_norm in your code is computed from "g + beta* w"(it is computed after regularization), not exactly the same as paper's "g".