MetaDistil icon indicating copy to clipboard operation
MetaDistil copied to clipboard

verified publisher icon JetRunner
→

Metadata

Code for ACL 2022 paper "BERT Learns to Teach: Knowledge Distillation with Meta Learning".

Results 3 MetaDistil issues
Sort by recently updated
recently updated
newest added

Hard to reproduce the results of GLUE benchmark

15 comment

Thanks for your excellent work. I tried to do grid search on the settings that you described in your paper and codes, but it is still hard for me to...

Harry-zzh avatar Harry-zzh

Teacher could not acquire grads from student's cross entropy loss

According to the article, in the second step, we calculate CrossEntropy loss of S' on samples from a separate quiz set, and then calculate the gradients of CE loss with...

TheBobbyliu avatar TheBobbyliu

Teacher model

Hi there, Could you share the teacher model on MRPC dataset? I have tried many hyper-parameters to reproduce the results but all of them failed.

JinyangGuo avatar JinyangGuo

← Metadata

79
Stars
14
Forks
Watchers

Owner

verified publisher icon JetRunner

Metadata

Code for ACL 2022 paper "BERT Learns to Teach: Knowledge Distillation with Meta Learning".

Back