sparseml
sparseml copied to clipboard
Square head distillation implementation for new SparseML framework
Example recipe for what this enables:
OutputDistillationModifier:
targets: ['layer.1', 'layer.2']
transforms: []
comparison: "square_head"
orig_scale: 1.0
distill_scale: 1.0