sparseml icon indicating copy to clipboard operation
sparseml copied to clipboard

Square head distillation implementation for new SparseML framework

Open markurtz opened this issue 2 years ago • 0 comments

Example recipe for what this enables:

OutputDistillationModifier:
    targets: ['layer.1', 'layer.2']
    transforms: []
    comparison: "square_head"
    orig_scale: 1.0
    distill_scale: 1.0

markurtz avatar Oct 20 '23 15:10 markurtz