How to fine-tune pre-trained model on a smaller dataset?

[alsbhn]

I was wondering how it is possible to fine-tune the pre-trained model on a smaller dataset? What about the implementation of coverage mechanism during the fine-tuning? Do you propose specific settings for hyperparameters (learning rate for example) and the number of iterations?