Knowledge Distillation

Posted on June 12, 2024 by MLNerds

This video talks about model compression and what knowledge distillation is. It talks about the distillation loss and the common frameworks employed for knowledge distillation.

CategoriesMachine Learning, Natural Language Processing

PreviousPrevious post: Positional Encoding in the Transformer Model

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Comment

Name *

Email *

Website

Notify me of follow-up comments by email.

Notify me of new posts by email.

Search for:

For more FREE Interview prep material: Subscribe with LinkedIn or Google

Tags

ACID properties activation function analytics anomaly detection atomicity bayesian modeling bi-grams bias in ML classification consistency decision boundary deep learning durability embeddings gender bias gradient descent hmm inner join join language model left join logistic regression maximum likelihood estimate metrics missing data MLE model complexity Natural Language Processing NLP optimization outer join outlier detection overfitting recall right join search sigmoid smoothing speaker segmentation speech sql tanh text generation topic modeling viterbi

Sign-In With Your Social ID

We hope you enjoy going through our content as much as we enjoy making it !