What is Layer Normalization

Posted on by MLNerds
Normalization of features is very common in ML pipelines. In Deep learning models, normalization of the intermediate activations helps combat ‘internal covariate shift’ that might hinder the learning process.


This brief video talks about Layer Normalization, a popular normalization technique used in recent generative models like GPT: Why we need it, how it is done, how it is different from batch norm and how we can incorporate it in code.

Leave a Reply

Your email address will not be published. Required fields are marked *