PMI : Pointwise Mutual Information, is a measure of correlation between two events x and y.
![\[PMI(x,y)\,=\,log(\frac{p(x,y)}{p(x)p(y)})\]](https://machinelearninginterview.com/wp-content/ql-cache/quicklatex.com-369e9e84e4abfd8ed92c281dcd6e0ec2_l3.png "Rendered by QuickLaTeX.com")
![\[log(\frac{p(x,y)}{p(x)p(y)})\,=\,log(\frac{number\,of\,times\,both\,occur\,together}{(number\,of\,times\,x\,occur)*(number\,of\,times\,y\,occur)})\]](https://machinelearninginterview.com/wp-content/ql-cache/quicklatex.com-d183cdb40eda67e108d13daf5dd83818_l3.png "Rendered by QuickLaTeX.com")
As you can see from above expression, 
is directly proportional to the number of times both events occur together and inversely proportional to the individual counts which are in the denominator. This expression ensures high frequency words such as stop-words are penalised.
pPMI: pPMI is the positive PMI. PMI is often used as a feature value within a word vector obtained through distributional semantics. Since can take any real value, for natural language tasks, it is common to compute pPMI as :
![\[pPMI\,=\,max(0, PMI)\]](https://machinelearninginterview.com/wp-content/ql-cache/quicklatex.com-0fc4639158d67ed160fcd7eb7fbb32e5_l3.png "Rendered by QuickLaTeX.com")