What are the different independence assumptions in hMM & Naive Bayes ?

Both the hMM and Naive Bayes have conditional independence assumption.

hMM can be expressed by the equation below :

    \[p(x, y)=p(x|y)p(y)\]

    \[p(x|y)p(y)\,=\,\prod_{t=1}^{T}p(x_{t}|y_{t})p(y_{t}|y_{t-1})\]

Second equation implies a conditional independence assumption: Given the state y_{t} observed variable x_{t} is conditionally independent of previous observed variables, i.e. x_{t-1},x_{t-2},... and x_{1}

Naive Bayes Model is expressed as:

    \[p(x|y)=p(x_{1},x_{2},...,x_{n}|y=y_{k})\]

x_{d} is the d^{th} feature for the data sample x and y_{k} is the k^{th} label for k class problem.

The above equation can be written as

    \[p(x|y)=p(x_{1}|y=y_{k})p(x_{2}|y=y_{k})...p(x_{n}|y=y_{k})\]

This implies a conditional independence assumption: given the class label, data features are independent of each other.

Leave a Reply

Your email address will not be published. Required fields are marked *