What are the different independence assumptions in hMM & Naive Bayes ?

Both the hMM and Naive Bayes have conditional independence assumption.

hMM can be expressed by the equation below :

$p(x, y)=p(x|y)p(y)$

$p(x|y)p(y)\,=\,\prod_{t=1}^{T}p(x_{t}|y_{t})p(y_{t}|y_{t-1})$

Second equation implies a conditional independence assumption: Given the state $y_{t}$ observed variable $x_{t}$ is conditionally independent of previous observed variables, i.e. $x_{t-1},x_{t-2},...$ and $x_{1}$

Naive Bayes Model is expressed as:

$p(x|y)=p(x_{1},x_{2},...,x_{n}|y=y_{k})$

$x_{d}$ is the $d^{th}$ feature for the data sample $x$ and $y_{k}$ is the $k^{th}$ label for $k$ class problem.

The above equation can be written as

$p(x|y)=p(x_{1}|y=y_{k})p(x_{2}|y=y_{k})...p(x_{n}|y=y_{k})$

This implies a conditional independence assumption: given the class label, data features are independent of each other.

Leave a Reply Cancel reply