### Generative classifiers

A Generative Model explicitly models the **actual distribution of each class**. A Generative Model learns the **joint probability distribution p(x,y). **It predicts the conditional probability with the help of **Bayes Theorem**.

- Assume some functional form for
**P(Y), P(X|Y)** - Estimate parameters of
**P(X|Y), P(Y)**directly from training data - Use Bayes rule to calculate
**P(Y |X)**

Examples:

- Naïve Bayes
- Bayesian networks
- Markov random fields
- Hidden Markov Models (HMM)

### Discriminative classifiers

A Discriminative model models the **decision boundary between the classes**. A Discriminative model learns the **conditional probability distribution p(y|x)**.

- Assume some functional form for
**P(Y|X)** - Estimate parameters of
**P(Y|X)**directly from training data

Examples:

- Logistic regression
- Scalar Vector Machine
- Traditional neural networks
- Nearest neighbour
- Conditional Random Fields (CRF)s

### Some test questions

- **What are the problems these models can solve? **Classification tasks.
- **Which model learns joint probability? **The generative model.
- **Which model learns conditional probability? **the discriminative model.
- **What happens when we give correlated features in discriminative models? **They will produce similar output as they are correlated?
- **What happens when we give correlated features in generative models? **We can’t tell as we do not learn the decision boundaries?
- **Which models works very well even on less training data? **The discriminative model will work very well as only one set of parameter has to be learned?
**Is it possible to generate data from with the help of these models? Don’t know.****Which model will take less time to get trained? The discriminative model will take less time as only one set of parameter has to be learned?**- **Which model will take less time to predict output? **The discriminative model will take less time as prediction can be performed directly by the learned conditional probability. The generative model additionaly needs to apply Bayes’ Theorem.
- **Which model fails to work well if we give a lot of features? **The generative model?
- **Which model prone to overfitting very easily? **The generative model?
- **Which model prone to underfitting easily? **The discriminative model?
- **What happens when training data is biased over one class in Generative Model? **The joint probability will be biased as class probabilites will be biased. Thus, the conditional probability will be biased, too.
- **What happens when training data is biased over one class in Discriminative Models? **It happens the same like in the generative case. We will receive biased conditional probability.
- **Which model is more sensitive to outliers? **The discriminative model is more sensitive to outliers as we model the boundaries. Each outlier will shift the class boundaries.
- **Are you able to fill out the missing values in a dataset with the help of these models? **Yes — if the dataset offers features.