# Naive Bayes classification using Scikit-Learn

## Naive Bayes classification - Sklearn¶¶

Naive Bayes classifier builds directly on conditional probability and this

$$p(y|x) = \frac{p(y \cap x)}{p(x)}$$

from the above formula, $p(y \cap x)$ can be written as

$$p(y \cap x) = p(x | y).p(y)$$

thus

$$p(y|x) = \frac{p(x|y).p(y)}{p(x)}$$

In machine learning, Naive Bayes is used to compute conditional probability of predicted class $y$ occuring given all the predictor variables $x$. In other words, Bayes theorem relates P(outcome/evidence) (what we want to predict) to P(evidence / outcome) (training set).

The algorithm builds the conditional probability and applies prediction. The algorithm is called naive because it assumes independence between predictor variables, while in reality this may not be true in all cases.

$$P(A_i | B_j) = \frac{P(B_j | A_i)P(A_i)}{P(B_j | A_1)P(A_1) + P(B_j | A_2)P(A_2) + … + P(B_j | A_k)P(A_k)}$$

Consider $A_1 ,… A_k$ as $k$ predictor variables in machine learning. The Naive Bayes classifier will build the conditional probabilities of $p(B_j|A_k)$ to later predict what would $p(A_i | B_j)$ be.

### Naive Bayes on Iris dataset¶¶

#### EDA¶¶

In [1]:
import seaborn as sns

In [2]:
iris.head()

Out[2]:
sepal_length sepal_width petal_length petal_width species
0 5.1 3.5 1.4 0.2 setosa
1 4.9 3.0 1.4 0.2 setosa
2 4.7 3.2 1.3 0.2 setosa
3 4.6 3.1 1.5 0.2 setosa
4 5.0 3.6 1.4 0.2 setosa
In [3]:
iris.shape

Out[3]:
(150, 5)
In [9]:
iris['species'].value_counts().plot(kind='bar')

Out[9]:
<matplotlib.axes._subplots.AxesSubplot at 0x1a0d0e03c8>
In [13]:
%matplotlib inline
sns.pairplot(iris, hue='species')

Out[13]:
<seaborn.axisgrid.PairGrid at 0x1a17a970b8>

#### Create features and labels¶¶

In [2]:
X_iris = iris.drop('species', axis=1)
X_iris.shape

Out[2]:
(150, 4)
In [3]:
y_iris = iris['species']
y_iris.shape

Out[3]:
(150,)

X_iris is in caps as X is a vector of multiple features for each record, whereas y_iris is in small case as it is a scalar for each record.

In [ ]: