Types Of Errors

$$\begin{array}{|l|l|l|l|}
\hline
& & \textbf{Predicted fraud?} & \\ \hline
& & \textbf{Y} & \textbf{N} \\ \hline
{\textbf{Is it actually fraud?}} & \textbf{Y} & +/+ \text{(true positive)} & -/+ \text{(false negative – type 2)} \\ \hline
{\textbf{}} & \textbf{N} & +/- \text{(false positive – type 1)} & -/- \text{(true negative)} \\ \hline
\end{array}$$

Precision: how often a classifier is right when it says something is fraud $(\frac{\text{true positives}}{\text{true positives}+\text{false positives}})$
Recall: how much of the actual fraud that we correctly detect $(\frac{\text{true positives}}{\text{true positives}+\text{false negatives}})$

$$\begin{array}{|l|l|}
\hline
\textbf{Conservation (flag fewer transactions)} & \textbf{Aggressive (flag more transactions)}\\\hline
\text{high precision (few false positives)} & \text{low precision (many false positives)}\\\hline
\text{low recall (miss some fraud)} & \text{high recall (catch most fraud)}\\\hline
\end{array}$$

Harmonic mean of $x$ and $y$ $=$ $\frac{1}{\frac{1}{2}(\frac{1}{x}+\frac{1}{y})}$

$F_1$ $=$ $\frac{1}{\frac{1}{2}(\frac{1}{\text{precision}}+\frac{1}{\text{recall}})}=\frac{2\times\text{precision}\times\text{recall}}{\text{precision}+\text{recall}}$