What is AUC ?
AUC is the area under the ROC curve. It is a popularly used classification metric.
Classifiers such as logistic regression and naive bayes predict class probabilities as the outcome instead of the predicting the labels themselves. A new data point is classified as positive if the predicted probability of positive class is greater a threshold. Each threshold leads to a different classifier. Hence, typical metrics such as accuracy and F1 score depend on the threshold one picks. AUC for such classifiers gives an aggregated metric across thresholds.
Why do we care about AUC and the ROC curve?
AUC is popular because
- It is a threshold independant metric – Helps evaluate the model without being dependent on the specific threshold we choose
- The ROC curve is often used to chose the threshold
Some classifiers such as an SVM or a perceptron give the class labels directly as the outcome and not class probabilities.
Does is make sense to compute the AUC metric for classifiers such as the SVM which give class labels as outcome?
The answer is YES. We can compute the AUC for the SVM Classifier using techniques such as Platt Scaling.