Poorly calibrated probabilities but good classification in confusion matrix

Issue

This Content is from Stack Overflow. Question asked by mapleleaf

I have an imbalanced data set. My goal is to balance sensitivity and specificity via the confusion matrix. I used glmnet in r with class weights. The model does well at balancing the sensitivity/specificity, but I looked at the calibration plot, and the probabilities are not well calibrated. I have read about calibrating probabilities, but I am wondering if it matters if my goal is to produce class predictions. If it does matter, I have not found a way to calibrate the probabilities when using caret::train().



Solution

This question is not yet answered, be the first one who answer using the comment. Later the confirmed answer will be published as the solution.


This Question and Answer are collected from stackoverflow and tested by JTuto community, is licensed under the terms of 
CC BY-SA 2.5. - CC BY-SA 3.0. - CC BY-SA 4.0.
people found this article helpful. What about you?