Correlation Is Not Enough: Selecting Features Without Throwing Away Predictive Information
Dropping every feature whose correlation with the target is below 0.1 feels like hygiene. It is often the opposite: you are deleting signal your model could have used, and you will never see it in your metrics because the comparison you would need is the one you never ran.
Most of us learned feature selection from a heatmap. Compute Pearson's r between each column and the target, sort, keep the top k, move on. It is fast, explainable to a stakeholder in one slide, and wrong often enough to matter.
What correlation actually measures.
Pearson's r answers a narrow question: if I draw the best straight line through this feature against the target, how tightly do the points hug it?
r_{XY} = \frac{\operatorname{cov}(X, Y)}{\sigma_X \, \sigma_Y}
That definition has four blind spots, and each one maps to a way you lose information:
1. It is linear.A feature with a...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
