Clustering Isn't Discovery - It's a Hypothesis You Still Have to Test
An analyst runs KMeans(n_clusters=4) on a dataset of customer purchase history; it gets back four groups, and he/she writes up the report as "from the data, it revealed four customer segments!" Management simply loves this - after long periods struggling to grasp their customer base, they now have data-backed insights on real names associated with each segment (Loyal Regulars, Bargain Hunters, At-Risk, New & Exploring), and pretty soon all the strategy meetings will silently assume those four groups are real and nobody even dares to question.
Unfortunately, those groups are notreal, and clustering does not "discover" hidden nuggets of insights in the data. Instead, clustering imposes a partitioning based on what the analyst chooses as a similarity rule. If the analyst had chosen another reasonable rule, clustering can also fit the same dataset in a different way. Let's be clear - the four groups are a hypothesis...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
