Clustering Has No Ground Truth: Impossibility, Validation, and What a Cluster Can Promise

In the largest published comparison of cluster validity indices, 30 indices across 6,480 synthetic configurations, only one, the silhouette, picked the partition closest to the known answer more than half the time (Arbelaitz et al., 2013, Pattern Recognition 46(1)). On 20 real UCI datasets, judged against class labels with the Adjusted Rand Index, the average index succeeded 10.0% of the time.

That is not a story about bad indices. It is the practical face of a theorem: clustering has no single correct answer to recover, only answers relative to a definition. It bites every time a team runs HDBSCAN on sentence embeddings or k-means on customer vectors and then asks a number whether the result is true.

Why this matters: Clustering is now a default step in embedding pipelines: topic discovery, deduplication, retrieval sharding, segmentation, eval slicing. Those systems ship decisions on clusters no index can certify as correct. Knowing which property an algorithm gives up, which bias each score carries, and where representation choices leak in separates a cluster you can act on from a pattern you manufactured.

TL;DR

• No clustering function satisfies scale-invariance, richness and consistency together (Kleinberg, 2002); single-linkage achieves any two. Choosing an algorithm is choosing what to give up.
• k-means and k-median are not even consistent. Once \(n\) is large relative to \(k\), tightening one cluster can change the optimal partition.
• Internal indices are definitions, not referees. Silhouette and Calinski-Harabasz reward compact convex groups, the same thing centroid methods optimise.
• External indices inherit the labels' opinion. In the worked example below, one partition scores ARI 1.0 against one reasonable labelling and 0.32 against another; its silhouette stays 0.659.
• Unadjusted scores inflate at small N/K. With 100 items in 10 clusters, random partitions share 0.46 nats of mutual information out of a possible 2.30 (Vinh et al., 2010).
• Stability is not correctness. Idealised k-means with the wrong k is perfectly stable whenever its objective has a unique optimum (Ben-David et al., 2006).
• Embedding pipelines add two definitions upstream. 2D UMAP and t-SNE embeddings of single-cell data showed mean 30-neighbour Jaccard distance above 0.7 from the original space (Chari & Pachter, 2023).

At a Glance

flowchart LR
    D["Raw objects"] --> R["Representation and metric"]
    R --> A["Algorithm as definition"]
    A --> P["Partition"]
    P --> I["Internal index"]
    P --> E["External labels"]
    P --> S["Stability check"]
    I --> U["Downstream use decides"]
    E --> U
    S --> U
    A -.->|"shared geometry"| I

    classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff
    classDef amber fill:#b45309,stroke:#fbbf24,stroke-width:1px,color:#fff
    classDef emerald fill:#047857,stroke:#34d399,stroke-width:1px,color:#fff

    class D blue
    class R,A purple
    class P teal
    class I,E,S amber
    class U emerald

Every box injects a definition of "cluster." The dashed edge is what makes validation circular: an internal index usually shares the algorithm's geometric assumptions.

Before Anyone Proved It Was Impossible

Lloyd's quantisation algorithm, still what most libraries run when you call k-means, was published in 1982 (Lloyd, IEEE Trans. Inf. Theory 28(2)). Judging a clustering came in pieces: Hubert and Arabie corrected the Rand index for chance in 1985 (Journal of Classification 2), and Rousseeuw introduced silhouettes in 1987 (J. Comput. Appl. Math. 20).

timeline
    title Defining and validating clusters
    1982 : Lloyd publishes the k-means quantiser
    1985 : Hubert and Arabie adjust the Rand index
    1987 : Rousseeuw introduces the silhouette
    1996 : DBSCAN defines density-connected clusters
    2001 : Ng Jordan and Weiss analyse spectral clustering
         : Gap statistic compares against a null
    2002 : Kleinberg proves the impossibility theorem
         : Ben-Hur Elisseeff and Guyon select k by stability
    2006 : Ben-David von Luxburg and Pal critique stability
    2012 : von Luxburg Williamson and Guyon ask science or art
    2013 : HDBSCAN published and 30 indices benchmarked
    2022 : BERTopic popularises UMAP plus HDBSCAN

The decades multiplied definitions rather than converging. DBSCAN made a cluster a maximal density-connected set with everything else as noise (Ester et al., KDD 1996). Normalised cuts (Shi & Malik, IEEE TPAMI 2000) and spectral clustering (Ng, Jordan & Weiss, NIPS 2001) made it a weak cut in a graph. Each produced qualitatively different partitions of the same data. Then Kleinberg, borrowing the axiomatic method behind Arrow's theorem, asked whether the disagreement was structural. It was.

[IMAGE: Four panels of one 2D dataset (two crescents, a dense blob, scattered noise) coloured by k-means k=3, a Gaussian mixture, DBSCAN and spectral clustering. k-means halves the crescents; DBSCAN recovers them and greys the noise. Caption: "Four algorithms, four definitions of 'cluster,' four defensible answers."]

How the Impossibility, the Indices and Stability Work

Kleinberg's three properties

A clustering function \(f\) takes \(n\) points with a symmetric distance function \(d\) (no vectors, no ambient space) and returns a partition \(\Gamma\). Kleinberg asks for three things. Scale-invariance: \(f(\alpha d) = f(d)\) for all \(\alpha > 0\), so changing units changes nothing. Richness: every partition of the points is the output for some \(d\). Consistency: call \(d'\) a \(\Gamma\)-transformation of \(d\) if it shrinks (or keeps) distances inside clusters of \(\Gamma\) and stretches (or keeps) distances between them; then

Making clusters tighter and further apart should not change the answer.

Theorem 2.1. For every \(n \ge 2\), no clustering function satisfies all three.

The proof explains why. Partition \(\Gamma_0\) refines \(\Gamma_1\) if every cluster of \(\Gamma_0\) lies inside one of \(\Gamma_1\); an antichain is a set of partitions where none refines another. Theorem 3.1: scale-invariance plus consistency force \(\text{Range}(f)\) to be an antichain. Consistency gives each achievable \(\Gamma\) a forcing pair \(a < b\): any \(d\) with within-cluster distances at most \(a\) and between-cluster distances at least \(b\) must output \(\Gamma\). If \(\Gamma_0\) refines \(\Gamma_1\), build \(d\) with tiny distances inside \(\Gamma_0\)'s clusters, medium ones between \(\Gamma_0\)-clusters sharing a \(\Gamma_1\)-cluster, and large ones elsewhere. It forces \(\Gamma_1\). Multiply by a large \(\alpha\): the medium distances now clear \(\Gamma_0\)'s threshold while the tiny ones stay under it, so the same data, rescaled, forces \(\Gamma_0\). Scale-invariance forbids that. Without a length scale, a function cannot tell two nearby clusters from one cluster with internal structure, because zooming turns one into the other. All partitions do not form an antichain, so richness fails.

[IMAGE: Two copies of the same eight points. Left, at original scale: two tight pairs sit inside each of two groups, and the forcing thresholds a and b are drawn as circles that make the output two clusters. Right, the same configuration multiplied by alpha: the gaps between pairs now exceed the finer partition's threshold, so the forcing argument demands four clusters. Caption: "Kleinberg's proof in one picture: without a length scale, zooming turns one cluster with structure into several clusters."]

Any two properties are achievable with single-linkage stopping rules (Theorem 2.2):

Fixing \(k\) abandons richness. A distance threshold such as DBSCAN's \(\varepsilon\) abandons scale-invariance. That table is the theory of algorithm choice in miniature.

Why k-means is not consistent

For \(k \ge 2\) and any continuous, non-decreasing, unbounded \(g\), the \((k,g)\)-centroid function picks centres \(T\) minimising

\(g(x)=x\) is k-median and \(g(x)=x^2\) is k-means. Kleinberg's Theorem 4.1: for \(n\) large relative to \(k\), none is consistent. Take a large loose group \(X\) and a small tight group \(Y\); with \(k=2\) the optimum puts a centre in each. Split \(X\) in half and shrink distances within each half. Only within-cluster distances changed, a legal transformation, yet the optimum now spends both centres on \(X\) and folds \(Y\) into a half. \(Y\) vanishes because another cluster got tighter. That is the objective, not Lloyd's heuristic.

The rebuttal: axioms on quality, not on functions

Ackerman and Ben-David called the impossibility "an artifact of the specific formalism" (NIPS 2008). Restated for clustering-quality measures, which score a (data, partition) pair, scale-invariance, consistency and richness, plus an isomorphism-invariance axiom, form a consistent set, satisfied by measures computable in polynomial time. Both sides are right. You can coherently score partitions; you cannot commit to one answer without breaking an intuition. And many different measures satisfy the axioms while disagreeing with each other.

Internal indices encode a definition

The silhouette of point \(i\) in cluster \(C\) is

where \(a(i)\) is the mean distance to the rest of \(C\) (cohesion), \(b(i)\) the mean distance to the nearest other cluster (separation), and \(s(i) = 0\) for singletons. Averaged distances reward groups whose members are all mutually close: compact and convex. A crescent scores badly however clean its surrounding gap. Calinski-Harabasz, \(\frac{\text{tr}(B_k)}{\text{tr}(W_k)} \cdot \frac{n-k}{k-1}\), has the k-means objective as its within-cluster term.

Von Luxburg, Williamson and Guyon accept such scores as objective functions but not as referees: "For every score preferring one clustering over the other one can invent another score which does the opposite" (JMLR W&CP 27, 2012). Density-based indices such as DBCV (Moulavi et al., SDM 2014) exist because classic ones penalise what DBSCAN is built to find.

External indices encode the labels' opinion

External indices compare a partition with reference labels via a contingency table \(n_{ij}\) with row sums \(a_i\) and column sums \(b_j\). The Rand index counts agreeing pairs but has a high chance baseline. The Adjusted Rand Index subtracts the expectation under a hypergeometric model with fixed marginals:

Observed agreement minus chance, over the best achievable minus chance: 0 is chance, 1 is identical, negative is worse than chance. Normalised mutual information has no such correction, and its average between random clusterings "grows as N/K becomes smaller"; at 100 items in 10 clusters chance MI is 0.4618 nats against a 2.30 maximum, and at 1,000 items its bounds fall below 0.08 (Vinh et al., JMLR 11). Lei et al. showed an index's bias can even reverse with the reference: the Rand index's preference for more clusters can become a preference for fewer when the ground-truth class sizes change (arXiv:1606.05596).

Chance correction fixes arithmetic, not meaning. Von Luxburg et al.'s example: images labelled by whether they contain a car, clustered cleanly into greyscale and colour. A reasonable clustering, a terrible score.

[IMAGE: Line chart of the average NMI and ARI between two independent random partitions as the number of clusters K rises from 2 to 50 at fixed N = 100. NMI climbs steadily as K grows while ARI stays near zero. Caption: "Without chance correction, more clusters look like more agreement (after Vinh, Epps and Bailey, 2010)."]

Stability, and why it does not certify k

Stability-based selection (Ben-Hur, Elisseeff & Guyon, PSB 2002; Lange et al., Neural Computation 2004) clusters many subsamples and prefers the \(k\) whose solutions agree most:

for independent size-\(n\) samples \(S_1, S_2\). Ben-David, von Luxburg and Pál analysed an idealised algorithm that always finds the global optimum. As von Luxburg's survey restates it (arXiv:1007.1075): if the population objective has a unique global minimum, instability tends to zero as \(n\) grows; if it has several, typically through symmetry, it stays positive. The true number of clusters appears nowhere in that condition.

Take three clusters, two close together. With \(k=2\) the optimum always merges the close pair, so the wrong \(k\) is perfectly stable. Only an equilateral layout would make \(k=2\) flip between merges. The survey's refined analysis, partly conjectural by its own account, rescues a narrower claim: instability tends to flag a \(k\) that is too large, since the extra boundary cuts through high density. And measured instability often reflects Lloyd's local optima across seeds, a property of the optimiser rather than the data.

[IMAGE: Two diagrams of a three-blob distribution resampled five times with k=2. Left, asymmetric layout: the same merge every time, labelled "stable, k wrong." Right, equilateral layout: three different merges, labelled "unstable, k wrong." Caption: "Stability tracks uniqueness of the optimum, not the number of real clusters."]

Seeing It in Motion

Each algorithm answers a different question

flowchart TB
    Q["What must a cluster promise"] --> K["Low variance around a centre"]
    Q --> G["High mixture likelihood"]
    Q --> DB["Density at one scale"]
    Q --> H["Density persisting across scales"]
    Q --> SP["Weak cuts in a graph"]
    K --> K2["k-means gives up richness"]
    G --> G2["GMM assumes ellipses"]
    DB --> D2["DBSCAN gives up scale-invariance"]
    H --> H2["HDBSCAN abstains as noise"]
    SP --> S2["Spectral inherits the kNN graph"]

    classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef amber fill:#b45309,stroke:#fbbf24,stroke-width:1px,color:#fff

    class Q blue
    class K,G,DB,H,SP purple
    class K2,G2,D2,H2,S2 amber

HDBSCAN builds the hierarchy of DBSCAN solutions over every \(\varepsilon\) and keeps clusters that persist while holding mass (Campello, Moulavi & Sander, PAKDD 2013; McInnes et al., JOSS 2017). It swaps a distance scale for a minimum cluster size, a scale counted in points, and sidesteps forced assignments by labelling some points noise.

The embedding pipeline and where definitions leak in

flowchart LR
    T["Documents"] --> EN["Sentence encoder"]
    EN --> UM["UMAP to 5 dims"]
    UM --> HD["HDBSCAN min size 10"]
    HD --> TOP["Topics with c-TF-IDF words"]
    HD --> OUT["Outlier topic -1"]
    EN -.->|"metric chosen"| L1["Leak"]
    UM -.->|"density reshaped"| L2["Leak"]
    HD -.->|"noise excluded"| L3["Leak"]

    classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff
    classDef rose fill:#be123c,stroke:#fb7185,stroke-width:1px,color:#fff
    classDef slate fill:#334155,stroke:#64748b,stroke-width:1px,color:#e2e8f0

    class T blue
    class EN,UM,HD purple
    class TOP teal
    class OUT slate
    class L1,L2,L3 rose

This is BERTopic's default wiring as of September 2026: UMAP with n_neighbors=15, n_components=5, min_dist=0.0, cosine metric, then HDBSCAN with min_cluster_size=10 and excess-of-mass selection (Grootendorst, arXiv:2203.05794; source).

A stability protocol, and the step that fools it

sequenceDiagram
    participant A as Analyst
    participant R as Resampler
    participant C as Clusterer
    participant M as Comparator
    A->>R: 50 subsample pairs at 80 percent
    R->>C: Subsamples
    C->>M: Partitions for k from 2 to 8
    M->>M: ARI on shared points
    M->>A: Mean agreement per k
    Note over M,A: High agreement means a unique optimum
    A->>C: Rerun chosen k with 20 seeds
    C->>A: Seed-to-seed ARI
    Note over A,C: Low seed agreement is optimiser noise

The protocol separates sampling variation, which says something about the data, from optimisation variation, which says something about the algorithm. Conflating them is the usual mistake.

By the Numbers

Sources: Kleinberg, 2002; Arbelaitz et al., 2013 (Davies-Bouldin* is their modified variant); Vinh et al., 2010; MNIST runs from the umap-learn clustering guide, single runs by the library authors rather than an independent benchmark; Chari & Pachter, 2023.

HDBSCAN's 0.054 and 0.998 are the same clustering; only the treatment of noise differs. And the 0.924 is agreement with digit labels that happen to align with the density UMAP accentuates. The umap-learn guide itself calls UMAP-before-clustering "somewhat controversial," noting UMAP "does not completely preserve density" and "can also create false tears in clusters."

[IMAGE: Grouped bars for the three MNIST pipelines: ARI on all points and fraction clustered. The PCA-50 HDBSCAN pipeline shows a tiny ARI bar and a 17% coverage bar annotated "0.998 on covered points." Caption: "One partition scores 0.05 or 0.998 depending on how noise is scored."]

A Concrete Example

Six support tickets, embedded and projected to one dimension so every distance is an absolute difference:

The product team labels by area: \(L_A\) = [billing ×3, auth ×3]. The support team labels by intent, and ticket 11 is an oddly worded refund: \(L_B\) = [refund, refund, refund, login, login, refund]. Two candidates: \(P_2 = \{0,1,2\}\mid\{6,7,11\}\) (k-means with \(k=2\), sum of squares 16.0) and \(P_3 = \{0,1,2\}\mid\{6,7\}\mid\{11\}\) (k-means with \(k=3\), sum of squares 2.5).

Step 1: silhouette. \(a\) is mean distance within the cluster, \(b\) mean distance to the nearest other cluster.

For point 6 in \(P_2\): \(a = (1+5)/2 = 3\), \(b = (6+5+4)/3 = 5\), \(s = 2/5\). The silhouette narrowly prefers \(P_2\), whoever labelled the tickets. Note the singleton rule costs \(P_3\) about 0.13 on its own; give point 11 the 0.8 its neighbours get and \(P_3\) wins.

Step 2: ARI of \(P_2\) against \(L_B\). Contingency rows refund = [3, 1], login = [0, 2].

• Within-cell pairs: \(\binom{3}{2} + \binom{2}{2} = 4\).
• Row pairs \(\binom{4}{2} + \binom{2}{2} = 7\); column pairs \(3 + 3 = 6\); total \(\binom{6}{2} = 15\).
• Chance: \(7 \times 6 / 15 = 2.8\). Maximum: \((7+6)/2 = 6.5\).
• \(\text{ARI} = (4 - 2.8)/(6.5 - 2.8) = 0.324\).

Step 3: ARI of \(P_3\) against \(L_B\). Rows refund = [3, 0, 1], login = [0, 2, 0]. Cells 4, rows 7, columns \(3 + 1 + 0 = 4\). Chance \(28/15 = 1.867\), maximum 5.5, so \(\text{ARI} = 2.133/3.633 = 0.587\).

Step 4: a bad partition. \(P_{bad} = \{0,1,2,6,7\}\mid\{11\}\) against \(L_B\): cells 4, columns \(\binom{5}{2} = 10\), chance \(70/15 = 4.667\), maximum 8.5, ARI \(= -0.174\). Its unadjusted Rand index is 0.40, which reads like partial agreement.

No point moved between columns. The product team's labels crown \(P_2\), the support team's crown \(P_3\), the silhouette leans \(P_2\), and the sum-of-squares elbow (16.0, 2.5, 1.0 for \(k\) = 2, 3, 4) leans \(P_3\). The honest report is "\(P_2\) matches product areas, \(P_3\) matches intent, and geometry barely separates them," not "silhouette 0.66."

[IMAGE: Number line 0 to 11 with six points drawn twice: top coloured by product area with the P2 split dashed, bottom coloured by intent with the P3 splits dashed, each with a small silhouette and ARI readout. Caption: "Same points, same silhouette, different winner."]

Where It Breaks

Validating with the algorithm's own objective

Choosing \(k\) for k-means by maximising Calinski-Harabasz, then citing the score as evidence of structure, measures how well k-means optimised k-means. Both silhouette and CH also return a best \(k\) on data with no clusters, because they rank partitions without testing against a null. The gap statistic adds that null (Tibshirani, Walther & Hastie, JRSS B 2001), moving the judgement into the choice of reference distribution.

Treating benchmark labels as truth

Ullmann et al. took a recently published algorithm and showed how dataset choice, parameter tuning and competitor selection make it look better than it is (Advances in Data Analysis and Classification, 2022). Combined with reference-dependent index bias, an NMI leaderboard says as much about label sets as about algorithms.

Scoring noise

Whether −1 is one cluster, dropped, or reassigned changes ARI by an order of magnitude, as MNIST shows. Dropping noise rewards abstention; keeping it punishes honesty. BERTopic users meet this when a large share of a noisy corpus lands in topic −1 and an outlier-reduction pass reassigns it, quietly changing what each topic contains.

UMAP rewrites the density HDBSCAN reads

HDBSCAN defines a cluster by persistent density; UMAP preserves fuzzy neighbourhoods, not density. With min_dist=0.0 it packs neighbours as tightly as possible, manufacturing the contrasts HDBSCAN looks for. Chari and Pachter's above-0.7 neighbourhood Jaccard distances were for 2D; BERTopic reduces to five dimensions, which should distort less, but I have found no comparable measurement for text embeddings. The symptoms are topics that change with UMAP's seed and small clusters that exist only because n_neighbors=15 emphasised local structure. The umap-learn guide raises n_neighbors to 30 for clustering because small values yield "fine grained cluster structure that may be more a result of patterns of noise in the data than actual clusters."

The encoder picks the metric first

Encoders trained for retrieval encode retrieval similarity. Two reports of one bug in different languages may sit far apart; unrelated tickets sharing a template may sit close. In Kleinberg's terms \(d\) is the input, and a model trained for something else chose it.

[IMAGE: Three 2D UMAP views of the same 5,000 sentence embeddings with n_neighbors 5, 15 and 50, coloured by HDBSCAN cluster with noise grey, cluster count and noise fraction annotated from a real run. Caption: "The number of 'topics' is partly a UMAP hyperparameter."]

Alternative Designs

The last row is the stance von Luxburg, Williamson and Guyon argue for: judge clusterings by "the usefulness for the particular task under consideration," and build a taxonomy of clustering problems instead of a universal benchmark. They quote Mercier (1912): a classification that serves its purpose "is a good classification, however 'artificial' it may be."

How It Is Used in Practice

Topic discovery. BERTopic's separation of clustering from labelling makes each stage swappable. Its paper admits it assumes one topic per document. Durable practice pins UMAP's seed, sweeps n_neighbors and min_cluster_size, and treats topics that appear under only one setting as hypotheses.

Segmentation. When clusters drive an action such as a pricing tier or onboarding flow, the metric is downstream: holdout lift or an A/B test. Silhouette only flags degenerate partitions.

Data curation. Clustering embeddings for deduplication or balancing is quantisation, not discovery; k-means fits because nobody claims centroids are natural kinds.

Exploratory science. Following Chari and Pachter's critique, cluster in the higher-dimensional space, treat 2D plots as illustration, and confirm clusters with a measurement outside the embedding.

A five-line report covers it: representation and metric; algorithm and the property it gives up; index and the definition it shares; stability across resamples and across seeds, separately; the downstream check, or a statement that none exists.

Insights Worth Remembering

1. An algorithm is a definition of "cluster," not a way of finding one. Kleinberg's theorem guarantees each resolves the same conflict differently.

2. Fixing k and fixing epsilon are the same kind of decision. One sacrifices richness, the other scale-invariance.

3. An index sharing the algorithm's geometry cannot validate it. Calinski-Harabasz contains the k-means objective.

4. ARI against labels measures agreement with a taxonomy. Report whose labels, not just the number.

5. Correct for chance when clusters are small. At N/K = 10, random partitions share a fifth of the maximum mutual information.

6. Stable is not true. With a unique optimum, a wrong k is perfectly stable; stability only rules out sample and seed artefacts.

7. Noise handling is a scoring decision disguised as a result. One MNIST clustering scores 0.054 or 0.998.

8. In embedding pipelines most of the clustering happens before the clustering algorithm. The encoder picks the metric and UMAP reshapes density.

Open Questions

Do axiomatic properties predict task performance? Ackerman and Ben-David showed quality-measure axioms are satisfiable. Whether any property set predicts which algorithm suits a task has been argued, not demonstrated.

How much does 5- to 10-dimensional UMAP distort text-embedding neighbourhoods? Severe loss is measured for 2D single-cell data. The moderate reductions in BERTopic-style pipelines appear unmeasured; a careful benchmark could settle it.

Can stability detect a k that is too small? Theory mainly supports flagging k too large, partly via conjecture. Finite-sample designs that catch under-clustering remain open.

What would a neutral clustering benchmark look like? Task-based benchmarks with downstream metrics have been called for since 2012; none is widely adopted. Whether one could resist the over-optimism Ullmann et al. document is unknown.

Sources and Further Reading

1. Kleinberg, J. (2002). "An Impossibility Theorem for Clustering." NIPS 15. PDF
2. Ackerman, M., & Ben-David, S. (2008). "Measures of Clustering Quality: A Working Set of Axioms for Clustering." NIPS 21. Proceedings
3. von Luxburg, U., Williamson, R. C., & Guyon, I. (2012). "Clustering: Science or Art?" JMLR W&CP 27, 65-79. PMLR
4. Ben-David, S., von Luxburg, U., & Pál, D. (2006). "A Sober Look at Clustering Stability." COLT 2006, 5-19. doi:10.1007/11776420_4
5. von Luxburg, U. (2010). "Clustering Stability: An Overview." Foundations and Trends in ML 2(3). arXiv:1007.1075
6. Ben-Hur, A., Elisseeff, A., & Guyon, I. (2002). "A Stability Based Method for Discovering Structure in Clustered Data." PSB, 6-17. PDF
7. Arbelaitz, O., et al. (2013). "An Extensive Comparative Study of Cluster Validity Indices." Pattern Recognition 46(1), 243-256. doi
8. Vinh, N. X., Epps, J., & Bailey, J. (2010). "Information Theoretic Measures for Clusterings Comparison." JMLR 11, 2837-2854. PDF
9. Lei, Y., et al. (2016). "Ground Truth Bias in External Cluster Validity Indices." arXiv:1606.05596
10. Rousseeuw, P. J. (1987). "Silhouettes." J. Comput. Appl. Math. 20, 53-65. doi
11. Hubert, L., & Arabie, P. (1985). "Comparing Partitions." Journal of Classification 2, 193-218. doi
12. Ester, M., Kriegel, H.-P., Sander, J., & Xu, X. (1996). "A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise." KDD-96. PDF
13. Campello, R. J. G. B., Moulavi, D., & Sander, J. (2013). "Density-Based Clustering Based on Hierarchical Density Estimates." PAKDD 2013. doi
14. Ng, A. Y., Jordan, M. I., & Weiss, Y. (2001). "On Spectral Clustering: Analysis and an Algorithm." NIPS 14. Proceedings
15. Grootendorst, M. (2022). "BERTopic: Neural Topic Modeling with a Class-Based TF-IDF Procedure." arXiv:2203.05794
16. McInnes, L., Healy, J., & Melville, J. (2018). "UMAP." arXiv:1802.03426; umap-learn clustering guide
17. Chari, T., & Pachter, L. (2023). "The Specious Art of Single-Cell Genomics." PLOS Computational Biology 19(8). doi
18. Ullmann, T., et al. (2022). "Over-Optimistic Evaluation and Reporting of Novel Cluster Algorithms." Advances in Data Analysis and Classification 17. doi

Free to read, no ads, no sign-up. If it was useful you can buy me a coffee.