Every BIM lead who has ever run k-means on a pile of occupant surveys knows the bad part: you have to tell the algorithm how many groups exist before it has seen the data. Pick three, get three. Pick seven, get seven. The number was never in the data; it was in your guess.

Two fresh arXiv preprints from June 2026 quietly attack that exact sin, from opposite ends of the world. The first (arXiv:2606.12305, Statistics > Methodology) builds a Bayesian nonparametric Mallows model for clustering ranking and preference data. The second (arXiv:2605.31511, Statistics > Applications) uses the same statistical engine to sort patients into disease subtypes — aggressive versus non-aggressive cancer, for example. Neither paper cites the other. The “one method, two worlds” framing is mine, not theirs. But the shared spine is unmistakable.

←TODAY: In 2026, two independent teams used the Dirichlet process to let the data decide how many clusters exist — in movie ratings and in cancer cohorts. →3012: The buildings and bodies we model in Zurich-3012 are never one type; they are a count we have to discover, not declare. Fulcrum: The same humility — stop guessing the number of groups — is what makes both a recommender and a diagnosis safer.

The shared trick: a process that grows its own clusters

Both papers lean on the Dirichlet process mixture model (DPMM). Instead of fixing the cluster count K up front, the DPMM treats K as something to infer alongside everything else — an “infinite” mixture where only a finite number of clusters end up non-empty. The preference paper plugs this into the Mallows model (Colin Mallows, 1957: a distribution over rankings centred on a consensus order with a dispersion parameter). It ships inside the existing open-source R package BayesMallows — originally out of the Norwegian Computing Centre — and it handles the messy reality of incomplete rankings and pairwise comparisons. On movie ratings, it recovers preferences well enough to recommend films from discarded ratings. That is the honest engineering value: it works on the broken data you actually have, not the clean matrix you wish you had.

Where the two diverge: MCMC vs variational

Here is the genuinely useful disagreement. The preference paper stays with the gold-standard, slow MCMC Metropolis-Hastings sampler. The medical paper throws it out for coordinate ascent variational inference (CAVI), reporting comparable accuracy at substantially lower compute cost — and frames that speed explicitly as a patient-safety feature: a clinician under time pressure cannot wait for a chain to converge. That is the same accuracy-versus-latency trade every BIM data engineer meets when a Bayesian routine is too slow to sit inside a CAD plugin. When the sampler won’t fit in production, variational inference is often the recovery move, not a downgrade.

The clustering definition itself carries weight. A PLOS Medicine cohort study on UK Biobank (Silva and colleagues, Université Paris Cité) found that count-based versus clustering-based definitions of multimorbidity produce different mortality and prognosis conclusions. How you group people changes what you conclude about them. MIT News (11 June 2026) adds a sharp adjacent result: you cannot recover preference correlations from pairwise comparisons alone — you need triplets, the “power of three.” A useful caution for anyone clustering design-review votes.

Atelier: At PAZ we run preference aggregation on every multi-criteria design review — jurors rank options, and we pretend the consensus is clean. A nonparametric Mallows pass on that voting data would tell us whether the panel is one mind or three rival camps, without us deciding the answer in advance. That is the difference between a tabulated vote and an honest one.

Hack: This Hack teaches you to let cluster count emerge from ranking data instead of guessing it (DOMAIN: AI/ML). Install the open R package and cluster a tiny preference set — watch how many groups survive. Run it once before you ever hard-code a K.

install.packages("BayesMallows")
library(BayesMallows)
# rankings: rows = people, cols = items (1 = top choice)
fit <- compute_mallows(setup_rank_data(potato_visual))
assess_convergence(fit)        # then inspect the cluster posterior

One sober caveat: neither preprint, in the material I read, confirms author affiliation or quantitative speedup factors — verify the PDFs before you quote a number. And note the regulatory edge: under the EU AI Act, the medical variant is high-risk clinical decision support, so any “faster” subtyping tool inherits conformity and transparency duties before it touches a patient.

So here is the move: next time you reach for k-means on occupant, voting, or typology data, stop and ask whether the cluster count is data or dogma. Run a DPMM pass first.

Source: arXiv