Best-of lists are everywhere and almost none of them say what they are measuring. That omission is where most of the problem sits, because a list without stated criteria cannot be argued with, cannot be checked, and quietly presents one person's viewing history as a general assessment.

This is worth understanding as a reader and as anybody who might compile one, since the fixes are simple and they turn an unfalsifiable opinion into something genuinely useful.

Availability is the hidden filter

A compiler can only rank what they have encountered, and what they have encountered was determined by what was easy to find, in their language, on the platforms they use. Work that is out of print, untranslated, or hosted somewhere they do not visit is absent, and its absence looks like a judgement.

That filter is often stronger than any editorial one. A list of the best anything from a large field is usually a list of the most available things in that field, ranked by somebody's taste, and it is more honest to describe it that way.

Popularity feeds back

Lists draw on what is discussed, and what is discussed becomes more discussed by appearing on lists. That loop concentrates attention on a small number of works and makes it progressively harder for anything else to appear, regardless of merit.

The same dynamic drives the recommendation systems described in the piece on video platforms. Ranking and recommendation both reward what is already visible, which is a fact about the mechanism rather than about the works.

Criteria decide everything

Best according to what? Drawing quality, storytelling, historical importance, influence, how well it holds up now, or how much the compiler enjoyed it. These produce completely different lists, and a ranking that does not say which one it is using is not comparable to any other ranking.

Stating criteria is a single sentence and it changes the document entirely. It lets a reader decide whether the list is relevant to them, which is the only thing a list can usefully do.

What a useful list contains

The criteria, the pool considered, the exclusions and why, and enough about each entry that a reader can decide independently. That last point matters: a list that only ranks is asking for trust, and a list that describes lets somebody disagree usefully.

Describing rather than ranking is frequently the better format anyway. Ten works with a paragraph each on what makes them distinctive is more useful than the same ten in an order that nobody can verify.

Recency and the shape of memory

Lists skew heavily toward the recent and toward the compiler's formative period, which are two different distortions pulling in opposite directions. Both are ordinary human memory effects and both are invisible from inside.

Checking a draft list against a chronology usually reveals a decade with nothing in it, which is a prompt to ask whether that decade was genuinely weak or simply outside the compiler's reading.

Reading somebody else's list

The useful questions are: what pool did this come from, what are the criteria, what is conspicuously absent, and does the compiler have a stake. A list from somebody who publishes in the field is not disqualified by that, and it is worth knowing.

Treated as a description of one informed person's encounters, a list is a genuinely useful thing. Treated as a measurement, it is misleading, and most of the disappointment with rankings comes from the second reading.

Numbers imply a precision that is not there

Ordered rankings suggest that the difference between the third and fourth entries is meaningful, and it almost never is. Compilers know this and readers argue about it anyway, which is arguably the point of the format commercially and is not much use to anybody trying to find something to read.

Grouping instead of ranking solves it honestly. A handful of works described as essential, a larger set described as worth your time, and a note on what each offers gives a reader more usable information than a numbered order ever does, and it removes an argument that was never about the works.

Making a list that is worth reading

State the criteria, describe the pool, name the exclusions, write enough about each entry to be argued with, and check for decades you have skipped. That is a list somebody can use rather than one they can only agree or disagree with.

Our own editorial approach is described in the principles page, and corrections are welcome through the letters page.