Yardstick
What we measure, and what we refuse to measure
This page exists so that you can decide how much weight to give anything we publish. If our method does not suit the way you choose games, at least you will know that before you read a verdict rather than after.
The route
Two passes, one gap, one verdict
The sequence below is followed for every entry without exception. Where we have had to break it — a game that shut down its servers before the second pass, for instance — we say so in the entry and mark it as incomplete rather than publishing a verdict we have not earned.
- First pass, close to release. Played to the credits or to a defensible stopping point, on default settings, with a running note file kept in the moment rather than reconstructed afterwards. Nothing is published at this stage, and we do not post first impressions.
- The gap of at least three months. The game is uninstalled. We avoid reading other coverage of it, partly to keep our second pass honest and partly because it is impossible to un-read a strong opinion.
- Second pass, after the patches have landed. Same person wherever possible, same machine, entirely fresh save. No use of the earlier notes until the second pass is finished and timed.
- The comparison. Both note files are read aloud to the whole desk. The differences between them are the actual subject of the write-up.
- The verdict. One of three, agreed by majority. Where the desk splits badly we print the disagreement in the entry instead of hiding it behind a compromise.
The three verdicts
No numbers, no stars, no percentages
We dropped scores in our first month. The arguments were never about the game, they were about whether a particular game was a seven or a seven and a half, which is not a useful conversation to have and certainly not a useful thing to publish. What replaced them is deliberately blunt.
| Verdict | What it means | What it does not mean |
|---|---|---|
| Runs again | The second pass was as good as the first or better. The game rewards knowing it. | That it is flawless, or that everyone at the desk enjoyed it. |
| Runs once | A worthwhile first playthrough with little left underneath it. Play it, then move on. | That it is a poor game. Several of our favourite entries this year sit here. |
| Left unfinished | The second pass was abandoned. We say where and why. | That it is broken or badly made. Often the opposite. |
Verdicts are never revised quietly. If a third pass changes our mind, the original verdict stays visible with a dated note beneath it.
Hardware
The three machines in the room
We are a small desk with a small amount of equipment, and we would rather describe it plainly than imply a laboratory we do not have. Every entry names which machine was used for each pass. Where a game behaved differently across two of them, the entry says so.
| Machine | Standing in for | Notes |
|---|---|---|
| The upstairs desktop | A well-specified but not extravagant home PC, three years old | Our default for first passes. Kept deliberately un-upgraded so that it ages the way a reader’s machine ages. |
| The living-room console | A current-generation console on a mid-sized television | Used for anything with a console release. Also where we check subtitle legibility at viewing distance. |
| The handheld | A portable PC-class handheld | Increasingly where second passes happen, because that is where most of us actually play now. |
We do not publish frame-rate charts. Not because they are worthless, but because doing them properly requires equipment and repetition we cannot honestly claim, and half-done performance data is worse than none. Where a game stuttered badly enough to affect play, we describe what happened and on which machine.
Access
Six things we check every time
Each pass includes the same accessibility pass, recorded in the note file whether or not it makes the final write-up. None of the five of us is a specialist in this area, so we describe what we found rather than grading it, and we are always glad to be corrected by readers who rely on these options.
- Subtitles. Can they be resized, given a background, and told to name the speaker?
- Remapping. Is every input rebindable, including the ones buried in menus and vehicle sections?
- Timed input. Are there quick presses or repeated taps, and can they be turned off or slowed?
- Colour reliance. Does anything important depend on telling two colours apart with no second cue?
- Audio-only cues. Is there ever information carried by sound alone, with nothing on screen to match it?
- Motion and shake. Can camera sway, blur and screen shake be reduced, and does the setting actually work?
Questions we get asked
The awkward ones, answered
Three months is long enough for the memory of a game to become general rather than specific, and long enough for most of the significant patches to arrive. Six months would be better for the first purpose and worse for the second, since by then the game being discussed has often changed enough that we are effectively reviewing a different edition. One month is not long enough to forget anything.
It is a compromise, chosen on purpose. Where a game has changed substantially inside the gap, we note the patch dates in the entry so you can weigh it yourself.
Yes, and we say so in every entry where one was used. We accept the game and nothing else: no travel, no hardware, no hospitality, no early access in exchange for a publication date. Because we never publish inside three months of release, embargoes are not a lever anyone can pull on us, which is one of the quieter benefits of working slowly.
We have declined copies twice, both times because acceptance was conditional on something we were unwilling to agree to in advance.
The verdict is decided by majority, and the minority view goes into the entry in its own words. We had a three-two split on the corridor shooter in the current log and printed both positions, because a reader deciding how to spend a weekend is better served by knowing that reasonable people found it a chore on a second run than by a single smoothed-over sentence.
Because two full playthroughs of a thirty-hour game is sixty hours of work before a word is written, and none of us does this as a full-time job. Fourteen to eighteen entries a year is what five people with other commitments can produce honestly. We would rather publish a small, reliable log than a large one padded with things nobody actually finished.
Several. We do not write about competitive shooters, sports games, or anything whose substance is a rolling seasonal calendar, because our method cannot describe them fairly — a game that is deliberately different every eight weeks makes a three-month gap meaningless. We also avoid horror, for the ordinary reason that nobody here enjoys it enough to judge it well.
Naming the gaps seems more useful than quietly having them. The voices page lists what each of us does cover.
Hold us to it
If we have broken our own method, say so
The method above is the whole basis of the site, so a departure from it is a more serious error than a factual slip. If an entry does not show two dated passes, or a verdict does not match what the text describes, that is worth an email.