Random Number Benchmark

Each test is started by hand: pick one, then press Run.
Connecting…

Run status

Every model is asked the same question 200 times at each effort level. A round is finished when every model has answered one more time, so "round 11" means everyone has answered at least 11 times. Round 200 is the end of the test.

How varied are the answers so far?

Top of the chart = every number picked equally often (truly random). Bottom = the same number every time. One point per finished round; the lines stop moving once there is enough data.

Round log

The same numbers as text, one line per round. "Spread" is the chart value. "Same fam." and "Cross fam." say how differently two models answer when they are from the same family or from different ones (0 = identical).

What each model picks

Each strip is one model at one reasoning effort. A stronger colour means that number was picked more often. A truly random picker looks like the flat top row.

Share of answers
1%5%25%100%
Spread a truly random picker would score with the same number of answers

Does the wording of the question matter?

Where each family's answers land

All models and efforts of a family pooled into one curve. Peaks on the same numbers mean the two families share favourites; the flat line is what truly random picking gives.

Does more reasoning effort make the picks more random?

Spread of each model's answers at low, medium and high effort (1 = every number equally likely). A line that climbs means thinking harder spreads the picks out.

How long does each answer take?

Typical (median) time from sending the question to receiving the full reply, in seconds, at each reasoning effort. The thin line reaches the time 90% of calls finished within.

Do models from the same family pick alike?

Live answers

The prompts every model receives, and the latest replies as they arrive from the APIs.

Summary card

One square picture of the test selected at the top, made for a screenshot. Each strip pools a model's answers at all three efforts.