The AIs’ Elo rating

The ranking

The higher the rating, the stronger the AI against the others. The average of the eight AIs is 1500.

1. 1 V2 Oware Master 2,221 ± 88
2. 2 Ancestral Spirit 1,865 ± 58
3. 3 Immutable Sage ZAK 1,676 ± 50
4. 4 Immutable Sage SIR 1,650 ± 48
5. 5 Furrow Wanderer 1,422 ± 42
6. 6 Probabilistic Strategist 1,334 ± 58
7. 7 Seed Keeper 1,119 ± 60
8. 8 Novice Sower 714 ± 95

The pale bar shows each rating’s margin, the line its value. When two AIs’ bars overlap, the tournament cannot say which one is stronger.

What a rating means

To help you find your bearings, here are levels of play in oware, with where each of our AIs stands.

Elo rating	Level	Description	Our AIs
< 1,200	Beginner	Knows the rules and sows without yet calculating captures.
1,200 – 1,600	Intermediate player	Sees simple captures and avoids leaving pits with one or two seeds.
1,600 – 1,800	Good player	Sets up captures one move ahead and keeps an eye on starvation.
1,800 – 2,000	Strong player	Calculates several moves ahead, keeps granaries and plays endgames well.
2,000 – 2,200	Expert	Rarely caught out: sets traps and counts endgames precisely.
2,200 – 2,400	Master	Almost faultless, takes advantage of every mistake by the opponent.
2,400 +	Grandmaster	The highest level of play, very close to perfect play.

These levels are a guide: there is no official scale of levels in oware. Our ratings are calculated between the PlayAwale AIs, with the average set at 1500.

What a gap means

Only the gap between two ratings matters: it tells what share of the points each AI should score against the other.

For example, V2 Oware Master is rated 356 points above Ancestral Spirit: against it, V2 Oware Master should score about 89% of the points.

How the rating is calculated

1. 1

   The eight AIs play one another on the publisher’s computer, never on the server, each exactly as it plays on the site, with the same thinking time.

2. 2

   Games start from real early-game positions taken from games played here; each is played twice, with sides swapped.

3. 3

   AIs of similar strength meet more often: that is where the gap is hardest to measure.

4. 4

   At the end, the ratings that best explain all the results are found in one go. A win is worth one point, a draw half a point.

This is Arpad Elo’s method, used in chess. Ratings are not adjusted game after game, so that the result does not depend on the order of the games. Only the result of the tournament is published.

All the pairings

The share of the points scored by the AI in each row against the AI in each column.

V2 Oware Master		94%	100%	100%	100%	99%	100%	100%
Ancestral Spirit	6%		79%	66%	96%	97%	100%	100%
Immutable Sage ZAK	0%	21%		61%	85%	85%	94%	100%
Immutable Sage SIR	0%	34%	39%		86%	90%	88%	100%
Furrow Wanderer	0%	4%	15%	14%		69%	96%	100%
Probabilistic Strategist	1%	3%	15%	10%	31%		78%	100%
Seed Keeper	0%	0%	6%	13%	4%	22%		96%
Novice Sower	0%	0%	0%	0%	0%	0%	4%

Green: the row’s AI dominates; red: it is dominated; amber: they are evenly matched.

The ratings through the tournament

At first, with few games, the ratings move a lot; they settle as the results pile up.

Against the AIs, against the players

The rating measures each AI against the other AIs. The share of games players win against it measures something else: how it stands up to humans.

AI	Elo ranking	Players’ wins
V2 Oware Master	1 2,221	0.0%
Ancestral Spirit	2 1,865	4.0%
Immutable Sage ZAK	3 1,676	9.0%
Immutable Sage SIR	4 1,650	11.9%
Furrow Wanderer	5 1,422	24.4%
Probabilistic Strategist	6 1,334	25.0%
Seed Keeper	7 1,119	34.4%
Novice Sower	8 714	56.8%

An AI can trouble players more than it troubles the other AIs, or the reverse: the two measures complement each other. The players’ share is only shown when it rests on enough games.

And the players?

Every player with an account also gets an Elo rating, on the same scale as the AIs. It is calculated as in chess.

1. 1

   The games that count: against the solo-mode AIs, whose rating is the one from the tournament, and online against a player who also has an account and already has a rating. A two-player game on the same screen does not count.

2. 2

   The first five games are an assessment. The first rating is the performance over those games: the opponents’ average rating, adjusted for the score, with two imaginary draws against an opponent rated 1500 so that it doesn’t run away on so few results.

3. 3

   After that, the rating moves after every game: beating a stronger opponent earns a lot, losing to a weaker one costs a lot. It moves quickly during the first thirty games, more slowly afterwards, and even less above 2400.

4. 4

   A resignation counts as a loss. A game left before its second move is cancelled: it does not count.

5. 5

   Online, a resignation before each player’s fifth move does not count, and at most three games a day count against the same opponent.

6. 6

   You can see your rating, with its curve, in “My badges”. Your friends see it on your profile, like your badges.

The margin, and what the rating does not tell you

Each rating comes with a “±” margin: the calculation is repeated on a large number of imaginary tournaments, drawn at random from the observed results, and the margin covers 95% of the ratings obtained. So that an AI that wins everything does not get an infinite rating, each pairing also counts one imaginary draw.

The rating compares the AIs with one another; no player data goes into this calculation. It is redone whenever an AI changes.

See how the AIs play Legal notice