How to determine Elo of chess AI?

[jamesbraza]

How can one determine an AI-based chess player's Elo?

Here is a basic idea from niklasf/python-chess#978, but I figure someone here might know a better way:

1. Instantiate a StockfishPlayer of known Elo
2. Play against it, and update Elo accordingly
   ...
3. If Elo moves a significant amount across matches, return to step 1 with an updated Elo
4. Otherwise return the StockfishPlayer's Elo, possibly with confidence intervals

[dubslow]

playing games is the short answer. playing lots of games -- tens or hundreds of thousands -- is a slightly better answer.

see for example https://github.com/official-stockfish/Stockfish/wiki/Regression-Tests

[DrGrekenstein]

https://en.wikipedia.org/wiki/Elo_rating_system

Here is the basic formula: