Skill Assessment and Prediction in Game Play and Cheating Detection

Skill assessment means rating the quality of a player's actions and choices, while prediction means projecting an accurate distribution of them based on what we know about the game situations and the player.  Online game providers wish both to rate players and predict how they will adapt to new game scenarios and role-playable characters.  Designers of multiple-choice and other tests need both to rate skill according to test scores and project a distribution of results reflecting the difficulty of the test.  The twin goals share many mathematical tools, but it is important to separate them.

Chess affords a data-rich environment to delineate these issues.  We will show how several features of models distinguish them and show the importance of this for detecting cheating with computers in human chess games.  In cheating detection it is desirable to base judgments on factors beyond "this player of rating X played unbelievably well."  The talk will also demonstrate some unexpected natural regularities of human play that emerge from the data, including a scaling phenomenon absent in computer play and a strong correlation to skill obtained solely on moves where players err, and some divergences among statistical fitting methods that can be employed.

Initial parts were joint with Guy Haworth of Reading whose Bayesian variant model will be discussed, and later parts are joint with my PhD student Tamal Biswas of Buffalo, for instance https://rjlipton.wordpress.com/2015/10/06/depth-of-satisficing/ and the relation to test-taking.