AI researchers have used all sorts of video games, from Street Fighter to Pictionary, to test new models — often more for amusement than utility. But Anthropic said that Pokémon proved to be a useful benchmark for Claude 3.7 Sonnet, which can effectively “think” through the sorts of puzzles the game contains.

Claude 3.7 Sonnet is the latest AI model to play video games successfully. Last March, researchers used ChatGPT to play classic first-person shooter Doom, managing to get to the last room in the game once.

Anthropic benchmarked its latest AI model, Claude 3.7 Sonnet, on the classic game Pokémon Red on Nintendo's Game Boy. The model performed significantly better