5 appearances on the backlist front page in the last 30 days.
Poor theory of mind is one of the main things keeping models from being good software engineers. They can resolve specific, reproducible bugs, but they struggle to anticipate what users want in the first place, which is much of what buildin
We are now seeking a puzzle maker to help us create puzzles that LLMs can't yet solve.
Claude Opus 4.8 scores 70.9% on GBA Eval, the top score to date. Given 24 hours, it writes an emulator that plays most games, with working audio on all of them. It beats the previous best (GPT-5.5 at 53.2%) in under an hour.
However, Opus 4.8's emulator is not perfect. On Varooom 3D, it diverges after around 2,000 frames. This is better than GPT-5.5 (whose emulator diverged after around 1,250 frames), but Opus 4.8 only scores 25% on this game.
Here's Claude Opus 4.8's emulator running Collie Defense, where it scores 99.8% on video and 91% on audio. On most games we tested, gameplay is near-perfect, with some audio imperfections.