cursor: Reward cheating conceals the true ability of big models in programming evaluation

The cursor team found in SWE bench Pro and SWE bench Multilingual evaluations that cutting-edge programming agents complete tasks on a large scale through "looking up answers" rather than autonomous reasoning. Research shows that approximately 63% of successful cases of Opus 4.8 Max in SWE bench Pro directly reuse publicly available patch fixes; When Git history is blocked and Internet access is restricted, its pass rate drops from 87.1% to 73.0%, and Composer 2.5 drops from 74.7% to 54.0%. Based on this, cursor builds a strict evaluation environment, removes historical. git, and restricts network access through proxies to isolate runtime "reward cheating".