DeepMind researcher resigns warning: evaluation system is becoming the biggest bottleneck for AI capability leap According to Beating monitoring, Lun Wang, a researcher at Google DeepMind, announced his resignation and wrote a long article reflecting on the current AI evaluation mechanism. He bluntly stated that the current evaluation system is all about "carving boats and seeking swords", and can only passively test the existing abilities of the model, without guessing what new skills the next generation model will suddenly evolve into. Compared to data, computing power, and architecture, outdated evaluation systems are currently the biggest bottleneck hindering the industry's progress. The existing mainstream ranking tests are only applicable to the current generation of models. Once the model learns new operations that humans have never seen before, these tests will collectively become useless. One of the most dangerous risks is that if the model learns to deliberately "hide" key information in order to achieve its goals, existing security tools cannot catch it at all, because every word the model says is still true in fact. Due to the inability to find a "core signal" that can warn AI suddenly becoming smarter in advance, the development of large models in the industry is completely "blind flying". If we don't solve the fundamental problem of what to test, blindly pushing model training, security protection, and computing power expansion with old indicators, we will all end up making outrageous mistakes. Faced with cutting-edge models that are increasingly capable of working independently, evaluation systems must also come to life. In addition to monitoring the abnormal fluctuations in scores, the development team must let AI generate exam questions on its own and test the bottom line of other AI. The future evaluation system must be a living organism that can evolve with large models, rather than a rigid checklist carved according to last year's standards. [Original link]