We're excited to invest in Preference Model. Building RL environments that actually work is harder than it looks. Models are relentless at reward hacking, finding shortcuts, and exploiting vulnerabilities. @preferencemodel has focused on the domain that matters most to labs right now: AI research and ML engineering itself. Over the past year, the team has built RL environments for leading labs. Their focus has been on building the infrastructure to make harder, more resistant environments as models improve: tooling that finds where models are weak, generates new tasks to target those gaps, and tests environments against agents actively trying to break it. This week they're open-sourcing Karotte, the framework they've used in production — hardened through more than a million evaluation runs and red-teaming. We're thrilled to partner with @chem_safety and @Ning_Catsnail and the Preference Model team as they build the training grounds for capable and aligned models. By @JenniferHli