Vals AI CEO Rayan Krishnan on the unlimited tokens experiment that revealed an anxious obligation for engineers to use models all the time, everywhere: "I wanted to do a tokenmaxxing experiment. I was able to get unlimited access for our team for a month for some of the coding tools. We had a lot of engineers spending between one to two billion tokens a day." "I went back and did some math, and it looked like in that month, we spent roughly $1.5 million worth of tokens... It was actually 10x more we were spending in tokens than employee salary for that month." "We cannot continue with this mode of operation for the next month. How do we intelligently figure out what are the right tools we should use, for what teams, and what projects?" "We found some pretty surprising insights. The Cognition Devin tool is actually very token-efficient, and so that's a place we've chosen to adopt more." "There are a lot of places where you can get better pricing models out of subscriptions as opposed to token-based pricing." "It's actually informed our strategy for how we can effectively tokenmaxx without spending $1.5 million per month." @RayanKrishnan @JenniferHli(a16z)