There's a growing belief among researchers that the in-context learning (ICL) of today's frontier models isn't the right approach for problems that require genuine discovery, like novel mathematics, adversarial scenarios, and knowledge too tacit to express in language. a16z's Malika Aubakirova argues that continual learning, in which models update their weights to internalize new knowledge, is needed and outlines the approaches startups are taking to make it possible at scale: https://www.a16z.news/p/why-we-need-continual-learning(a16z)