If you ask five physicians a question, you might get eight different answers. Karan Singhal of @OpenAI on how OpenAI decides what its models should say about health: "One of the first things that we did when we first started focusing on health at OpenAI was really build this kind of global physician network of hundreds of experts that we work with." "Across the globe, across dozens of specialties and languages and countries of practice, to think about the right ways to aggregate the wisdom of experts so that we weren't relying, especially in areas where there could be disagreement, on one or two or three physicians to kind of adjudicate." "Let's say the aggregated wisdom of the physicians is pointing in one direction, and then maybe the user actually wants you to say something comforting even when the situation's not comforting. There it's basically a prioritization of safety." "It's very important to us that we take our obligation very seriously when there's hundreds of millions of health conversations coming in." "We have evaluations that track these kind of difficult situations... Over time, we've not only improved how we've done that evaluation working with physicians, but also seen pretty dramatic model improvements across the board." @thekaransinghal @JayRughani