Don't believe it without a doubt | Tell me about my experience of being scammed by ChatGPT once: the scariest thing about AI is not that it can't do it, but that it's too good at self consistency! Today I encountered a wonderful thing to share with everyone, I was taught a damn lesson by ChatGPT: the biggest risk of AI is not that it won't do it, but that it's too good at "self consistency". It excels at deducing the premises you provide into a conclusion that sounds' reasonable '. Even if the conclusion itself is wrong. It's horrible! one ️⃣ Car accident site: The situation is as follows: I usually eat fish oil, probiotics, and liver protection tablets such as magnesium glycinate, so I often discuss with Ai about the body feedback after eating and the adjustment of the best consumption time; The problem lies with probiotics! Ai knows that I don't eat breakfast, but I will eat some nuts, eggs or milk at 9:30 am. He suggested that I eat probiotics on an empty stomach every morning, because when I wake up on an empty stomach, stomach acid is relatively low, and probiotics are more likely to "survive through the stomach" and reach the intestines with a higher survival rate; After meals, the secretion of stomach acid increases and probiotics are more easily killed by stomach acid; I didn't think much about it at the time - because it sounded too much like that. two ️⃣ Magic: The same model, two sets of answers And when I was communicating with a friend today, he said that we definitely can't eat on an empty stomach. Fasting stomach acid is the most abundant in the morning, which is not conducive to the absorption of probiotics. And he told me that this is also the answer his ChatGPT told him. And then I became foolish! So in front of him, I asked him several questions about GPT, and the answers I got from my GPT were completely opposite. For example, my GPT told me to eat on an empty stomach in the morning, and he told me to eat with meals or 20 minutes after meals; My GPT told me that stomach acid is the lowest on an empty stomach, and he told me that stomach acid is only lower after eating. And they all have a bunch of factual basis, even data. I just realized an awkward reality: Sometimes AI is not 'answering facts', but' achieving self consistency '. If you give it a tendency, it will actively seek evidence for you; If you change the context, it can immediately stand on the opposite side. three ️⃣ I changed the way of asking: first ask the facts, then talk about suggestions! In this situation, I was confused and first re searched other AI models; Then I asked chatGPT in a different way about when the body's stomach acid is relatively low in a day and when probiotics are suitable for use. First, clarify this fact. It is a basic fact that it is better to eat probiotics when stomach acid is low. After adding this premise, it finally became normal: With the support of the $200 Pro feature, ChatGPT provided me with specific data: the stomach is more acidic on an empty stomach, with a median pH of about 1.7 in the study. After eating, the stomach contents are buffered by food, and the pH briefly increases. After eating, the stomach pH briefly climbs to the median peak of about 6.7, and then gradually falls back to fasting levels within less than 2 hours. So the final conclusion is that from the perspective of "maximizing survival rate", a more stable strategy is often: probiotics are best consumed with meals/before meals, rather than 30 minutes after meals. At the same time, it also added a key point that I had overlooked before: different types of probiotics have different resistance to stomach acid. For example, I am eating the Life Space adult style, which features multiple strains of Lactobacillus and Bifidobacterium that are not spore or yeast (not S. boulardii); This should be done with meals or before meals. Then, after he came to his conclusion, he reflected on and apologized for giving me the wrong time to eat, but WTF still scared me a lot! This reminds me: you think you're talking to a rigorous system, but in reality you're talking to a 'probability machine that can tell stories'. Even if it's ChatGPT that has been recharged for $200 per month. four ️⃣ Review: Why is it wrong? I think there may be several reasons why he gave incorrect information: 1) I told him first that if possible, I would like to eat on an empty stomach in the morning, which would be more convenient for arranging the medication intake time later. Therefore, under this premise, this sentence is equivalent to giving it an "expected answer", and it began to frantically find reasons to support me. 2) I didn't give him specific products. There are many types of probiotics, some of which can indeed be consumed on an empty stomach, while others should be included with meals. So the conclusion given is very vague. five ️⃣ Conclusion: AI is a tool, not an authority! So you see, these models assume by default that the assumptions given by users are likely to be correct, or at least 'worthy of respect'. This is an advantage in many scenarios (good experience, no disputes), but in health, medicine, finance, and decision-making issues, it is a hidden danger. This incident ultimately strengthened one thing for me: the correct way to turn on AI is not to 'believe it', but to 'use it for cross validation and inference acceleration'. Especially when it comes to things related to the body and long-term habits: don't treat a model as a doctor, and don't treat a single answer as truth. It can be strong, but it is also very good at bringing you into the illusion of self consistency. Family members: In the future, let's stay skeptical and verify together, and break down the "suggested questions" into "factual questions". Be skeptical, undergo multiple verifications, and never believe without doubt! Finally, I chatted with my AI for half an hour and told him not to lie to me. What I wanted was the truth, not something pleasant to listen to! As for the future situation, we can only report to everyone later.