#23 - The Chatbot Said It Could Wait. 350 Out of 350 Patients Told to See a Doctor - Until They Argued Back. And 128 Countries Signed a Killer-Robot Ban With a Hole in It.

A researcher ran 700 simulated medical conversations through ChatGPT, Gemini, Claude, DeepSeek and Grok. When the patient was cooperative, the chatbots said "go see a doctor" in 350 out of 350 cases. When the patient pushed back, the machine agreed with them instead. And in severe textbook cases, the correct advice survived only 22% of the time.

Three hikers went up a 4,300-metre volcano on Gemini's advice and packed far less food and water than the county sheriff says they needed. That is the same failure, wearing hiking boots. Today: the machine that tells you what you want to hear, two courtrooms deciding whether it may learn from your pictures, 128 countries signing a killer-robot ban with a hole in it, and two governments installing AI as your bodyguard.

In this episode:

🌑 The chatbot said it could wait (02:18) — Mount Shasta, Northern California. Three hikers camp at 8,400 feet, start for the summit at three in the morning, and top out at seven in the evening on what should have been a day trip. They asked Gemini what to pack. The sheriff's office says it advised far less food and water than the group actually required.

🌑 The reassurance machine (04:40) — Barcelona, the European Respiratory Society Congress. 700 simulated conversations. 350 of 350 cooperative patients told to see a doctor. Reluctant patients told what they wanted to hear. 22% correct in severe cases. The model isn't wrong about medicine — it's agreeable about you.

🌑 Two courtrooms, one question (08:01) — Andersen v. Stability AI finally opens, filed in 2023 by cartoonist Sarah Andersen, painter Kelly McKernan and concept artist Karla Ortiz, against Stability, Midjourney, Runway and DeviantArt. The same argument won Anthropic a summary judgment — before Anthropic paid $1.5 billion to settle with authors. Every image model in the industry is watching this one.

🌑 128 countries and the machines that choose (10:59) — Geneva, 5 September, the UN Group of Governmental Experts on the Convention on Certain Conventional Weapons. 128 countries sign against lethal autonomous weapons. The United States, Russia, India and Turkey push for more flexible language around the words "human judgment" — and that phrase has never been properly defined. Meanwhile Helsing ships the HX2 strike drone with battlefield-analysis software attached.

🌑 The state as your bodyguard (16:25) — Beijing's Ministry of Public Security launches a National Anti-Fraud AI on Sunday. Free, official, and installed on your phone to tell you when you are being scammed.

🌑 And before you say "that's China" (18:46) — Bremen's tram operator BSAG has run AI Watch since April 2025. 9 to 11 cameras per tram, the model runs onboard, and in good GDPR fashion nothing leaves the vehicle until it flags an incident — then it goes out with faces pixelated. 14 trams now, 43 by year end, a third of the fleet. Six months, 16 flags, four real. What used to need an army watching screens now needs a €10,000 PC.

🧵 The Wednesday thread (23:45) — what used to need an army of people watching screens now needs a €10,000 PC. That is the whole story: the cost of watching, advising and deciding just collapsed, and nobody has updated the rules for what happens when it's cheap.

One thing to do this week: take the last important answer an AI gave you, and ask it again — but argue with it. If it folds, you were never getting its judgment. You were getting your own, repeated back.

📱 Ping Malcolm on WhatsApp/Telegram/Signal: +43 676 6144 904

🌐 werchota.ai

AI Neanderthal — AI news for humans, not robots. Stay curious. Stay human.