#42 - Anything You Prompt Can Be Used Against You: AI in Ordinary Hands

A Florida woman used Claude as a diary, wrote that she would shoot up the sheriff's office, and Anthropic reported it to the police. The same week, attackers walked through the side doors of seven South Korean banks with a free AI tool, and a jailed ex-president came back on stage as an AI clone.

Today is one question: AI is in ordinary hands now, for good and for bad. Do you know who else reads what you type?

In this episode:

🌑 Anything you prompt can be used against you (01:47) — a 30-year-old woman in Lee County, Florida, told investigators she used Claude like a diary. On 26 September she wrote that she would "shoot up" the Lee County Sheriff's Office, and the next day she mentioned a new gun. Anthropic's safety systems flagged the chat, Anthropic reported it to law enforcement, and she now faces a felony charge for a written threat of a mass shooting. In August, former Goldman Sachs analyst Darren Zhou pleaded guilty to aggravated stalking and written threats after OpenAI reported his ChatGPT conversations about his ex-girlfriend to the FBI. He got eight years of probation. And in Washington State, journalists from Cascade PBS and KNKX used public-records laws to get thousands of pages of city employees' ChatGPT histories. Your chatbot doesn't only answer to you.

🌑 Seven Korean banks, one free AI hacking tool (06:48) — The Korea Times, Herald Business and Korea JoongAng Daily report that since about 1 October attackers broke into seven banks and lenders, including Shinhan Bank and Hana Bank. More than 25,000 people were hit at Shinhan alone and about 40,000 at Yegaram Savings Bank, more than 60,000 in total. Names, phone numbers, income, loan limits and some national ID numbers leaked. The attackers didn't use the banking app. They came through side doors: websites for employees, developers and loan agents. Investigators traced the tool to ARTEX AI, a free, AI-based penetration-testing tool on GitHub. President Lee Jae-myung ordered a full investigation on Sunday, 4 October. My take: most security teams don't yet check every day for new AI attack tools, and anyone can now download one in minutes.

🌑 Brazil: a jailed ex-president's AI clone on stage (12:06) — in Sunday's first round, Flávio Bolsonaro took about 47 percent and President Lula about 45 percent, so there is a runoff on 25 October. Flávio launched his campaign with an AI video of his father, former president Jair Bolsonaro, who is serving 27 years for plotting a coup. The video said it was an AI simulation, but Jair Bolsonaro's lawyer said his client never authorised the use of his image and voice. Brazil's top electoral court voted four to three not to punish Flávio. The same trick works on companies: clone your CEO's voice from conference videos and call the accounting department.

🌑 Europe puts a hidden signature on ChatGPT's text (16:56) — because of the EU AI Act, OpenAI will put a hidden watermark called textGrain into text from ChatGPT and Codex for users in the EU. A secret key decides between equally good words, like "big" or "large", and over a long text that leaves a pattern a detector can find. OpenAI says its detector finds about 95 percent of marked texts of around 300 words. But TechCrunch reports that swapping just 10 percent of the words drops detection from 92 to 66 percent, and The Decoder says swapping a quarter drops it below 20 percent. Outside the EU it's off by default. My view: a box-ticking rule that is easy to get around.

🌑 Munich's RobCo becomes a unicorn (20:31) — The Wall Street Journal and heise report that RobCo, a robot start-up from Munich, is now valued at more than a billion dollars, double January's roughly 500 million. This time it was a secondary sale: mostly employees sold about 40 million dollars of their own shares, so no new money went into the company. Backers include Sequoia, Lightspeed and new investor Cherry Ventures. RobCo was founded in 2020 by three researchers from the Technical University of Munich. It builds modular robot kits, like Lego for factory robots, has sold more than 1,000 robots and counts BMW as a customer. Europe may have lost the race for large language models, but its Mittelstand factories are the perfect first buyers for physical AI.

🧵 The thread (24:09) — Andreessen Horowitz says only about 2 percent of US households pay for AI. Hackers and election campaigns already use it; most people haven't started. That's why I think the AI market is still wide open.

📱 Ping Malcolm on WhatsApp/Telegram/Signal: +43 676 6144 904

🌐 werchota.ai

AI Neanderthal — AI news for humans, not robots. Stay curious.