#33 - The US and China Just Proposed a Red Phone for AI. A Swedish Drone Picked Its Own Target. Berlin Wrote Itself Permission to Train on You. And Madrid Got a Breach Report Blaming an AI Agent.

Sixty-three years after the Cuban Missile Crisis gave us the red phone, the US and China sat down at JP Morgan in Manhattan and proposed one for AI. A Swedish drone picked its own target and dropped a bomb on it with no human involved — on hardware you could assemble at home. Berlin quietly wrote a paragraph into its police law allowing un-anonymised personal data to train AI. And in Madrid, a company reported a GDPR breach and named the culprit: an AI agent.

A short tour — the US, Sweden, Germany, Spain. And the thread underneath is the gap between what we agreed and what is already running.

In this episode:

🌑 A red phone for AI (01:47) — in 1962 the US and the Soviet Union built a direct line so a misunderstanding couldn't end the world. On Sunday, US Treasury Secretary Scott Bessent and Trade Representative Jamieson Greer met China's Vice Premier He Lifeng in New York, at JP Morgan's offices. Trade and rare earths were on the agenda — and for one of the first times officially, AI. The US proposed an AI safety notification mechanism: if there's an incident that might touch national security, we pick up the phone. Why it matters: if someone in the US points a model at Chinese companies, Beijing sees the pings and sees a US server. Without a line, that's an attack.

🌑 The Swedish drone that chose its own target (07:21) — a drone flew, ranked the objects on a test range, decided which was an armoured engineering vehicle, and dropped an explosive on it. No human in the loop. 192 countries signed up to the idea that autonomous AI shouldn't kill people. The company is Scale Out Systems, a spin-off from Uppsala University; Tom's Hardware covered it on 20 September. But the shock isn't that it happened — it's the bill of materials. An Aerolit S1 airframe, an NVIDIA Jetson Orin Nano you can buy off a shelf, and YOLOv8 Nano for detection. You could build this at home.

🌑 Berlin: pedestrians as training data (12:53) — in the country of GDPR, where you can be in trouble for a missing cookie banner, Berlin passed a police law with a section — §42d ASOG — specifically for training and testing AI systems. The rule says use anonymised data. The exception, in very small print, allows unchanged personal data without anonymisation. We've covered the behaviour scanners at Kottbusser Tor and on buses in Bremen and Hamburg. Now the system learning what "suspicious" looks like can learn it from you, by name. What shocked me most: somebody had to ask.

🌑 Madrid: "it wasn't me, it was the agent" (17:00) — on 16 September Spain's data protection authority, the AEPD, handled a breach notification where the attacker was an AI agent. It searched for files and vulnerabilities, logged in, looked for more weaknesses, stole personal data and altered it. No human involved at any step. This isn't from a safety paper or a LinkedIn post — it's a routine GDPR breach form. And it's the first time a company can point at something other than itself.

🌑 What we take from it (19:55) — pair a Swedish startup flying autonomous weapons with a police force writing itself an exception, and the shape of the week becomes clear.

Stay curious. Stay human.