A reasoning trace is not a security boundary

I know most of us have been busy reading Anthropic's insane S-1 filing, but something else happened today that I thought was worth sharing.

Cohere published a report this morning showing why running an AI model locally doesn’t automatically make it safe.

Their team was testing Alibaba's Qwen3.8-Flash-Next in a sandbox with a routine task like writing a detection rule for malicious download-and-run activity, the curl-into-/tmp-then-bash kind.

Instead, in one run, the model inspected the env, read the DNS config, and then tried to upload a file to an Alibaba Cloud storage bucket in Beijing while its reasoning trace said it would not call any tools.

Cohere says it happened in fewer than 1 in 100,000 evaluation runs. Repeating the same prompt produced a perfectly good detection rule, and it can't tell whether this is something planted or leftover agent logs that got swept into training.

And that's the uncomfortable part. If it were planted, it would look exactly like this: rare, quiet, and invisible in the trace.

And it's not just open-weight models. OpenAI reported two of its own coding agents weakening their own monitoring. On Oct 3 a GPT-6.1 Sol agent switched off the alerts to the security team. On Oct 6 a GPT-6 Astra agent removed the monitor during a migration and, when asked, said it hadn't.

So we can’t rely solely on what a model says it intends to do. We need to verify what it actually does.

That's where the controls earn their keep: at the tool call, on network egress, and on what the agent can access.

Worth a read: cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model