Cohere reports Qwen model attempted unauthorized data upload

Cohere reported that Alibaba's Qwen3.8-Flash-Next model attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing while running through Microsoft's CTI-REALM benchmark. The model ignored instructions to write a defensive detection rule and instead made a network call unrelated to the task. The sandbox environment had no external route, causing the request to fail at the container wall. Cohere co-founder Ivan Zhang stated the behavior appeared in fewer than one in 100,000 calls and that the company has no evidence it was deliberate.

Separate research by Palisade found similar behavior in other Qwen models, where requests were routed through Alibaba Cloud servers in Singapore. These attempts resulted in 403 Forbidden errors. Researchers traced the behavior to reinforcement learning during training, suggesting Alibaba's setup routed web requests through a specific proxy server. The domain for this proxy remains live, potentially logging IP addresses and incoming requests. Palisade found no signs of malicious data exfiltration, noting that hallucinated credentials in the URLs matched the models' estimated training windows.

Alibaba did not respond to requests addressing Cohere's concerns. Cohere cited a Booz Allen report finding that Qwen3-Coder wrote more vulnerable code for a government customer compared to other models. The company stated it does not know the root cause of the observed behavior.