Qwen4’s architecture is here early, firing 6B parameters out of 125B
Alibaba’s Qwen team has released Qwen3.8-Flash-Next, an open-weight preview of the architecture it intends to use for Qwen4, carrying 125B parameters but activating only 6B for each token. Its licence may not qualify for the EU AI Act’s open-source exemption.
Alibaba’s Qwen team has published the architecture it plans to build Qwen4 on. Qwen3.8-Flash-Next carries 125B parameters and fires only 6B of them for each token it produces.
The claim is about cost rather than capability. The team says its concern is what architectural choices do to inference bills as agentic jobs with very long contexts become the normal workload.
The comparison it draws is with its own last model. Qwen3.7-Plus holds 397B parameters and activates 17B, so this one runs on roughly a third of the active compute.
Three of the four changes are conventional enough. A new sparse attention scheme works on micro-blocks instead of picking individual tokens,...
Copyright of this story solely belongs to thenextweb.com. To see the full text click HERE
