Qwen3.5 2B: the one we use ourselves
1.6GB, a dedicated runtime, a KV cache kept between turns. The smallest model of the catalogue on which an agentic loop holds.
Between 0.8B and 4B there was a gap, and everybody who used the site found it. The small model answers immediately but loses the thread after three files. The big one is excellent and asks for 3GB of download. Qwen3.5 2B sits exactly in the middle, and in practice it is the model we use ourselves most of the time.
The numbers
| Budget | Value |
|---|---|
| Download | ~1.6GB |
| Working set | ~2.2GB |
| Minimum GPU buffer | 1GB |
| Precision | q4f16 |
| Runtime | qwen3_5 |
1.6GB is a real download. It is not a download you make by accident. But it is also the threshold from which the model stops being a demonstration and starts being a tool: it holds a conversation of twenty turns, it reads several files before answering, and above all it chains tool calls without breaking the format.
A dedicated runtime
You will notice in the table that the runtime is not the same as for the 0.8B.
Qwen3.5-2B-ONNX-OPT passes through a specific path, qwen3_5, and not
through the generic text-generation path.
The reason is the KV cache. The generic path re-prefills the whole prompt at each turn, which is acceptable for a chat of a few messages and catastrophic for an agent that accumulates tool results. The dedicated runtime keeps the cache between turns and re-prefills only what has changed. On a long conversation, the difference is not a few percent, it is a factor.
This is what the context meter in the interface shows you: the number of tokens in the prompt, and among them how many were read again from the cache instead of being recomputed. When the second number is large, your turn is fast. When it falls to zero, something has invalidated the prefix — usually a system prompt that changed, or a message edited in the middle of the history.
The sampling
The same preset as the rest of the Qwen family:
{ "temperature": 0.7, "top_p": 0.8, "top_k": 20 }
At 2B, the model supports a hotter sampling better than the small ones, but we have kept the official values. They are good, and a preset that varies from one model to another for no reason is a preset that one can no longer debug.
Our recommendation
If you have 4GB of free VRAM and a serious connection, this is the model to take. It is the default choice in the Agent for good reasons: it is the smallest of the catalogue on which the agentic loop works reliably, turn after turn, without the tool call format degrading.
Below, look at LFM2.5 350M or Qwen3.5 0.8B. Above, there is Qwen3.5 4B and Gemma 4 E4B.