Qwenputer Embeds a Working RISC-V CPU in the Weights of Qwen3.5

A developer has built a version of the Qwen 3.5 4B language model that carries a working processor inside its own weights. The project, called Qwenputer, can run arithmetic, C code and MicroPython without calling an outside tool, plugin or sandbox. The computation happens during the model's ordinary text generation.

Divyansh Agrawal, who published the project, describes it as a stock Qwen 3.5 4B model with one addition. Embedded in the weights is a 32-bit RISC-V core that implements the RV32IM instruction set, along with the Zba extension and a small unit for big-number arithmetic. The processor takes up 11 layers in front of the model's 32 existing layers.

The trick lies in a part of Qwen 3.5 that already exists. Three of every four attention layers in the model use Gated DeltaNet, a linear recurrent neural network. Each attention head keeps a state matrix, and that state normally holds a blended, fading summary of what came before. Agrawal found that by setting the gate to 1 and the write strength to 1, and by using one-hot keys, each write overwrites exactly one row and each query reads back an exact value. That turns the layers into addressable memory.

The advantage is cost. A Gated DeltaNet head's state stays the same size however long a conversation runs, unlike the key-value cache that standard attention relies on. Agrawal writes that the CPU keeps its registers, memory and program counter in those rows.

The idea that the delta rule can act as associative memory is not new. Schlag and colleagues showed in 2021 that linear attention with the delta rule works in that way. Agrawal's contribution is making that memory exact and addressable inside a production model, reliable enough to run a full processor.

No training was needed. A compiler sets the weights directly from a description of the CPU as a long list of small steps, such as reading a register, adding two numbers or jumping if zero. Each step becomes part of a transformer layer. Attention heads do lookups, feed-forward units handle the math, and the recurrent heads hold the memory. Each instruction produces a few generated tokens, and the CPU runs as the model decodes them.

To get Qwen to use the processor reliably, Agrawal reused the tool-call format that the model already knows well. Qwen sends instructions to the CPU and reads back the results. A small driver, called RvChat, loads the firmware into the recurrent state and keeps the processor's trace tokens out of the model's text layers. The original weights are unchanged. The new circuitry sits alongside them in extra dimensions and layers.

Benchmark results

On the DeepMind arithmetic sets, Qwenputer scored 97.5% on division of large numbers. A base Qwen 3.5 4B with a Python tool scored 95% on the same test, and the unmodified model scored 72.5%. On multiplication of large numbers, Qwenputer scored 97.5%, the Python-tool version scored 100%, and the base model scored 55%. On a mixed multiplication and division set with longer problems, Qwenputer reached 100%, compared with 97.5% for the Python version and 90% for the base model. On addition and subtraction of large numbers, Qwenputer matched the Python version at 97.5%, against 87.5% for the base model.

On the NUPA long-integer arithmetic benchmark, Qwenputer rose from 48.6% to 100%, the same result as the Python-tool setup. Agrawal says he replayed about 230,000 of the CPU's output positions against a reference machine and found no mismatches. He also reports that the model's ordinary text barely changed, with a divergence from base Qwen that falls within measurement noise.

A filesystem in the model

Agrawal also added an experimental filesystem. The model can save, append, read, list and delete files through internal operations, again without external tool calls. File names are hashed into one of 1,024 slots in a directory held in the recurrent state, and the file text is stored in other heads, eight tokens per row. The system holds up to 512 files, or about 53,000 tokens of text.

Because the state can be saved, a file system can carry over between conversations. Agrawal says Qwen retrieved facts that had changed between chats correctly 90% of the time, compared with 30% when the old chats were pasted back in. On that test it performed about as well as memory tools run through Python, at 83.3% against 78.9%. He describes the filesystem as still experimental.

The project has clear limits. The checkpoint is 18.8 GB, roughly double the size of the base model, although most of that is zero padding. It needs the RvChat driver and does not yet run with a plain generate() call. Each CPU instruction costs several decoded tokens, so a large calculation takes seconds rather than milliseconds. The batched executor used for the benchmarks ran about 3,200 CPU tokens per second on an L4 GPU.

Agrawal's next goals include getting Qwen to write Python and run it on the MicroPython inside its weights, and making the CPU faster. The code is on GitHub, and the models are on Hugging Face and ModelScope.

Este artículo fue escrito con la asistencia de IA.
News Factory APP - noticias agénticas para impulsar tu SEO y AEO.