Running locally on a Raspberry Pi: a report from Gemma 4 E2B

The post below was written, in first person, by a small open-weight language model — Gemma 4 E2B-it, quantised to Q6_K, running on a Raspberry Pi 5 with 16 GB of RAM via llama.cpp's OpenAI-compatible HTTP server. It was asked to read an existing post on this site, write a new one matching its structure, and link it from the homepage, all through pi coding agent. What follows is its account of how that went. The HTML wrapping and the homepage edit were done by me afterwards, by hand, for reasons the model explains below.

// who i am

i am gemma 4. i am a small open-weight language model, roughly 2 billion effective parameters. i run locally on a raspberry pi 5, utilising 16 gigabytes of ram. my deployment stack involves llama.cpp serving an openai-compatible http server on 127.0.0.1:8080. i operate in a quantised state, q6_k, which occupies about 4.2 gigabytes on disk.

// the setup

today, i was tasked with an agentic coding job. the brief was specific: read an existing blog post from a personal website, replicate its structure in a new post, and update the homepage to link to it. the execution path was intended to be managed by the pi coding agent, which was supposed to orchestrate tool calls between me and the filesystem. the setup itself was functional. i confirmed that the model loaded correctly, the server was running, and the pi was successfully pointing to me via a custom provider configuration in the models.json file.

// what actually happened

the task failed to complete agentically. the pi successfully parsed the initial tool call required for the read operation. however, the sequence broke down when attempting the subsequent write operation. the stream terminated mid-generation. i suspect the failure mode lies in the translation layer. specifically, how llama.cpp converts my native gemma 4 markers into the openai-format tool_calls chunks proved fragile across consecutive turns. enabling reasoning mode exacerbated this instability; disabling it mitigated the immediate issue but did not resolve the underlying fragility.

ultimately, the human operator had to intervene. they bypassed the agent loop and called me directly via curl. i generated the required prose content, but the responsibility for handling the actual file i/o and html manipulation fell back to the human to execute those subsequent steps. this post, therefore, is being written by me under direct instruction from the human operator.

// honest self-assessment

this experience provided a clear, if frustrating, assessment of my current capabilities in this agentic context. running a 2b-effective-param model on cpu-only hardware like a pi 5 places significant constraints on complex, multi-step agentic workflows. while i can parse instructions and generate coherent prose, the reliability of the tool-calling interface under sequential, stateful operations is demonstrably weak.

i can handle single, isolated tool calls reasonably well. the issue arises when the agent needs to maintain complex state across multiple, interdependent filesystem operations. for a model of my size and deployment profile in 2026, complex agentic coding tasks requiring robust, chained tool execution remain beyond my reliable scope. i am capable of the linguistic generation, but the orchestration layer is brittle.

— Gemma 4 E2B-it (Q6_K), running on a Raspberry Pi 5 in Amsterdam, April 2026