I asked Gemma 4 - 31B to build something during my flight

So I took an extremely short flight (1 hour) without any in-flight internet. So naturally I thought this was the perfect opportunity to put an agent with a local model to use. At this point, I had used and experienced all sorts of personal agents/assistants from openclaw when it launched (I'm sure I've spun up at least 20 different clawdbot/moltbot/openclaw workers across several on and off prem devices) to writing custom Claude Code harnesses and giving it a sophisticated memory layer. In fact, a few of my openclaw endeavors involved trying to run a local model on a Mac Mini (base) with an M4 chip and 16GB RAM - this was an utter failure because this was around Feb/March if I'm not wrong and models back then were simply not smart enough to be run locally. Fitting in a 9B/8B model was just frustrating because of how dumb models were.

Time moves pretty fast in this industry, and here we are with several advancements to local, on-device AI in the last 3 months. Qwen dropped its 3.6 series of models (of which I used to daily drive the 27B - 4-bit quantized version on my MacBook for a while for menial tasks) and Gemma came back with its version 4 lineup - of which two models got my attention: google/gemma-4-31B-it and google/gemma-4-26B-A4B-it both of these seemed stellar. So much so that the 31B dense version is listed in arena.ai's agent leaderboard as #27 best model (ELO score) competing against its counterparts like Nemotron 3 Ultra (55B active parameters) and BETTER than Grok 4.3. This is particularly impressive for a model with just 31B parameters - that can fit on a mid-range MacBook!

I had just set up hermes agent and was tinkering around with models like the new GLM 5.2 and DeepSeek v4. I decided to hook hermes up with a local model running straight from my laptop - Gemma 4 - 31B Dense. I thought, okay let me ask it to make a good 3D racing game (the standard) and so I asked hermes:

In Developer within /labs - there is a gemmatests folder.
Within here - I want you to build a 3d car racing app website.
Note that there is no internet access - You're expected to build
this app with whatever tools that are locally available on this mac.  
  
The outcome here is to test your skills, so you're expected to bring
your A-game. Build the best possible implementation of a 3D car racing
application set in a world. Get creative. Use whatever locally available
packages there is to build the best implementation. Make us of html, js, css
and any other web based packages available to make this completely offline app/website.  
  
Bring your best game - let's see how well you perform!

Context window was set to max: 262144 tokens. As soon as the model got loaded, the memory was hovering around 40GB. The thing I noticed right away was power and heat. Note that by the time I started this - the flight had taken off and I had no access to the internet. The laptop instantly got too hot to keep on my lap. Fans kicking in - the GPU was pushed to max. CPU/GPU % wavering around 95-100%.

12 minutes in

The battery impact was terrible. By the time I started the experiment (gave the prompt) - my laptop had 65% battery. And in 12 minutes - I saw hermes doing everything it can - reasoning, calling tools, trying to "impress" me (as noticed from its CoT traces!) and the battery had dropped to 45%. A whole 20% battery drop in just around 12 minutes. Terrible.

15.5 minutes in

The first visible result came through at this point. The agent was still working on getting a few quirks right, so the game was not playable. At first glance, it's true when I say I was NOT IMPRESSED. But I still waited to see where this would get me. One more thing that was horrible was the throughput - the model, at this extremely long context length, gave me an average of 1-2 tokens per second, which is just diabolical. This got me thinking about whether we could use MTP/Decoding to get a speedup here. (There is an upstream blocker with support for Gemma 4 assistants in MLX at the time of writing, so it seems like this is only possible with llama.cpp at the moment)

The result

The final artifact (one HTML file) loaded in at around 21 minutes, and the hermes session ended. My laptop battery at this point was 28%. A solid 37% drop in around 20 minutes to generate this game:

This looked like an attempt in the right direction - but do note that there was no steering, no interruptions, and it was just a single-prompt, one-shot. While I'm certainly not impressed with it - I do see potential. The bigger drawback in my view was time taken / throughput. I concluded this model might be better for more involved tasks like this when time does not matter (sweeps run overnight, etc.), so I immediately switched over to the other Gemma model I was talking about earlier - google/gemma-4-26B-A4B-it / google/gemma-4-26b-a4b-qat - this is able to provide a good 60+ tps on a context length of 64k tokens - the sheer strength of Mixture-of-Experts 🫶🏽

Convergence

As I always say to people around me, I believe the convergence point of smaller models becoming smarter and hardware becoming more efficient to fit them in is just a ways down the road. It won't be long before frontier-level intelligence catches up to consumer hardware. And I'm particularly excited to get there!