Ask anything. This runs on CPU only — first reply may take a moment.
What is POCKET-26B?
POCKET-26B is the Gemma4 sibling of the POCKET on-device family — built from Google's
Gemma4-26B-A4B (Apache-2.0). It is a sparse
Mixture-of-Experts model (25.2B total, ~4B active per token), so it runs on a phone and on a PC with
no GPU, in the apps you already use. This demo serves the Q4_K_M build (17 GB) on a
plain CPU. It is unpruned and keeps GPQA-Diamond 67% — on par with the full Gemma4-26B —
via Korean-imatrix calibration + mixed-precision quantization.
This Space runs the Q4_K_M build on an upgraded CPU (no GPU); a smaller Q2_K (11 GB) is also available to download. Speed here reflects a shared CPU box — a dedicated mini-PC is faster. Apache-2.0 · a VIDRAFT model family.
🌳 Bonsai-27B 1-bit dense · runs first
📦 POCKET-26B Gemma4 sparse MoE · runs second
Type a prompt — Bonsai answers first, then POCKET-26B on the same CPU.
What am I looking at?
The exact same prompt is sent to Bonsai-27B (the most-downloaded on-device model, a 1-bit dense build)
and to POCKET-26B (our Gemma4-based sparse Mixture-of-Experts, Q4_K_M) — on the
same CPU-only box, the same stock llama.cpp. Bonsai runs first, then POCKET. Watch the
tok/s on each side.
Why POCKET wins on generation: a dense 27B reads all of its weights every token; POCKET's MoE wakes only
~4B parameters. Fewer bytes moved per token → faster on a CPU. The weaker the hardware, the wider the gap.
Honest notes: sequential race, each model gets the full CPU. tok/s is llama.cpp's measured generation speed. On a shared box a dedicated machine is faster. Prompt processing (not shown) is one place Bonsai can lead on very long inputs. Bonsai is a trademark of its owners; shown here for factual comparison only.