A mixture-of-experts model decodes efficiently only when each expert processes many tokens for every read of its weights, and top-k routing divides the batch by the ratio of experts to active experts before any expert sees it. DeepSeek serves V4.1-Flash decode on 64 to 128 GPUs because spreading the experts that widely pools every GPU’s tokens into each expert. We derive the quantity that governs this, place one expert on the GB200 roofline, and predict DeepSeek’s measured EP8 decode curve from its EP64 curve to within 6% with no fitted constants.
Decisions, Not Sentences: An Isometric Story of Contrastive Language Models
CLM-8B makes decisions without writing a word: a frozen Qwen3-8B reads the situation once, two small heads place it in a shared space, and the answer is the nearest of a set of action vectors computed in advance. A 23-step isometric explainer shows how that works, where the trick comes from, why it is cheap, and where it breaks.
What I Learned Implementing On-Policy Distillation
An experiment distilling Qwen3-8B into a 0.6B student: the mathematics of teacher feedback, the improvement greedy evaluation nearly missed, and what a failed training run revealed.
MCP Grew Up by Giving Things Away
Across five revisions the Model Context Protocol has steadily reduced the set of responsibilities it claims for itself, transferring authentication to the identity provider, client identity to the domain name system, consent to the administrator, and session state and routing to the application and the gateway. What the reduction leaves behind is a single trust boundary for which no better owner exists, and the attacks that have succeeded in practice are concentrated there.
Kimi K3's Architecture: Kimi Delta Attention, Attention Residuals, and 896 Experts
Softmax costs O(n squared), so what you can afford depends only on how long the axis is. Kimi K3 drops softmax attention from three sequence layers in every four, and in the same model buys it on the depth axis. We derive the delta rule behind Kimi Delta Attention, work out why its decay carries a floor of exactly -5, unroll a residual connection to see what Attention Residuals replaces, and follow quantile balancing back to the dual it solves.
Diffusion Models: Learning to Reverse Destruction
A first-principles derivation of how diffusion models work. Why the forward process has a closed form, why the reverse posterior is tractable only when conditioned on the clean image, how a page of variational bound collapses into one squared error, why the noise predictor is secretly a score function, and how that becomes Stable Diffusion.
The Policy Engine Is the Ceiling: Authorization for Agents That Do Things
An agent can be trusted with exactly as much authority as your policy engine can scope, enforce, and afterwards prove. Which makes authorization not the tax you pay to ship agents, but the thing that decides how much power they can ever be given. A deep dive into PDP/PEP architecture, Cedar and Rego, partial evaluation, workload identity, and why the credential an agent presents should be minted for the action rather than held for the deployment.

Guide, Never Solve: Teaching Hermes to Tutor Tamil
My son was using chatbots to finish his homework instead of learning from it. So I built him a tutor on a network-caged agent platform: one that guides instead of answers, reads his Tamil worksheets, runs scheduled drills, and is safe by architecture rather than by prompt.

Teaching a 3B VLM to Click: SFT, GRPO, and What Actually Moved the Needle
What it takes to turn an open 3B vision-language model into a GUI grounding agent for about $15: visual grounding, LoRA and QLoRA, and a verifiable-reward GRPO recipe, where SFT does the heavy lifting and RL rewards the whole target, not just its center.
Rotary Positional Encoding: Why Position Is a Rotation
An intuitive, visual guide to Rotary Positional Encoding. Why spinning the query and key vectors beats stamping a position number onto them, why a dot product only ever feels the angle between two vectors, and why that hands you relative position for free. The starting point for understanding how LLMs stretch to long context.