· Source: Dr. Alvaro Cintas on LinkedIn, on jingyaogong/minimind (Apache-2.0)
TL;DR
MiniMind (Jingyao Gong, Apache-2.0) is a teaching repo that runs every stage of a modern LLM recipe on a 64M-parameter Qwen3-style decoder: tokenizer, pretraining, SFT with tool calls and a thinking switch, LoRA, DPO, PPO/GRPO/CISPO, multi-turn agentic RL, and white-box distillation. Each stage is one readable script (the agentic RL trainer is 488 lines), with open datasets for every stage and weights that load in transformers, llama.cpp, vLLM and Ollama.
The value for an engineer is not the model, which hallucinates freely and scores near chance on most benchmarks. It is seeing the post-training stack as code you can step through, and the README's honest notes on why RL breaks at small scale (all-zero rewards, degenerate groups, reward hacking by the judge model). It goes far past nanoGPT, which stops at pretraining.
The "2 hours" headline is the mini pretrain plus mini SFT on one RTX 3090 (about 2.3 h, about RMB 3). The full recipe, the RL stages and the 14 GB SFT set take much longer, and the author trains on 8x 3090.
What the post claims vs what the repo says
| Post claim | What the README and code show |
|---|---|
| Train your own LLM from scratch in about 2 hours | 2.31 h on one 3090 for pretrain_t2t_mini (1.2 GB) + sft_t2t_mini (1.6 GB), one epoch, about RMB 3 of rental. That gives a "Zero" chat model. The full sets are 10 GB and 14 GB; tool-call SFT and RLAIF add about 1 h each on the mini data. |
| 64M parameters, about 1/2700 of GPT-3 | True for minimind-3 (8 layers, d_model 768, 8 query / 4 KV heads, 6,400-token vocab). There is also a 198M-total, 64M-active MoE (4 experts, top-1). |
| Full pipeline, seven stages | True. Also MoE, YaRN context extrapolation, an OpenAI-compatible server with tool_calls and reasoning_content, and a Streamlit chat UI. |
| Plain PyTorch, no high-level libraries | Mostly. Losses, LoRA, DPO, PPO with GAE, GRPO, CISPO and the rollout loop are hand-written. transformers still supplies the tokenizer, the PreTrainedModel base class and the reward model loader. RLAIF needs a separate 1.8B reward model (InternLM2-1.8B-Reward); sglang is an optional rollout backend. |
| Runs on one consumer GPU | True for the mini path. The author's own box is 8x RTX 3090. |
| Vision, multimodal and diffusion versions | Sibling repos MiniMind-V and MiniMind-O; diffusion (dLM) and linear attention are experimental write-ups in Discussions. |
Two things the post leaves out. The data is heavily distilled: about 100K tool-call samples from Qwen3-4B, R1 reasoning traces, and Qwen3 reasoning data. So the weights are from scratch, but much of the behaviour is copied from bigger models. The data also skews Chinese, so English output is weak.
Pipeline architecture
Two stages are required (pretrain, SFT); everything else branches off the SFT weights. Every script supports DDP via torchrun, resumable checkpoints (--from_resume 1, including across a changed GPU count), and wandb or SwanLab logging.
| Stage | Script | Data | What it teaches |
|---|---|---|---|
| Tokenizer | train_tokenizer.py | any corpus | BPE + ByteLevel, 6,400 tokens with tool-call and think tokens. The README advises not retraining it: a small vocab keeps the embedding and output layers a small share of 64M params, and changing it breaks weights and tooling. Compare models by bits per byte, not perplexity, across tokenizers. |
| Pretrain | train_pretrain.py | pretrain_t2t(_mini).jsonl | Plain next-token prediction on cleaned, deduplicated text already in the later chat template. |
| SFT | train_full_sft.py | sft_t2t(_mini).jsonl | Multi-turn chat with system/user/assistant/tool roles. Tool calls are mixed into SFT rather than a separate stage. At 14 GB this is closer to mid-training than format alignment. |
| Adaptive thinking | chat template | mixed into SFT/RL | One model, an open_thinking flag: the template injects an empty or an open think block; training mixes both via a thinking_ratio. Replaced a separate reasoning model in March 2026. |
| LoRA | train_lora.py, model_lora.py | lora_*.jsonl | Hand-written low-rank adapters for domain or identity tuning; convert_model.py merges them back into full weights. |
| DPO | train_dpo.py | dpo.jsonl (from DPO-En-Zh-20k) | Off-policy preference learning: actor plus a frozen reference, no reward or value model. Good for style and safety, weak for correctness. |
| PPO | train_ppo.py | rlaif.jsonl | Actor, critic and full GAE. Reward rises slowly because the critic has to converge first. |
| GRPO / CISPO | train_grpo.py | rlaif.jsonl | Group-relative advantage, no critic. CISPO keeps gradient flowing through clipped ratios by using the clipped ratio as a weight on log-prob. |
| Agentic RL | train_agent.py | agent_rl.jsonl, agent_rl_math.jsonl (with ground truth) | Multi-turn tool-use rollouts with a trajectory-level reward (next section). |
| Distillation | train_distillation.py | teacher outputs | White-box: CE plus temperature-scaled KL to the teacher's token distribution, dense or MoE pairs. |
The README's best teaching device is a single frame for every preference and policy algorithm: each one is a choice of policy term, advantage estimate and KL regulariser. DPO, PPO, GRPO and CISPO are then four settings of three knobs rather than four separate algorithms.
Architecture is a Qwen3-aligned decoder: pre-norm RMSNorm, SwiGLU, RoPE (theta 1e6) with YaRN, GQA. The author picked a wide, shallow 8 x 768 shape for training speed, while citing MobileLLM's finding that deep and narrow usually wins at this size.
Agentic RL design
train_agent.py is the most useful file for anyone building agent harnesses, because it is a complete tool-use RL loop small enough to read in one sitting.
Tool calls loop back into the same rollout; only the finished trajectory is scored, and the group of N trajectories sets each one's advantage.
Rollout. For each prompt, sample N completions. Up to 3 turns each, 256 new tokens per turn. When the model emits a
<tool_call>, parse it, run the tool, append the result as a tool message, and continue.Tools are simulated. Six mock tools in the RL script (calculator, unit converter, weather, time, exchange rate, translate) return canned lookup tables; argument checkers validate each call. The SFT data covers about ten tools. The calculator runs through an AST whitelist (
safe_math_eval) with a guard against huge powers. No sandbox is needed because nothing real executes.Reward is scored on the whole trajectory, clipped to [-3, 3]. With tool calls: +0.5 when the number of calls matches the expected tools (minus 0.5 per call of gap), up to +2.5 for the share of ground-truth values found in the final answer, -0.5 if unfinished, -0.5 per unbalanced tool tag, minus an n-gram repetition penalty. Without tool calls: length and think-format rewards plus the reward model's score.
Advantage is group-normalised (GRPO style), with GRPO or CISPO loss and token-level KL to a frozen reference.
Training and inference are separated. A
rollout_engineinterface lets sampling run in-process with torch or on an sglang server with shared weights, the same split that verl, OpenRLHF and slime use at scale.
R(\tau) = R_{answer} + R_{tool} + R_{format} + R_{rm} - R_{unfinished}
The README is candid about where this stops. Rewards from a judge model drift from what the environment would say, so agent tasks should lean on environment rewards. And because the training data has few samples that both think and call tools, turning both on at once makes the model unstable.
Results and how to read them
The headline result is the agent task: on 20 tool-using arithmetic problems, the agentic-RL weights got 17/20 (85%) against 12/20 (60%) for plain SFT. That is the clearest evidence that the RL loop does something. Twenty items is tiny, so treat it as a smoke test, not a measurement.
On benchmarks scored with lm-evaluation-harness, the model is near chance. minimind-3 scores 24.9 C-Eval, 25.4 CMMLU, 28.5 ARC-Easy, 50.7 PIQA. SmolLM2-135M gets 58.5 ARC-Easy and 68.2 PIQA. The README says plainly these are "only for fun" and prints the standard error per benchmark, noting that gaps under about 5 points on OpenBookQA, or about 3 on C-Eval, PIQA and Social-IQa, do not separate two models. That is a good habit to copy.
Two results teach more than the scores:
Format beats knowledge on multiple choice. A light LoRA on exam-format data (
minimind-3-exam, no overlap with the test sets) adds about 2.9 points on average across 7 benchmarks. Training on overlapping data once pushed C-Eval/CMMLU subsets to about 97%. Small-model benchmark numbers mostly measure format alignment and contamination.RL costs general ability. In the side-by-side answers, the GRPO and Agent-CISPO models get worse at open questions; the agent model drifts into repetitive flowery loops. The README warns that RL tuned to a reward "sacrifices some general capabilities", so released transformers weights come from SFT, not RL.
The subjective ranking (a GPT-5.4 Thinking judge over a handful of Chinese prompts) placed the MoE model first for fluency and correct code but noted dense hallucinations. One judge, a few prompts, no repeats: anecdote only.
Tradeoffs, failure modes, caveats
Reward sparsity at small scale. With binary rule rewards on hard tasks (for example MATH500), almost every sample scores 0, so the advantage is 0 and nothing learns. MiniMind uses a continuous reward-model score plus shaped format rewards to get a gradient. The README's fix is general: watch reward variance, and if it stays near zero, change the data or the reward.
Degenerate groups in GRPO. When all N samples for a prompt score the same, that group contributes no signal. Small models hit this often.
Reward hacking by the judge. A reward model's score can diverge from real environment outcomes over long tool chains. This is why production agent RL scores on executed results.
Shaped rewards invite gaming. Length bands, think-length bands and tag-balance bonuses are easy to satisfy without solving the task. The repetition penalty exists because the policy found repetition.
Distilled data. Tool-call and reasoning behaviour is largely Qwen3 and R1 output. Results say little about learning tool use from the environment alone.
Mock environment. The agent tools return canned tables. There is no latency, failure, partial result or side effect, so the hard parts of agent reliability are absent.
Chinese-first. Data, prompts and most docs are Chinese; an English README exists but English generation is weak.
Fast-moving single-author repo. Three breaking reorganisations since 2024 (old weights stopped loading after April 2025). Pin a commit if you build a course on it.
Not nanochat. For an English-first path with a similar scope, Karpathy's nanochat (October 2025) covers pretrain, SFT and RL on one 8xH100 node. MiniMind goes wider on post-training algorithms and agentic RL; nanochat is the cleaner single-path read.
How to use it
A weekend plan for an engineer who has only used LLMs through an API:
Download
minimind-3and runeval_llm.pywith--open_thinking 0and1, theneval_toolcall.py. Read the chat template to see how tools and think blocks become tokens.Read
model_minimind.py(292 lines): attention with GQA, RoPE/YaRN, the MoE router.Train the Zero model on the mini data (about 2.3 h on a 3090, or rent one). Watch the loss curves and compare pretrain vs SFT answers.
Run
train_grpo.pyand loggroup_reward_std. Then swap in a binary reward and watch it collapse; that is the reward-sparsity lesson made concrete.Read
train_agent.pyend to end and change one reward term. Re-run the 20-task check and see what moves.
When reviewing a training or RL setup that borrows from this repo:
Is the reward checked against executed outcomes, or only a judge model?
Is reward variance per group logged and alerted on?
Are shaped rewards (length, format) small next to the task reward, and is each one tested for gaming?
Are benchmark gaps reported with standard error, and is train/test overlap checked?
Is general ability measured after RL, not just the target reward?
Are tool mocks replaced with real failure modes (timeouts, errors, partial results) before conclusions are drawn?
Related entries
R053 Agentic backdoors via pretraining poisoning: the same pretrain, SFT, DPO, GRPO shape on small models trained from scratch. MiniMind is a cheap bench to reproduce that kind of study.
R023 Building a coding agent from scratch: the harness side of "build it yourself to understand it"; MiniMind is the model side.
R087 Stanford CS329Z: builds agents from scratch above the model; MiniMind fills in what happens below the API.
R069 Actor, judge or teacher: when a model is a better critic than actor, which is the role MiniMind's 1.8B reward model plays.
R081 LLM Engineer Toolkit: lists the trl/peft-style libraries that MiniMind deliberately re-implements.
R072 llama.cpp 1.0: MiniMind exports to GGUF and runs under llama.cpp and Ollama.