R081 · AI education › Learning paths for AI engineers

LLM Engineer Toolkit: a library shortlist, not a ranking

Read the original

learning-resourceslibrary-catalogsagent-platformsagent-harnessdeterministic-workflows

Kalyan KS's LLM Engineer Toolkit is a single README that sorts open-source LLM libraries by the job they do, one row per library: name, a one-line description (often the project's own tagline) and a link. Denis Bakanov's LinkedIn post (2026-11-09) is a tour of it. Treat it as a map of the stack's layers and a shortlist per layer; it has no stars, benchmarks, maintenance status or licence column, so it cannot rank anything.

What it is

The README advertises "120+" libraries; the table rows actually number 177 across 15 sections (counted 2026-11-15; a few names repeat across sections). Application development is split into sub-layers: frameworks, data preparation, multi-API access, routers, memory, interface, low code and cache.

SectionRowsExamples
Application development27LangChain, LlamaIndex, Haystack, LiteLLM, mem0
Agents27LangGraph, CrewAI, Pydantic AI, OpenAI Agents SDK, smolagents, Browser Use
Training and fine-tuning18Unsloth, TRL, Axolotl, torchtune
Evaluation18Ragas, DeepEval, OpenAI Evals
Others13mergekit, Vanna, pandas-ai, MarkLLM
RAG12Chonkie, rerankers, sqlite-vec
Monitoring11Phoenix, Helicone, MLflow
Safety and security10NeMo Guardrails, Guardrails, LLM Guard, Garak
Prompts9DSPy, LLMLingua, Promptify
Inference9vLLM, llama.cpp, Ollama, TensorRT-LLM
Data extraction8Crawl4AI, Docling, LlamaParse
Structured outputs6Instructor, Outlines, XGrammar
Data generation4DataDreamer, Promptwright, EasyInstruct
Embedding models3Sentence-Transformers, Model2Vec, Text Embeddings Inference
Serving2LangCorn, LitServe

The repo is also a funnel: its header promotes the author's interview-questions book (with a discount code) and newsletter. That does not make the list wrong, but it explains the breadth-over-judgment shape.

How to use it

Use it to see what layers exist and to name two or three candidates per layer; then choose with your own criteria. Bakanov says as much: the notes are short, there are no stars or benchmarks per library, "so the picking is still your job."

His sharper point is that the right pick is sometimes no framework. His teams used LangGraph in two products, but wrote a sales agent's pipeline directly on the OpenAI SDK so the model could not skip or loop a step. That is the workflow-over-agent argument (R015, R079): when the steps are known, put control flow in code and let the model fill slots, instead of handing the loop to a framework that lets the model choose.

A practical selection pass per layer:

  1. Decide whether the layer needs a library at all (a 50-line loop, a retry wrapper or a JSON schema check is often enough).

  2. Shortlist from the list, then check each candidate's last release, open-issue trend, licence and who maintains it.

  3. Prefer libraries that sit below your control flow (inference, structured outputs, tracing) over ones that own it (agent frameworks), because the latter are the hardest to remove.

  4. Run one real task from your product through the top two before adopting either.

Gaps and staleness

The list is organised around the 2024 LLM-app stack (framework, RAG, eval, monitoring) and is thin on the layers that agent systems added since. As of 2026-11-15 it has none of these:

Some rows are dated. OpenAI's Swarm is listed beside the Agents SDK that replaced it, and AutoGen appears without noting Microsoft's move to a successor framework. Without a last-release column, a reader cannot tell a maintained project from an abandoned one; check each repo before adopting it.

Review checklist

When a design doc says "we'll use library X from the toolkit list":

Sources

Connections

See R081 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.