FitLLM Engine

FitLLM is an open-source, zero-dependency engine that checks whether a local LLM fits on a GPU or Apple Silicon Mac using architecture-aware memory math.

Live: https://fitllm.run · Bilingual · Free · No ads · No login

Open engine: fitllm-engine (MIT · npm fitllm-engine · npx fitllm)

Zero dependencies. One readable file: engine.js. Conformance-vector tested. MIT.

Quick start

npx fitllm "Gemma 4 12b" --gpu "RTX 4090"     # one line, exit 0 fits / 1 won't — run before you download
npx fitllm --top --gpu 4090                    # what can this hardware run?
npx fitllm --top --detect --json               # detect every supported local GPU; agent-ready JSON
npx fitllm Qwen/Qwen3-32B --detect --json      # public Hugging Face ID or URL; unsupported configs fail closed
npx fitllm "Gemma 4 12b" --detect --json --why # include architecture, evidence, assumptions, and exact memory inputs
npm install fitllm-engine                      # use the same engine as a library (see Usage)

Remote MCP server

Connect any Streamable HTTP MCP client to https://fitllm.run/api/mcp:

{
  "mcpServers": {
    "fitllm": {
      "url": "https://fitllm.run/api/mcp"
    }
  }
}

Tools

• check_llm_fit — check one model against a GPU, multi-GPU rig, or Mac and return the verdict, memory breakdown, and a fix when it does not fit.
• what_fits_on_hardware — rank the supported local models that fit the given GPU, multi-GPU rig, or Mac.
• list_supported — list the built-in model and hardware names accepted by the fit checker.

The server is read-only, stateless, and requires no authentication.

npx fitllm "GLM-4.7-Flash" --gpu 4090     # ✓ FITS — 21.9/24 GB, free 2.1 GB
npx fitllm "gpt-oss-120b" --mac 64        # ✗ WON'T FIT → what to change to make it fit
npx fitllm "Qwen 3.6 35B" --gpu "5090 + 3090"   # multi-GPU rig — VRAM pools (56GB), even mixed cards
npx fitllm --top --detect --json          # detect hardware + what fits, for people or agents
npx fitllm "Gemma 4 12b" --detect         # one verdict on this machine

--detect reads all nvidia-smi adapters, uses Apple Silicon unified memory on arm64 macOS, and can resolve an exact catalog GPU name through Windows/WSL PowerShell. It never uses Win32_VideoController.AdapterRAM, serials, PNP IDs, or full environment dumps. Intel-only, ambiguous, and unsupported adapters stop with exit 2 instead of borrowing a nearby GPU's memory.

Why a CLI? The "will it run?" question is born in the terminal — one line before ollama pull. No install, no tab-switching, and it reads your actual hardware with --detect instead of asking you to know your VRAM. Exit code 0/1 makes it a pre-download guard:

# in your model-pull script — stop BEFORE the 40 GB download:
npx fitllm "gpt-oss-120b" --detect || { echo "won't fit — aborting pull"; exit 1; }

Put the guard in the download path

The CLI exit contract composes directly with model runtimes. The download or launch runs only after a FITS or TIGHT verdict; invalid inputs stop with exit 2.

npx fitllm "Gemma 4 12b" --detect && ollama pull gemma4:12b
npx fitllm "Llama-3.1-8B-Instruct" --detect && llama-cli -m ./Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf

For CI, use the repository's composite Action. It needs no secret and returns the full --json --why result as steps.preflight.outputs.result:

permissions:
  contents: read
steps:
  - name: Check model memory before download
    id: preflight
    uses: click6067-ship-it/fitllm-engine@v2.15.0
    with:
      model: Gemma 4 12b
      gpu: RTX 4090
      ctx: '8192'

Set exactly one of gpu or mac. The Action preserves the CLI contract: exit 0 means fits/tight, 1 means it will not fit, and 2 means the request is invalid. Inputs cross the GitHub expression boundary through environment variables and are passed to Node as a Bash argument array.

Agents can collect a typed measurement without uploading anything:

npx fitllm measure "Qwen 3.6 27B" --detect --measured 15.3 --kind system_total_peak --unit GiB --runtime "llama.cpp b6400"

The command validates the conditions and prints a candidate JSON object plus a prefilled GitHub issue URL. Submission remains a human action. Public Hugging Face IDs (org/model) are fetched with a bounded config/index reader and accepted only when parseHfConfig() supports the architecture.

This is the open calculation core of FitLLM. The math is open so you can audit it.

Ask an LLM "does Qwen 3.6 fit my GPU?" and it pattern-matches to an architecture from its training cutoff — and usually says no. Catalog-based calculators lag new releases. The CLI, API, and MCP use a curated catalog pinned to official configs. The web calculator can additionally inspect a pasted Hugging Face ID's official config.json live, so supported architectures work on day-one releases — including the hybrid / sliding-window / MoE structures that naive formulas get wrong.

Covers Apple Silicon unified memory (M1–M6, Pro/Max/Ultra — up to the 512GB Mac Studio), NVIDIA GPUs (RTX 20/30/40/50, workstation RTX 6000 Ada / RTX PRO 6000, datacenter A100/H100/H200/B200), AMD Radeon (RX 7000/9000, PRO W7900) and multi-GPU presets (2×3090, 2×4090, 4×3090) — with GGUF Q-tier weight quantization kept separate from KV-cache quantization. Hardware entries carry their source URLs per-value in engine.js; new entries require ≥2 independent sources (CONTRIBUTING).

Why most LLM memory calculators are wrong

Almost every "can I run this LLM?" calculator estimates the KV cache with the textbook formula:

KV ≈ 2 × num_layers × num_kv_heads × head_dim × context_length × bytes

That assumes every layer keeps a full-context KV cache with one uniform head shape. True for Llama-1/2 — wrong for most 2025–2026 models:

An 11× error flips the verdict: a naive calculator says Gemma 4 31B won't fit in 64 GB at long context, when it fits comfortably.

The five things they ignore

1. Sliding-window attention (Gemma 2/3/4, gpt-oss): most layers only keep the last N tokens, so their KV stops growing. Only the global layers scale with full context.
2. Hybrid / linear attention (Qwen 3.6 / 3.8, many 2026 models): linear-attention layers use a fixed-size recurrent state, not a growing KV cache. That state is modeled too, as its own component (linearState) — it is a constant per sequence, so it never inflates the context curve.
3. MLA — Multi-head Latent Attention (GLM-5.2, GLM-5.3, GLM-4.7-Flash, DeepSeek family): the cache is a single low-rank latent (kv_lora_rank + RoPE dims) shared across all heads — per-head "2 × heads × head_dim" formulas over-count by an order of magnitude. Verified against the DeepSeek-V2 paper (arXiv:2405.04434) and the official DeepSeek-V3 inference code.
4. Heterogeneous head dims + MoE: global layers can use a different head_dim (Gemma 4: 512 vs 256). MoE keeps every expert in memory while activating only a few per token.
5. PLE — Per-Layer Embeddings (Gemma 4 e2b/e4b): the pinned llama.cpp/GGUF path assigns the per_layer_token_embd input-layer tensor to CPU/host buffers instead of accelerator memory, so only the non-PLE weights are counted against VRAM for the verified Gemma 4 e2b/e4b entries. Counting all 5.1B params against a GPU over-predicts e2b's resident weights by ~1.9× and flips small-card verdicts. This deduction is conditional and rests on that placement alone: the host memory the tensor needs is not budgeted by the GPU verdict, a runtime that loads PLE tensors onto the accelerator (vLLM, for example) invalidates the estimate, and unverified families keep their full weights resident. On Apple Silicon unified memory the full weights stay counted. The exact premise and its pinned sources are listed under Structural premises; a direct measurement on a Gemma 4 GGUF is welcome in issue #7.

This engine models each layer type separately, verified against official HuggingFace config.json files.

What it computes

Total = Parameters (quantization-adjusted)
      + KV cache (per layer kind: sliding / global / linear / dense)
      + Runtime overhead (quant metadata + KV block padding + activations + fixed)
      + macOS base (Apple Silicon unified memory)