DavidGadyan

LLM Cheatsheet Mind Map: Frontier vs Open-Weight Models, VRAM, Licences, Trading and Restricted Content

Frontier APIs against open weights: dense and mixture-of-experts architectures, the current open models of every major lab, how much GPU memory each needs, what their licences allow, and where they fit in trading and restricted-content work.

  1. How you get the model

    API, weights, source

    What it is
    Four ways a model reaches you: a proprietary API (weights never leave the vendor), open weights you download and run, a fully open-source release (weights plus code and data under an open licence), and gated or research-only checkpoints. Open-weight does not mean open-source: most "open" models publish weights only, under licences that range from Apache 2.0 to custom revenue-gated terms.
    When to use it
    Decide the access model first: it fixes where the data goes, who pays for GPUs and what the licence lets you ship.
    1. Frontier API

      Strongest, vendor-hosted

      What it is
      GPT, Claude, Gemini, Grok and Muse Wonder are reachable only through their vendors' APIs or cloud marketplaces. You get the strongest models, no hardware and per-token billing; data leaves your network under a processing agreement, and regional processing or zero retention are contract settings.
      When to use it
      Default for hard reasoning, long agentic coding and anything without a rule that forbids sending data out.
    2. Open weights

      Download and self-host

      What it is
      The trained weights are published (usually on Hugging Face) so you can run, quantise and fine-tune them on your own GPUs. Training data and code normally stay closed, and each checkpoint carries its own licence.
      When to use it
      Data that must stay on your hardware, fine-tuning, predictable high volume, or offline and air-gapped sites.
    3. Open source

      Weights, code and data

      What it is
      A release you can fully reproduce: weights, training code and enough data documentation under an open licence. Rare at frontier scale; an Apache 2.0 or MIT licence on the weights alone makes a model permissive, not reproducible.
      When to use it
      Research, audits and regulated settings that must explain how the model was built.
    4. Gated and custom terms

      Accept before download

      What it is
      Some repositories require you to accept a licence or share contact details before downloading (Llama 4, Muse Spark Open), and some licences add revenue thresholds, attribution or reseller clauses on top of the weights.
      When to use it
      Read the licence file in the repository, not the family's blog post: terms change between checkpoints of the same family.
    5. Private deployment

      VPC, on-prem, air-gapped

      What it is
      Open weights served by vLLM, SGLang or llama.cpp inside your VPC, on rented GPUs or on your own servers; prompts, embeddings and logs never leave the network. Both vLLM and SGLang expose OpenAI-compatible endpoints, so switching from an API is a base-URL change.
      When to use it
      Client data under NDA, health or financial records, proprietary trading logic.
    6. Hybrid routing

      API and self-hosted

      What it is
      A router sends each request to the cheapest model that does it well: sensitive fields to the self-hosted model, hard reasoning to a frontier API, fixed-option decisions to a decision model, with a fallback when a provider is down or a model retires.
      When to use it
      Most production systems: neither "all API" nor "all local".
  2. Architectures

    Dense, MoE, hybrid

    What it is
    How a model spends its parameters per token decides its memory, speed and cost. The two axes to keep apart: total parameters (what must sit in memory) and active parameters (what is computed per token).
    When to use it
    Read both numbers on every model card before buying or renting a GPU.
    1. Dense

      Every weight, every token

      What it is
      Every parameter takes part in every token: a 27B dense model keeps 27B weights in memory and computes all of them per token. Simple to serve, predictable latency, the easiest to fine-tune. Examples: Qwen3.8-27B, Gemma 4 31B, Muse Glimmer, Granite 4.1, Ministral 3.
      When to use it
      Single-GPU deployments and fine-tuning on your own data.
    2. Mixture of experts

      A router picks experts

      What it is
      A router sends each token to a few expert networks out of many, so compute follows the active parameters while memory follows the total. Qwen3-30B-A3B routes to 8 of 128 experts (30.5B total, 3.3B active); Kimi K3 to 16 of 896 plus 2 shared (reported).
      When to use it
      Frontier quality at mid-size compute, when you can afford the memory for all experts.
      Key figures
      • "MoE reduces computation, not memory": all experts must be loaded even though few are active per token.Source: Hugging Face blog, Mixture of Experts Explained
    3. Hybrid Mamba and attention

      Cheaper long context

      What it is
      State-space layers (Mamba-2) mixed with a few attention layers and MoE blocks: the KV cache grows only in the attention layers, so long contexts cost less memory. NVIDIA Nemotron 3 uses it; IBM Granite 4.0 did, while Granite 4.1 returned to dense.
      When to use it
      Long-running agents and long documents on limited GPU memory.
    4. Reasoning models

      Think, then answer

      What it is
      The model spends extra tokens working through the problem before the answer; gpt-oss exposes low, medium and high reasoning effort. More latency and more KV cache per request, better results on maths, code and planning.
      When to use it
      Open problems; skip it for grounded summarising, where it adds cost without accuracy.
      AI angle
      Budget the reasoning tokens in the KV cache, not just the answer.
    5. Microsoft Decision-1

      Scores fixed options

      What it is
      A decision model post-trained by Microsoft from Qwen3.5-9B: you pass a text and a fixed set of options and it returns a calibrated probability per option instead of generating text. Available in Microsoft Foundry; no downloadable weights were announced.
      When to use it
      Intent classification, tool and model routing, grading answers, continue-retry-escalate steps in agents.
      Key figures
      • $0.042 per million input tokens, output free; 32,768-token input limit.Source: Microsoft Learn, Foundry Models sold by Azure
    6. Classic classifiers

      Trees and encoders

      What it is
      Gradient-boosted trees (LightGBM, XGBoost, CatBoost) for tabular and numeric data, encoder transformers (ModernBERT, DeBERTa) for text labels, TabPFN for small tabular sets without training. They need a fraction of an LLM's memory and often run on CPU.
      When to use it
      Fraud, risk, price features and any numeric prediction: an LLM is not the right tool there.
  3. Frontier APIs

    No downloadable weights

    What it is
    The strongest proprietary models of each lab, as listed on 26 October 2026. Six of the eight language-model flagships publish a context window of one million tokens or more.
    When to use it
    Benchmark one of these against your best open candidate on your own evaluation set before deciding to self-host.
    1. GPT-6 Astra

      OpenAI flagship

      What it is
      OpenAI's flagship, released 3 September 2026, with a 1,050,000-token window. Cheaper tiers: GPT-6.1 Terra and GPT-6 Sol; OpenAI also publishes open weights separately as gpt-oss.
      When to use it
      General agents, tool calling, financial analysis.
      Key figures
      • $10 input / $50 output per million tokens.Source: AI Board report, 26 Oct 2026
    2. Claude Fable 5.1

      Anthropic flagship

      What it is
      Anthropic's flagship, released 1 September 2026, with a 1,000,000-token window; Claude Opus 5.5 and Sonnet 5.5 are the tiers most workloads start on. No open weights.
      When to use it
      Coding agents on large codebases, long documents, complex workflows.
      Key figures
      • $10 input / $50 output per million tokens.Source: AI Board report, 26 Oct 2026
    3. Gemini 4 Argon

      Google flagship

      What it is
      Google's flagship reasoning model, generally available on 13 October 2026, with a 1,048,576-token window. Google's open-weight line is Gemma.
      When to use it
      Video, audio and image understanding, million-token documents.
      Key figures
      • $2 input / $10 output per million tokens (introductory).Source: AI Board report, 26 Oct 2026
    4. Grok 4.7

      xAI flagship

      What it is
      xAI's flagship for coding, agents and knowledge work, released 21 September 2026, with a 500,000-token window and real-time access to the X ecosystem.
      When to use it
      Social sentiment and market chatter in near real time.
      Key figures
      • $2 input / $6 output per million tokens; the rate doubles above 200K tokens.Source: AI Board report, 26 Oct 2026
    5. Muse Wonder

      Meta flagship

      What it is
      Meta's most capable Muse model, released 12 October 2026 on the Meta Model API without open weights. Three days later Meta published Muse Spark Open with downloadable weights.
      When to use it
      Teams already on Meta's API; for self-hosting use Muse Spark Open or Llama 4.
      Key figures
      • $4 input / $20 output per million tokens, 1,000,000-token window.Source: AI Board report, 26 Oct 2026
  4. Open-weight models

    Every major lab

    What it is
    The downloadable models of each lab in November 2026, with total and active parameters, context and licence. Sizes run from phone-class Gemma 4 E2B to Kimi K3 at 2.8 trillion parameters.
    When to use it
    Shortlist by licence first, then by what fits your memory, then evaluate quality.
    1. Meta: Llama and Muse

      Four open lines

      What it is
      Llama 4 (April 2025) remains downloadable under the Llama 4 Community License; in 2026 Meta added Muse Glimmer (dense, Apache 2.0) and Muse Spark Open (frontier MoE, Muse Model License).
      When to use it
      The ecosystem most self-hosted deployments already run.
      1. Llama 4 Scout

        109B / 17B active MoE

        What it is
        Mixture of experts with 16 experts, 109B total and 17B active parameters and a 10M-token context window. Multimodal (text and image input).
        When to use it
        Very long documents and codebases on a single data-centre GPU.
        Key figures
        • Fits a single H100 GPU with Int4 quantisation.Source: Meta, Llama 4 announcement
      2. Llama 4 Maverick

        400B / 17B active MoE

        What it is
        Mixture of experts with 128 experts, 400B total and 17B active parameters, a 1M-token window and image input.
        When to use it
        General assistant and multimodal RAG when an 8-GPU host is available.
        Key figures
        • Runs on a single H100 host (8 GPUs).Source: Meta, Llama 4 announcement
      3. Muse Spark Open

        816B / 42B active MoE

        What it is
        Released 15 October 2026 on Hugging Face, Google Cloud and Microsoft Foundry: a mixture of experts with 256 experts and 8 active per token, 816B total and 42B active parameters, up to a 1M-token window (reported), under the custom Muse Model License.
        When to use it
        Frontier-class coding and agents on your own cluster, if the licence fits your business model.
        Key figures
        • Reported minimum of about 700 GB of GPU memory in NVFP4, about 2 TB in BF16.Source: The New Stack, Muse Spark Open
      4. Muse Glimmer

        30B dense, Apache 2.0

        What it is
        A dense model of about 30B parameters released in August 2026 for local agents, function calling and coding, under Apache 2.0: the most permissive licence Meta has used for a strong model.
        When to use it
        A commercial product on a 24–32 GB GPU without Llama's licence conditions.
    2. OpenAI gpt-oss

      Two MoE sizes, Apache 2.0

      What it is
      OpenAI's open-weight reasoning models, released 5 August 2025 under Apache 2.0 plus the gpt-oss usage policy, with configurable reasoning effort, tool use and a 128K window. Weights ship natively in MXFP4.
      When to use it
      Reasoning on your own hardware with the simplest licence of the frontier labs.
      1. gpt-oss-120b

        117B / 5.1B active

        What it is
        Mixture of experts with 117B total and 5.1B active parameters; the very low active count makes it fast once loaded.
        When to use it
        Production reasoning on one 80 GB or 96 GB GPU.
        In code
        vllm serve openai/gpt-oss-120b
        Key figures
        • Fits a single 80 GB GPU thanks to MXFP4.Source: OpenAI, gpt-oss model card
      2. gpt-oss-20b

        21B / 3.6B active

        What it is
        Mixture of experts with 21B total and 3.6B active parameters, aimed at local and on-device inference.
        When to use it
        The strongest reasoning model for a 16 GB laptop GPU.
        In code
        ollama run gpt-oss:20b
        Key figures
        • Runs within 16 GB of memory.Source: OpenAI, gpt-oss model card
    3. Google Gemma 4

      E2B to 31B, Apache 2.0

      What it is
      Five sizes: E2B (2.3B effective) and E4B (4.5B effective) for phones and edge, 12B dense, 26B A4B (MoE, 128 experts plus 1 shared, 8 active, 25.2B total / 3.8B active) and 31B dense, with 256K context on the larger three. The Gemma 4 model card states Apache 2.0, unlike earlier Gemma terms.
      When to use it
      Multimodal apps from the phone to a workstation GPU under one permissive family.
      Key figures
      • 12B, 26B A4B and 31B target consumer GPUs and workstations; E2B and E4B target phones and edge devices.Source: Gemma 4 model card
    4. Mistral AI

      European, mixed licences

      What it is
      Mistral publishes open weights under different licences per model: Apache 2.0 for Large 3, Small 4 and Ministral 3, a modified MIT for Medium 3.5, and the Mistral Ecosystem Open-Weight License for Large 4.
      When to use it
      EU sovereignty: the same model self-hosted or on Mistral's European endpoints.
      1. Mistral Large 3

        675B / 41B, Apache 2.0

        What it is
        Released 2 December 2025: a mixture of experts with 675B total and 41B active parameters, a 256K window and Apache 2.0, the largest permissively licensed MoE of its generation.
        When to use it
        A frontier-scale model you can resell or fine-tune without revenue clauses.
        Key figures
        • Estimate: about 675 GB of weights in FP8, so an 8 × H200 node (1,128 GB) holds it with room for KV cache.Source: Computed from the Mistral 3 announcement
      2. Mistral Large 4

        About 1T MoE, preview

        What it is
        Public preview since 6 October 2026, weights on Hugging Face since 29 October (reported): a multimodal mixture of experts of about one trillion parameters under the Mistral Ecosystem Open-Weight License, which lets you use and fine-tune it but requires a commercial licence to resell it as a model service.
        When to use it
        Internal frontier-scale workloads; not for a model-as-a-service business without a deal.
      3. Ministral 3

        3B, 8B, 14B dense

        What it is
        Dense vision-capable models of 3B, 8B and 14B parameters with a 256K window, Apache 2.0, released 2 December 2025.
        When to use it
        Edge devices, laptops and cheap fine-tunes.
    5. Alibaba Qwen

      Widest open line

      What it is
      Sizes from 0.6B to 2.4T with coder, vision, omni, embedding, reranker and guard variants. Qwen3 and Qwen3.8-27B are Apache 2.0; the Qwen3.8 Max-class weights use a custom licence.
      When to use it
      Private RAG, multilingual assistants, local coding agents, fine-tuned domain models.
      1. Qwen3.8-27B

        Dense, 262K, Apache 2.0

        What it is
        Dense multimodal model of 27B language parameters (August 2026), 262,144 tokens native context extensible to about one million, Apache 2.0. The card recommends SGLang or vLLM for serving.
        When to use it
        The most practical open model for one 24–48 GB GPU.
        In code
        vllm serve Qwen/Qwen3.8-27B --max-model-len 65536
        Key figures
        • Estimate: 27B × 4.8 bits ÷ 8 ≈ 16 GB of weights at Q4_K_M, 54 GB at BF16.Source: Computed from the Qwen3.8-27B model card
      2. Qwen3.8-2.4T-A95B

        Max-class open MoE

        What it is
        The open Max-class weights: 2.4T total and 95B active parameters, 512 experts with 10 routed plus 1 shared per token, 262,144 tokens native extensible to 1,010,000, under the custom "qwen3.8-max" licence.
        When to use it
        Data-centre clusters only; check the model-service clause before building a product on it.
        Key figures
        • Model-as-a-service providers above $50M a year reportedly need a separate licence.Source: SCMP, Qwen3.8-Max licence
      3. Qwen3-30B-A3B

        30.5B / 3.3B MoE

        What it is
        Mixture of experts from April 2025: 30.5B total, 3.3B active, 128 experts with 8 active, Apache 2.0. Fast like a 3B model, memory like a 30B one.
        When to use it
        Snappy local agents on a 24 GB GPU at Q4.
      4. Qwen3-Coder-480B-A35B

        Agentic coding MoE

        What it is
        Coding mixture of experts from July 2025: 480B total, 35B active, 160 experts with 8 active, 256K native context, Apache 2.0.
        When to use it
        A coding agent when the source code may not leave your network.
    6. DeepSeek

      Frontier MoE, MIT

      What it is
      DeepSeek publishes its frontier models as MIT-licensed mixtures of experts with sparse attention and a 1M-token window, the most permissive licence at that scale.
      When to use it
      Cost-efficient reasoning, maths and code, self-hosted or through its low-priced API.
      1. DeepSeek-V4-Pro

        1.6T / 49B, MIT

        What it is
        Released April 2026: 1.6T total and 49B active parameters, 1M-token window, MIT; experts stored in FP4 and the rest in FP8 (reported).
        When to use it
        Data-centre scale only: plan for a terabyte of GPU memory.
        Key figures
        • $1.32 input / $3.96 output per million tokens on DeepSeek's API at peak, half off-peak.Source: AI Board report, 26 Oct 2026
      2. DeepSeek-V4-Flash

        284B / 13B, MIT

        What it is
        The smaller V4: 284B total, 13B active, 1M window, MIT. V4.1-Flash (10 September 2026) followed with native image input.
        When to use it
        Private mid-tier workhorse on a 4–8 GPU server.
        Key figures
        • Estimate: 284B × 4 bits ÷ 8 ≈ 142 GB of weights at 4-bit.Source: Computed from the DeepSeek-V4-Flash model card
      3. DeepSeek-V3.2

        685B MoE, MIT

        What it is
        The December 2025 generation: about 685B total and 37B active parameters, 128K window, MIT, the first with DeepSeek Sparse Attention.
        When to use it
        Still widely quantised and served; a known quantity for existing clusters.
    7. Moonshot Kimi K3

      2.8T / ~104B MoE

      What it is
      Weights published in July 2026 (reported): 2.8T total and about 104B active parameters, 896 experts with 16 routed plus 2 shared, a 1,048,576-token window and image input, under the custom Kimi K3 License. Its predecessor Kimi K2 (1T / 32B) used a modified MIT.
      When to use it
      Long-horizon agentic coding on a multi-node cluster.
    8. Z.ai GLM-5.x

      Coding and reasoning MoE

      What it is
      GLM-5 (February 2026, 744B / 40B, MIT) and GLM-5.3-Flash (August 2026, about 320B / 18B multimodal MoE, MIT, reported). The flagship GLM-5.3 moved to a custom "glm-5.3" licence, so the family no longer shares one licence.
      When to use it
      Chinese and English coding agents; pick the Flash for MIT terms.
    9. NVIDIA Nemotron 3

      Hybrid Mamba MoE

      What it is
      Hybrid Mamba-2, attention and MoE models under the OpenMDW License 1.1, which allows commercial use: Nemotron 3.5 Lightning (30B / 3B, NVFP4 variant) for sub-agents and tool calling, and Nemotron 3 Ultra (550B / 55B) for data-centre reasoning.
      When to use it
      Long-running agents where KV-cache memory is the bottleneck.
    10. IBM Granite 4.1

      3B, 8B, 30B, Apache 2.0

      What it is
      Dense base and instruct models of 3B, 8B and 30B parameters (April 2026), Apache 2.0, designed for enterprise RAG, extraction and classification; IBM also publishes the Granite Guardian safety models.
      When to use it
      Document pipelines where licence clarity matters more than leaderboard rank.
  5. VRAM sizing

    Will it fit?

    What it is
    GPU memory must hold the weights, the KV cache of every live request, activations and the runtime's own buffers. The weights are the part you can calculate exactly; the rest grows with context length and concurrency.
    When to use it
    Size the GPU before choosing the model, not after the out-of-memory error.
    1. Weight memory formula

      Params × bits ÷ 8

      What it is
      Weight memory in bytes = total parameters × bits per weight ÷ 8. A 7B model needs 14 GB at 16-bit, 7 GB at 8-bit, 3.5 GB at 4-bit; a 70B model 140, 70 and 35 GB. Use total parameters, never active ones, and add 10–30 % for KV cache and runtime as a starting margin.
      When to use it
      The first check on any model card.
      In code
      weights_gb = params_billion * bits_per_weight / 8
      # 27B at Q4_K_M: 27 * 4.8 / 8 = 16.2 GB
    2. Quantisation formats

      Fewer bits per weight

      What it is
      Storing weights in fewer bits trades a little quality for a lot of memory. GGUF (llama.cpp, Ollama) is the laptop format; AWQ, GPTQ and FP8 are common on vLLM; MXFP4 and NVFP4 are the 4-bit floats of the newest GPUs.
      When to use it
      Q4 to fit a bigger model on the same GPU; 8-bit or 16-bit when accuracy on numbers and code matters.
      1. BF16, FP16 and FP8

        16 and 8 bits

        What it is
        BF16 and FP16 store 2 bytes per weight, the quality reference and the format most checkpoints ship in. FP8 (formats E4M3 and E5M2) halves that with hardware support on Hopper and Blackwell GPUs.
        When to use it
        Fine-tuning (BF16) and data-centre serving (FP8).
        Key figures
        • FP8 defines two encodings: E4M3 for weights and activations, E5M2 for gradients.Source: FP8 Formats for Deep Learning (arXiv 2209.05433)
      2. GGUF Q8_0 and Q6_K

        Near-lossless

        What it is
        llama.cpp's 8-bit and 6-bit formats: Q8_0 uses about 8.5 bits per weight and Q6_K about 6.56, both very close to 16-bit quality.
        When to use it
        Local models where you have the memory and want the quality.
        Key figures
        • Q8_0 ≈ 8.5 bits per weight, Q6_K ≈ 6.56.Source: GitHub, llama.cpp quantize README
      3. GGUF Q4_K_M

        The local default

        What it is
        The 4-bit K-quant most local guides start from, at roughly 4.8 bits per weight: about a third of the 16-bit size with a small quality loss.
        When to use it
        Fitting a 14B model on 16 GB or a 27–32B model on 24 GB.
        In code
        llama-server -hf <repo>:Q4_K_M -c 16384 -ngl 99
      4. MXFP4 and NVFP4

        Hardware 4-bit floats

        What it is
        Both store E2M1 4-bit floats with shared scales: MXFP4 (Open Compute Project) shares one 8-bit power-of-two scale per 32 values; NVFP4 (NVIDIA Blackwell) one FP8 scale per 16 values plus a per-tensor FP32 scale. gpt-oss ships in MXFP4.
        When to use it
        Large MoE models on Blackwell GPUs at 4-bit with native speed.
        Key figures
        • NVFP4: blocks of 16 values with an FP8 E4M3 scale.Source: NVIDIA blog, Introducing NVFP4
    3. KV cache

      Grows with context

      What it is
      Every token in every live request keeps a key and a value per layer: KV bytes = 2 × layers × KV heads × head dimension × bytes per value × tokens × batch. Illustration: a model with 48 layers, 8 KV heads and head size 128 in FP16 needs about 6.4 GB for one 32K-token request.
      When to use it
      Why a model that "fits" fails at long context or ten concurrent users; cap max-model-len or quantise the cache to FP8.
      In code
      kv_bytes = 2 * layers * kv_heads * head_dim * bytes * tokens * batch
      vllm serve <model> --kv-cache-dtype fp8 --max-model-len 32768
      Key figures
      • KV cache size = 2 × layers × KV heads × head dimension × precision bytes × sequence length × batch.Source: NVIDIA blog, Mastering LLM Techniques: Inference Optimization
    4. The MoE memory trap

      Total, not active

      What it is
      A 284B-A13B model computes like a 13B but must load 284B parameters. Memory planning uses the total; only speed benefits from the active count. Exceptions are tricks, not free: expert offloading to CPU RAM costs PCIe bandwidth on every token.
      When to use it
      Read "A3B", "A22B" in a model name as speed, never as memory.
    5. Offloading and multi-GPU

      Split the model

      What it is
      When weights exceed one GPU: tensor parallelism splits each layer across GPUs in a node (fast with NVLink), pipeline parallelism splits layers across nodes, and llama.cpp can keep some layers in system RAM. Each split adds communication; PCIe and network bandwidth become the bottleneck.
      When to use it
      Tensor parallel inside one 8-GPU node first; offload to CPU only for experiments.
      In code
      vllm serve <model> --tensor-parallel-size 8
      llama-server -m model.gguf -ngl 30   # 30 layers on GPU, rest in RAM
    6. Fine-tuning memory

      Full, LoRA, QLoRA

      What it is
      Full fine-tuning with Adam in mixed precision needs about 16–18 bytes per parameter (weights, gradients, two optimiser states, an FP32 master copy) before activations, so a 7B model needs over 110 GB. LoRA trains small adapters on frozen weights; QLoRA keeps the frozen base in 4-bit.
      When to use it
      QLoRA on one GPU for domain adaptation; full fine-tuning only with a multi-GPU budget and a reason.
      Key figures
      • Mixed-precision Adam training: about 18 bytes per parameter plus activations.Source: Hugging Face Transformers, Model training anatomy
      • QLoRA fine-tunes a 65B model on a single 48 GB GPU.Source: QLoRA paper (arXiv 2305.14314)
    7. Serving engines

      vLLM, SGLang, llama.cpp

      What it is
      vLLM and SGLang serve many users at once with paged KV cache and continuous batching on data-centre GPUs; llama.cpp and Ollama run GGUF models on laptops, Macs and CPUs. All four expose an OpenAI-compatible API.
      When to use it
      Ollama or llama.cpp to prototype, vLLM or SGLang for production traffic.
      In code
      ollama run qwen3:30b-a3b
      python -m sglang.launch_server --model-path <model> --tp 4
    8. GPU memory tiers

      16 GB to 8-GPU nodes

      What it is
      What fits at each memory size, for inference with a modest context. Bandwidth decides tokens per second; capacity decides what loads at all.
      When to use it
      Pick the tier from the largest model you must serve, plus its KV cache.
      1. 16 GB laptop GPU

        Up to 14B at Q4

        What it is
        7–8B at 8-bit or 16-bit, 14B at Q4 with a modest context, gpt-oss-20b, Gemma 4 E4B and 12B at Q4, plus every classic classifier. 32B needs heavy quantisation or CPU offloading; 70B does not fit.
        When to use it
        Development, private RAG prototypes, local classification and routing.
      2. 24–32 GB desktop

        RTX 4090, RTX 5090

        What it is
        RTX 4090 (24 GB, 1,008 GB/s) and RTX 5090 (32 GB, 1,792 GB/s): 14B at 16-bit, Qwen3.8-27B, Gemma 4 31B, Muse Glimmer and Qwen3-30B-A3B at Q4 with room for context.
        When to use it
        The serious local workstation: agents, RAG and QLoRA on small models.
        Key figures
        • RTX 5090: 32 GB GDDR7, 1,792 GB/s.Source: NVIDIA RTX 5090 spec page
      3. 48–96 GB workstation

        RTX PRO 6000 Blackwell

        What it is
        48 GB cards (RTX 6000 Ada, L40S) take 27–32B at 8-bit or a 70B at Q4. The RTX PRO 6000 Blackwell (96 GB GDDR7, 1,792 GB/s) holds gpt-oss-120b or a 70B at 8-bit on one card.
        When to use it
        Small-team production inference and QLoRA of 30–70B models.
        Key figures
        • RTX PRO 6000 Blackwell: 96 GB GDDR7 with ECC, 1,792 GB/s.Source: NVIDIA RTX PRO 6000 spec page
      4. 80–141 GB data-centre GPU

        H100, H200

        What it is
        H100 (80 GB HBM3, 3.35 TB/s) and H200 (141 GB HBM3e, 4.8 TB/s): gpt-oss-120b or Llama 4 Scout at Int4 on one GPU, 70B at 16-bit across two, high concurrency thanks to HBM bandwidth.
        When to use it
        Rented by the hour for production serving and fine-tuning.
        Key figures
        • H200: 141 GB HBM3e at 4.8 TB/s.Source: NVIDIA H200 spec page
      5. 180–288 GB accelerators

        B200, B300, MI355X

        What it is
        NVIDIA B200 (180 GB HBM3e in HGX systems) and B300 (288 GB), AMD MI300X (192 GB, 5.3 TB/s) and MI355X (288 GB, 8 TB/s): a 284B MoE at 4-bit fits on one or two cards, and Blackwell adds native NVFP4.
        When to use it
        Large MoE models with fewer GPUs and less cross-GPU traffic.
        Key figures
        • AMD Instinct MI355X: 288 GB HBM3E, 8 TB/s.Source: AMD Instinct MI350 series spec page
      6. 8-GPU nodes and clusters

        1 TB and beyond

        What it is
        8 × H200 gives 1,128 GB, 8 × B300 about 2.3 TB. Mistral Large 3 at FP8, Muse Spark Open at NVFP4 and DeepSeek-V4-Pro fit one node; Qwen3.8-2.4T and Kimi K3 need multiple nodes or aggressive quantisation.
        When to use it
        Frontier-scale open models for a company that must not use an API.
      7. Unified memory

        DGX Spark, Mac Studio

        What it is
        NVIDIA DGX Spark (128 GB LPDDR5x at 273 GB/s) and Mac Studio with M3 Ultra (up to 512 GB at 819 GB/s) share memory between CPU and GPU: huge capacity for the price, but several times less bandwidth than HBM, so large models load yet generate slowly.
        When to use it
        Running a 100–400B MoE on a desk for evaluation, one user at a time.
        Key figures
        • Mac Studio M3 Ultra: up to 512 GB of unified memory, 819 GB/s.Source: Apple Mac Studio tech specs
  6. Licences

    What you may ship

    What it is
    The licence decides whether you can sell a product on the model, fine-tune and redistribute it, and under what name. Families do not share one licence: Mistral, Qwen and GLM each use several.
    When to use it
    Read the LICENSE file of the exact checkpoint, then the acceptable-use policy it links.
    1. Apache 2.0

      Any purpose, keep notices

      What it is
      Use, modify and distribute "for any purpose" with no field-of-use limits; keep the notices, state your changes, and accept the patent clause. Used by gpt-oss (plus its usage policy), Gemma 4, Qwen3, Qwen3.8-27B, Muse Glimmer, Mistral Large 3, Small 4, Ministral 3 and Granite 4.1.
      When to use it
      The first filter for SaaS, client deliverables and redistributed fine-tunes.
    2. MIT

      Without restriction

      What it is
      Use "without restriction" on condition that the copyright and permission notice travel with the weights. Used by DeepSeek V3.2 and V4, GLM-5 and GLM-5.3-Flash, and Xiaomi MiMo-V2-Flash.
      When to use it
      Frontier-scale MoE with no commercial strings.
    3. Modified MIT

      MIT plus a display clause

      What it is
      MIT with an extra condition for very large users. Kimi K2: products above 100M monthly active users or $20M monthly revenue must show "Kimi K2" in the interface. Mistral Medium 3.5 uses its own modified MIT with revenue exceptions.
      When to use it
      Fine for almost everyone; check the threshold if you might cross it.
    4. Llama 4 Community

      700M MAU, "Built with Llama"

      What it is
      Commercial use allowed, but: companies above 700 million monthly active users on the release date must ask Meta for a licence; products show "Built with Llama"; derived model names start with "Llama"; the Llama 4 Acceptable Use Policy applies; and the multimodal rights are not granted to individuals or companies domiciled in the EU (their end users are exempt).
      When to use it
      EU companies building on Llama 4 vision: check the multimodal clause first.
    5. Muse Model License

      Notice plus reseller clause

      What it is
      Meta's custom licence for Muse Spark Open: products carry a "Built with Muse Spark Open" notice, and model-as-a-service providers above a revenue threshold need a written agreement with Meta (reported; sources disagree on the amount).
      When to use it
      Internal use and most products are fine; an inference-API business needs legal review.
    6. Revenue-gated custom licences

      Free until you are big

      What it is
      The 2026 pattern: weights free to use, but large model-as-a-service resellers need a separate deal. Reported thresholds: Qwen3.8-Max weights above $50M a year, MiniMax M3 above $20M revenue (with "Built with MiniMax M3"), and Kimi K3 for large providers; Mistral Large 4 requires a commercial licence for any model-service reseller.
      When to use it
      Using the model inside your own product is usually fine; selling raw inference is what triggers the clause.
    7. OpenMDW 1.1

      NVIDIA Nemotron 3

      What it is
      The open model licence of Nemotron 3 Ultra and Nemotron 3.5; it allows commercial use. Earlier Nemotron 3 releases used the NVIDIA Open Model License, so check which one a checkpoint ships with.
      When to use it
      NVIDIA-stack deployments that want commercial terms without revenue gates.
    8. Licence checklist

      Six questions

      What it is
      1. May I use it commercially? 2. Is there a user or revenue threshold? 3. Must I show attribution or name derivatives a certain way? 4. May I redistribute modified weights? 5. Which acceptable-use policy is attached? 6. Are some rights withheld by region (Llama 4 multimodal in the EU)?
      When to use it
      Run it per checkpoint and keep the answers with the model version in your registry.
  7. Trading use cases

    Text in, features out

    What it is
    In trading, LLMs turn unstructured text (news, filings, calls, social posts) into structured features, help write and review strategy code, and assist research. Prices, signals and execution stay with numeric models and deterministic code.
    When to use it
    Put the LLM before the model, never in the order path.
    1. Why self-host for trading

      The strategy is the IP

      What it is
      Prompts describe positions, rules and research ideas: the firm's intellectual property. An open model on your own GPUs keeps it inside, with fixed weights that do not change behind a version name, which also makes backtests reproducible.
      When to use it
      Strategy research and anything mentioning live positions; public-news summaries can still use an API.
    2. News and sentiment labels

      Classify at volume

      What it is
      Classify each headline or post by asset, event type and direction with a decision model or a small local model (Qwen 8B, gpt-oss-20b), and store the label with its timestamp as a feature.
      When to use it
      Thousands of items a day on a 16–24 GB GPU; Grok when you need X posts in real time.
      AI angle
      Calibrated probabilities from a decision model can gate which labels reach the feature store.
    3. Filings and earnings calls

      Structured extraction

      What it is
      Extract guidance, risk factors and management tone from filings and call transcripts into JSON with a schema, using a long-context model (Qwen3.8-27B at 262K, Llama 4 Scout) and validating every number against the source text.
      When to use it
      Fundamental features for medium-horizon strategies.
    4. Research copilot

      Read, compare, summarise

      What it is
      An agent that reads papers, reports and your own research notes, answers with citations and drafts hypotheses for a person to test. Reasoning models (DeepSeek-V4, gpt-oss-120b) self-hosted, or a frontier API for public material.
      When to use it
      Faster literature review; never an automatic source of trades.
    5. Strategy code assistant

      Local coding model

      What it is
      Qwen3-Coder or a Qwen3.8 model on your own GPU writes backtest code, indicators and data pipelines without sending the strategy to a third party; tests and a person review every change.
      When to use it
      Quant teams whose code must not leave the network.
    6. Signals stay numeric

      Trees, not tokens

      What it is
      Price and order-book features go to LightGBM, XGBoost or similar models with walk-forward validation; LLM-derived labels join them as extra columns. Latency-critical paths run in milliseconds without any LLM call.
      When to use it
      The signal model; the LLM only enriches its inputs.
    7. Look-ahead bias

      The model knows the future

      What it is
      A model trained on text up to its knowledge cutoff has read what happened after your backtest dates. Labels it produces for 2024 news may encode 2025 outcomes, inflating backtest results.
      When to use it
      Backtest only on periods after the model's cutoff, or keep the same pinned weights for labelling and live use.
    8. Regulators on AI

      Same rules apply

      What it is
      ESMA (30 May 2024): firms using AI remain fully responsible under MiFID II for organisation, conduct and client best interest. FINRA Regulatory Notice 24-09 (27 June 2024): existing supervision, communications and record-keeping rules apply to generative AI. The SEC withdrew its predictive data analytics proposal on 12 June 2025.
      When to use it
      Log prompts, outputs and model versions like any other record when the output reaches a client.
      Key figures
      • Firms using AI stay fully responsible under MiFID II.Source: ESMA public statement, 30 May 2024
    9. Human approval and limits

      Risk stays deterministic

      What it is
      Position limits, kill switches and order validation are code, not prompts. An agent may propose a trade; a rule engine and, for anything material, a person approve it.
      When to use it
      Every system where model output can move money.
  8. Restricted content

    Policies, guards, law

    What it is
    Frontier APIs refuse or limit some categories by policy. Self-hosting removes the vendor's filter but not the licence's use policy or the law: you become the operator responsible for what the model produces.
    When to use it
    Sensitive but legitimate work that a general-purpose API is tuned to refuse.
    1. API usage policies

      The vendor decides

      What it is
      OpenAI's usage policies set universal rules (protect people, respect privacy, keep minors safe, human review for high-stakes automation). Anthropic's Usage Policy adds extra requirements for high-risk uses such as legal, health and financial advice. Both can change and both apply to every request.
      When to use it
      Check the policy before designing a product around an API.
    2. Use policies in licences

      Llama 4, Gemma, gpt-oss

      What it is
      Open weights can still carry a use policy. The Llama 4 Acceptable Use Policy bans illegal activity, harassment, unlicensed professional practice, weapons, self-harm content and deception; the Gemma Prohibited Use Policy (for Gemma releases under the Gemma terms) also bans sexually explicit content made for gratification; gpt-oss ships Apache 2.0 with a usage policy.
      When to use it
      A policy in the licence binds you even on your own GPUs.
    3. Permissive licences

      No field-of-use limits

      What it is
      Apache 2.0 and MIT place no restriction on what the model is used for. That moves every content decision to you and to the law of the country where you operate; the base model's own safety training still shapes what it will write.
      When to use it
      Products with their own documented content policy and moderation.
    4. Legitimate sensitive uses

      Where self-hosting helps

      What it is
      Security red-teaming and malware analysis in a sandbox, medical and legal drafting on private records, moderation training data that must contain the harmful examples it learns to catch, and mature fiction for verified adults where it is lawful.
      When to use it
      A documented purpose, an authorised user base and logs: the conditions that make it defensible.
    5. Hard lines

      Illegal under any licence

      What it is
      Child sexual abuse material, help with weapons capable of mass harm, non-consensual intimate imagery of real people, fraud and targeted harassment are illegal or banned regardless of the licence; every use policy above lists them, and hosting the model yourself only makes you the one accountable.
      When to use it
      Block these at input and output with a guard model, whatever the product.
    6. Guard models

      Classify input and output

      What it is
      Small classifiers that sit in front of and behind the main model: Llama Guard 4 (12B, multimodal, MLCommons hazard categories), gpt-oss-safeguard 20B and 120B (reason over a policy you write, Apache 2.0), Qwen3Guard (0.6B, 4B and 8B, streaming variant, 119 languages, Apache 2.0), IBM Granite Guardian (harm, jailbreak and RAG groundedness, Apache 2.0) and ShieldGemma 2 (4B image safety).
      When to use it
      Your own content policy, enforced locally: gpt-oss-safeguard reads it as a prompt, so changing the policy needs no retraining.
      AI angle
      A 4–8B guard adds a few GB of VRAM and milliseconds per request.
    7. Operator duties

      Age, logs, disclosure

      What it is
      Verify age where content requires it, keep logs of prompts and outputs, offer reporting and human review, and label synthetic content: the EU AI Act (Regulation 2024/1689, Article 50) requires machine-readable marking of AI-generated content and disclosure of deepfakes.
      When to use it
      Any public product on self-hosted weights, especially in the EU.
      Key figures
      • Providers must mark AI-generated audio, image, video and text in a machine-readable format.Source: EU AI Act, Article 50

Sources, read 2026-11-11: the model cards and announcements of each lab on Hugging Face and their own sites; the AI Board report "AI Models for Executives 2026" (26 October 2026) for flagship names, dates and prices; llama.cpp, NVIDIA and Hugging Face documentation for quantisation, KV cache and MoE memory; NVIDIA, AMD and Apple spec pages; the Apache 2.0 and MIT licence texts, the Llama 4 and Gemma use policies, the OpenAI and Anthropic usage policies; ESMA (30 May 2024), FINRA Regulatory Notice 24-09, the SEC (12 June 2025) and Regulation (EU) 2024/1689. "Reported" marks figures confirmed only by press coverage; memory sizes computed as parameters × bits ÷ 8 are estimates for weights alone. Read the licence file of the exact checkpoint before you deploy it.