Skip to content

You are viewing the latest developer preview docs. Click here to view docs for the latest stable release.

vLLM

mimo

Initializing search

Home
User Guide
Developer Guide
Benchmarking
API Reference
CLI Reference
Community

vLLM

Home
User Guide
User Guide
- Getting Started
  Getting Started
  - Quickstart
  - Installation
    
    Installation
    
    GPU
    
    CPU
    
    TPU
  - Examples
    
    Examples
    
    Applications
    Applications
    
    API Server
    
    Chatbot
    
    Rag
    
    Basic
    Basic
    
    Offline Inference
    
    Online Serving
    
    Deployment
    Deployment
    
    Async LLM Streaming
    
    Helm Charts
    
    LLM Engine Example
    
    Sagemaker-Entrypoint
    
    Disaggregated
    Disaggregated
    
    Disaggregated Encoder
    
    Disaggregated Serving
    
    Ec Both Encoder
    
    Disaggregated Prefill V1
    
    Flexkv Connector
    
    KV Load Failure Recovery Test
    
    LMCache Examples
    
    Mooncake Connector
    
    Features
    Features
    
    Automatic Prefix Caching
    
    Batch Invariance
    
    Context Extension
    
    Data Parallel
    
    Kv Events
    
    Logging Configuration
    
    Custom Logits Processors
    
    LoRA
    
    Offline Inference with the OpenAI Batch file format
    
    Pause Resume
    
    Profiling
    
    Prompt Embed
    
    Reset Kv
    
    Sharded State
    
    Speculative Decoding
    
    Structured reads on DiffusionGemma
    
    Structured Outputs
    
    Tensorize vLLM Model
    
    Torchrun
    
    Generate
    Generate
    
    Batched Chat Completions Online
    
    Multimodal
    
    Qwen 1M Offline
    
    Trace Replay Offline
    
    Observability
    Observability
    
    Monitoring Dashboards
    
    Metrics
    
    Setup OpenTelemetry POC
    
    Prometheus and Grafana
    
    Pooling
    Pooling
    
    Classify
    
    Embed
    
    Plugin
    
    Reward
    
    Score
    
    Token Classify
    
    Token Embed
    
    Ray Serving
    Ray Serving
    
    Batch LLM Inference
    
    Elastic Ep
    
    Multi-Node-Serving
    
    Ray Serve Deepseek
    
    Run Cluster
    
    Reasoning
    Reasoning
    
    OpenAI Chat Completion Tool Calls With Reasoning
    
    OpenAI Chat Completion With Reasoning
    
    OpenAI Chat Completion With Reasoning Streaming
    
    OpenAI Responses Client
    
    RL
    RL
    
    Rdt vLLM Serve
    
    Rdt Weight Source
    
    RLHF Async New APIs
    
    RLHF Http IPC
    
    RLHF Http NCCL
    
    RLHF IPC Fsdp Ep
    
    RLHF M2N
    
    RLHF NCCL Fsdp Ep
    
    RLHF Sharded Rdt Small Ep
    
    RLHF Sparse NCCL
    
    Routed Experts E2E
    
    Skip Loading Weights In Engine Init
    
    Scale Out
    Scale Out
    
    Init
    
    Example Mm Serve
    
    Token Generation Client
    
    Speech To Text
    Speech To Text
    
    OpenAI
    
    Realtime
    
    Tool Calling
    Tool Calling
    
    Chat With Tools Offline
    
    OpenAI Chat Completion Client With Tools
    
    OpenAI Chat Completion Client With Tools Required
    
    OpenAI Chat Completion Client With Tools Xlam
    
    OpenAI Chat Completion Client With Tools Xlam Streaming
    
    OpenAI Responses Client With Mcp Tools
    
    OpenAI Responses Client With Tools
- General
  General
- Inference and Serving
  Inference and Serving
  - Offline Inference
  - Online Serving
    
    Online Serving
    
    Derenderer APIs
    
    Generative Scoring
    
    OpenAI-Compatible Server
    
    Renderer APIs
    
    Speech to Text APIs
    
    Tokens In <> Tokens Out API
    
    Trace Replay
  - Context Parallel Deployment
  - Data Parallel Deployment
  - Troubleshooting distributed deployments
  - Expert Parallel Deployment
  - Parallelism and Scaling
  - Integrations
    Integrations
    
    Claude Code
    
    Codex
    
    LangChain
    
    LlamaIndex
- Deployment
  Deployment
  - Using Docker
  - Using Kubernetes
  - Using Nginx
  - Frameworks
    Frameworks
    
    Anyscale
    
    AnythingLLM
    
    AutoGen
    
    BentoML
    
    Cerebrium
    
    Chatbox
    
    Crusoe
    
    Dify
    
    dstack
    
    Haystack
    
    Helm
    
    Hugging Face Inference Endpoints
    
    LiteLLM
    
    Lobe Chat
    
    LWS
    
    Modal
    
    Nebius Serverless AI
    
    Open WebUI
    
    Retrieval-Augmented Generation
    
    RunPod
    
    SkyPilot
    
    Streamlit
    
    NVIDIA Triton
  - Integrations
    Integrations
    
    AIBrix
    
    NVIDIA Dynamo
    
    KAITO
    
    KServe
    
    Kthena
    
    KubeAI
    
    KubeRay
    
    Llama Stack
    
    llm-d
    
    llmaz
    
    Production stack
- Training
  Training
- Configuration
  Configuration
- Models
  Models
  - Supported Models
  - Generative Models
  - Pooling Models
    
    Pooling Models
    
    Classification Usages
    
    Embedding Usages
    
    Reward Usages
    
    Scoring Usages
    
    Specific Model Examples
    
    Token Classification Usages
    
    Token Embedding Usages
  - Extensions
    Extensions
    
    Loading model weights with fastsafetensors
    
    Loading Model Weights with InstantTensor
    
    Loading models with Run:ai Model Streamer
    
    Loading models with CoreWeave's Tensorizer
  - Hardware Supported Models
    Hardware Supported Models
    
    CPU - Intel® Xeon®
    
    XPU - Intel® GPUs
    
    TPU
- Features
  Features
Developer Guide
Developer Guide
Benchmarking
Benchmarking
API Reference
API Reference
- vllm
  vllm
  - collect_env
  - connections
  - env_override
  - envs
  - exceptions
  - forward_context
  - logger
  - logits_process
  - logprobs
  - model_inspection
  - outputs
  - pooling_params
  - sampling_params
  - scalar_type
  - scripts
  - sequence
  - tasks
  - version
  - assets
  - benchmarks
  - compilation
  - config
  - cute_utils
  - device_allocator
  - distributed
  - engine
  - entrypoints
  - inputs
  - ir
  - kernels
  - logging_utils
  - lora
  - model_executor
    
    model_executor
    
    custom_op
    
    parameter
    
    utils
    
    determinism
    
    hw_agnostic
    
    kernels
    
    layers
    
    model_loader
    
    models
    
    models
    
    AXK1
    
    adapters
    
    afmoe
    
    aimv2
    
    apertus
    
    aria
    
    audioflamingo3
    
    bagel
    
    bailing_moe
    
    bailing_moe_linear
    
    bailing_moe_mtp
    
    bailing_moe_v3
    
    bailing_moe_v3_mtp
    
    bailing_moe_v3_vl
    
    bee
    
    bert
    
    bert_with_rope
    
    blip
    
    blip2
    
    bloom
    
    chatglm
    
    clip
    
    cohere2_moe
    
    cohere2_vision
    
    cohere_asr
    
    cohere_compass
    
    cohere_eagle
    
    colbert
    
    colmodernvbert
    
    colpali
    
    colqwen3
    
    colqwen3_5
    
    commandr
    
    config
    
    conformer_encoder
    
    cosmos3
    
    cosmos3_edge
    
    dbrx
    
    deepencoder
    
    deepencoder2
    
    deepseek_eagle
    
    deepseek_eagle3
    
    deepseek_mtp
    
    deepseek_ocr
    
    deepseek_ocr2
    
    deepseek_v2
    
    deepseek_vl2
    
    diffusion_gemma
    
    diffusion_gemma_sampler
    
    dots_ocr
    
    eagle2_5_vl
    
    ernie45
    
    ernie45_moe
    
    ernie45_vl
    
    ernie45_vl_moe
    
    ernie_mtp
    
    exaone
    
    exaone4
    
    exaone4_5
    
    exaone4_5_mtp
    
    exaone_moe
    
    exaone_moe_mtp
    
    extract_hidden_states
    
    falcon
    
    falcon_h1
    
    fireredasr2
    
    funasr
    
    funaudiochat
    
    gemma
    
    gemma2
    
    gemma3
    
    gemma3_mm
    
    gemma3n
    
    gemma3n_audio_utils
    
    gemma3n_mm
    
    gemma4
    
    gemma4_dspark
    
    gemma4_mm
    
    gemma4_mtp
    
    gemma4_unified
    
    glm4
    
    glm4_1v
    
    glm4_moe
    
    glm4_moe_lite
    
    glm4_moe_lite_mtp
    
    glm4_moe_mtp
    
    glm4v
    
    glm_ocr
    
    glm_ocr_mtp
    
    glmasr
    
    glmasr_utils
    
    gpt2
    
    gpt_j
    
    gpt_oss
    
    granite
    
    granite4_vision
    
    granite_speech
    
    granite_speech_plus
    
    granitemoe
    
    granitemoehybrid
    
    granitemoeshared
    
    h2ovl
    
    hrm_text
    
    hy_v3
    
    hy_v3_mtp
    
    hyperclovax
    
    hyperclovax_vision_v2
    
    idefics2_vision_model
    
    idefics3
    
    interfaces
    
    interfaces_base
    
    intern_vit
    
    internlm2
    
    interns1
    
    interns1_pro
    
    interns1_vit
    
    interns2_mobius
    
    interns2_preview
    
    internvl
    
    iquest_loopcoder
    
    isaac
    
    jamba
    
    jina
    
    jina_vl
    
    k2_horizon
    
    kanana_v
    
    keye
    
    keye_vl1_5
    
    kimi_audio
    
    kimi_k25
    
    kimi_k25_vit
    
    kimi_vl
    
    laguna
    
    laguna_dflash
    
    lfm2
    
    lfm2_moe
    
    lfm2_siglip2
    
    lfm2_vl
    
    lightonocr
    
    lilicorr
    
    llama
    
    llama4
    
    llama4_eagle
    
    llama_eagle
    
    llama_eagle3
    
    llava
    
    llava_next
    
    llava_next_video
    
    llava_onevision
    
    llava_onevision2
    
    longcat_flash
    
    longcat_flash_mtp
    
    longcat_flash_ngram
    
    mamba
    
    mamba2
    
    medusa
    
    midashenglm
    
    mimo
    
    mimo_audio
    
    mimo_mtp
    
    mimo_v2
    
    mimo_v2_mtp
    
    mimo_v2_omni
    
    minicpm
    
    minicpm3
    
    minicpm_eagle
    
    minicpmo
    
    minicpmv
    
    minicpmv4_6
    
    minimax_m2
    
    mistral
    
    mistral3
    
    mistral_eagle
    
    mistral_large_3
    
    mistral_large_3_eagle
    
    mixtral
    
    mllama4
    
    mlp_speculator
    
    modernbert
    
    module_mapping
    
    molmo
    
    molmo2
    
    moondream3
    
    moonvit
    
    moss_audio
    
    moss_transcribe_diarize
    
    muse_glimmer
    
    nano_nemotron_vl
    
    nemotron
    
    nemotron_h
    
    nemotron_h_mtp
    
    nemotron_nas
    
    nemotron_parse
    
    nemotron_vl
    
    nvlm_d
    
    olmo_hybrid
    
    olmoe
    
    openai_privacy_filter
    
    opencua
    
    openpangu
    
    openpangu_mtp
    
    openpangu_vl
    
    openvla
    
    opt
    
    orion
    
    ovis
    
    ovis2_5
    
    paddleocr_vl
    
    paligemma
    
    parakeet
    
    param2moe
    
    phi3
    
    phi3v
    
    phi4mm
    
    phi4mm_audio
    
    phi4mm_utils
    
    phi4siglip
    
    phimoe
    
    pixtral
    
    plamo3
    
    qianfan_ocr
    
    qwen2
    
    qwen2_5_omni_thinker
    
    qwen2_5_vl
    
    qwen2_audio
    
    qwen2_moe
    
    qwen2_rm
    
    qwen2_vl
    
    qwen3
    
    qwen3_5
    
    qwen3_5_mtp
    
    qwen3_asr
    
    qwen3_asr_forced_aligner
    
    qwen3_asr_realtime
    
    qwen3_dflash
    
    qwen3_dflash2
    
    qwen3_dspark
    
    qwen3_eagle3
    
    qwen3_moe
    
    qwen3_next
    
    qwen3_next_mtp
    
    qwen3_omni_moe_thinker
    
    qwen3_vl
    
    qwen3_vl_moe
    
    radio
    
    registry
    
    rnj1
    
    roberta
    
    rvl
    
    sarvam
    
    siglip
    
    siglip2navit
    
    skyworkr1v
    
    smolvlm
    
    solar
    
    stablelm
    
    step1
    
    step3_text
    
    step3_vl
    
    step3p5
    
    step3p5_mtp
    
    step3p7
    
    step_vl
    
    telechat2
    
    teleflm
    
    terratorch
    
    ultravox
    
    unlimited_ocr
    
    utils
    
    vision
    
    voxtral
    
    voxtral_realtime
    
    voyage
    
    whisper
    
    whisper_causal
    
    whisper_utils
    
    zamba2
    
    transformers
    
    offloader
    
    warmup
  - models
  - multimodal
  - parser
  - platforms
  - plugins
  - profiler
  - ray
  - reasoning
  - renderers
  - snapshot
  - tilelang_utils
  - tokenizers
  - tool_parsers
  - tracing
  - transformers_utils
  - triton_utils
  - usage
  - utils
  - v1
CLI Reference
CLI Reference
- vllm
  vllm
  - chat
  - complete
  - preload
  - run-batch
  - serve
  - bench
    
    bench
    
    latency
    
    mm-processor
    
    serve
    
    startup
    
    throughput
    
    sweep
    
    sweep
    
    plot
    
    plot_pareto
    
    serve
    
    serve_workload
    
    startup
  - launch
    
    launch
    
    render
Community
Community
- Contact Us
- Meetups
- Reviewers
- Sponsors
- Governance
  Governance
- Blog
- Forum
- Slack

Home
API Reference
vllm
model_executor
models

`vllm.model_executor.models.mimo` ¶

Inference-only MiMo model compatible with HuggingFace weights.

October 2, 2026

Made with Material for MkDocs