Skip to content

vllm.v1.kv_offload.cpu.common

Classes:

CPULoadStoreSpec

Bases: BlockIDsLoadStoreSpec

Spec for loading/storing KV chunks to/from CPU memory.

The inherited block_ids field holds chunk indices into the CPU cache (not GPU block IDs). The chunk_ids alias exposes the same array under the name used by the tiering layer.

Source code in vllm/v1/kv_offload/cpu/common.py 
16
17
18
19
20
21
22
23
24
25
26
class CPULoadStoreSpec(BlockIDsLoadStoreSpec):
    """Spec for loading/storing KV chunks to/from CPU memory.

    The inherited block_ids field holds chunk indices into the
    CPU cache (not GPU block IDs). The chunk_ids alias exposes
    the same array under the name used by the tiering layer.
    """

    @property
    def chunk_ids(self) -> np.ndarray:
        return self.block_ids