Skip to content

vllm.v1.worker.gpu.kv_connector

Classes:

  • KVConnector –

    KVConnector interface used by GPUModelRunner.

KVConnector

KVConnector interface used by GPUModelRunner.

Source code in vllm/v1/worker/gpu/kv_connector.py 
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
class KVConnector:
    """KVConnector interface used by GPUModelRunner."""

    def pre_forward(self, scheduler_output: "SchedulerOutput", **kwargs: Any) -> None:
        pass

    def finish_forward(self) -> None:
        pass

    def post_forward(self, finished_req_ids: set[str]) -> KVConnectorOutput | None:
        return None

    def no_forward(self, scheduler_output: "SchedulerOutput") -> ModelRunnerOutput:
        return EMPTY_MODEL_RUNNER_OUTPUT

    def set_disabled(self, disabled: bool) -> None:
        pass

    def reset_capture_state(self) -> None:
        pass