Skip to content

vllm.model_executor.layers.pooler.abstract

Classes:

  • Pooler –

    The interface required for all poolers used in pooling models in vLLM.

Pooler

Bases: Module, ABC

The interface required for all poolers used in pooling models in vLLM.

Methods:

Source code in vllm/model_executor/layers/pooler/abstract.py 
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class Pooler(nn.Module, ABC):
    """The interface required for all poolers used in pooling models in vLLM."""

    @abstractmethod
    def get_supported_tasks(self) -> Set[PoolingTask]:
        """Determine which pooling tasks are supported."""
        raise NotImplementedError

    def get_pooling_updates(self, task: PoolingTask) -> PoolingParamsUpdate:
        """Construct the updated pooling parameters to use for a supported task."""
        return PoolingParamsUpdate()

    @abstractmethod
    def forward(
        self,
        hidden_states: torch.Tensor,
        pooling_metadata: PoolingMetadata,
    ) -> PoolerOutput:
        raise NotImplementedError

get_pooling_updates(task)

Construct the updated pooling parameters to use for a supported task.

Source code in vllm/model_executor/layers/pooler/abstract.py 
24
25
26
def get_pooling_updates(self, task: PoolingTask) -> PoolingParamsUpdate:
    """Construct the updated pooling parameters to use for a supported task."""
    return PoolingParamsUpdate()

get_supported_tasks() abstractmethod

Determine which pooling tasks are supported.

Source code in vllm/model_executor/layers/pooler/abstract.py 
19
20
21
22
@abstractmethod
def get_supported_tasks(self) -> Set[PoolingTask]:
    """Determine which pooling tasks are supported."""
    raise NotImplementedError