vllm.model_executor.layers.pooler.common

Classes:

• PoolingParamsUpdate –

PoolingParamsUpdate dataclass

Attributes:

• requires_token_ids (bool) –

  Set this flag to enable CPU prompt token IDs for your pooler.

Source code in vllm/model_executor/layers/pooler/common.py

@dataclass(frozen=True)
class PoolingParamsUpdate:
    requires_token_ids: bool = False
    """Set this flag to enable CPU prompt token IDs for your pooler."""

    def __or__(self, other: "PoolingParamsUpdate") -> "PoolingParamsUpdate":
        return PoolingParamsUpdate(
            requires_token_ids=self.requires_token_ids or other.requires_token_ids,
        )

    def apply(self, params: PoolingParams) -> None:
        params.requires_token_ids = self.requires_token_ids

requires_token_ids = False class-attribute instance-attribute

Set this flag to enable CPU prompt token IDs for your pooler.