Skip to content

vllm.platforms.zen_cpu

Classes:

  • ZenCpuPlatform –

    CPU platform with AMD Zen (ZenDNN/zentorch) optimizations.

ZenCpuPlatform

Bases: CpuPlatform

CPU platform with AMD Zen (ZenDNN/zentorch) optimizations.

Model-load time (dispatch_cpu_unquantized_gemm in layers/utils.py): - Routes linear ops to zentorch_linear_unary. - When VLLM_ZENTORCH_WEIGHT_PREPACK=1 (default), eagerly prepacks weights via zentorch_weight_prepack_for_linear.

Source code in vllm/platforms/zen_cpu.py 
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class ZenCpuPlatform(CpuPlatform):
    """CPU platform with AMD Zen (ZenDNN/zentorch) optimizations.

    Model-load time (dispatch_cpu_unquantized_gemm in layers/utils.py):
      - Routes linear ops to zentorch_linear_unary.
      - When VLLM_ZENTORCH_WEIGHT_PREPACK=1 (default), eagerly prepacks
        weights via zentorch_weight_prepack_for_linear.
    """

    device_name: str = "cpu"
    device_type: str = "cpu"

    def is_zen_cpu(self) -> bool:
        # is_cpu() also returns True for this platform (inherited from CpuPlatform).
        return True

    # fp16 compute needs AVX512-FP16, which AMD ships from Venice (6th Gen
    # EPYC) onwards; Turin (5th Gen EPYC) and earlier have no fp16 kernels.
    @property
    def supported_dtypes(self) -> list[torch.dtype]:
        if torch.ops.zentorch.zentorch_is_fp16_supported():
            return [torch.bfloat16, torch.float32, torch.float16]
        return [torch.bfloat16, torch.float32]