Skip to content

vllm.model_executor.warmup.flashinfer_autotune_cache

FlashInfer autotune cache helpers.

Functions:

_normalize_cache_config_for_hash(cache_config)

Normalize cache config for hashing

Source code in vllm/model_executor/warmup/flashinfer_autotune_cache.py 
23
24
25
26
27
28
29
def _normalize_cache_config_for_hash(cache_config: CacheConfig) -> CacheConfig:
    """Normalize cache config for hashing"""
    if cache_config.mamba_block_size is None and cache_config.kv_cache_layout is None:
        return cache_config
    normalized = replace(cache_config, mamba_block_size=None)
    assert normalized.kv_cache_layout is None
    return normalized

resolve_flashinfer_autotune_v2_root()

Placement root for FlashInfer's managed autotune store.

Unlike :func:resolve_flashinfer_autotune_file, no vLLM-side identity (config hash, flashinfer version, arch) is folded into the path: the managed store keys entries per op and hashes the software/hardware environment itself. None selects FlashInfer's default root.

Source code in vllm/model_executor/warmup/flashinfer_autotune_cache.py 
 96
 97
 98
 99
100
101
102
103
104
105
def resolve_flashinfer_autotune_v2_root() -> Path | None:
    """Placement root for FlashInfer's managed autotune store.

    Unlike :func:`resolve_flashinfer_autotune_file`, no vLLM-side identity
    (config hash, flashinfer version, arch) is folded into the path: the
    managed store keys entries per op and hashes the software/hardware
    environment itself. ``None`` selects FlashInfer's default root.
    """
    override_dir = envs.VLLM_FLASHINFER_AUTOTUNE_CACHE_DIR
    return Path(override_dir).expanduser() if override_dir else None