Skip to content

vllm.models.kimi_k3.nvidia.specialized_decode.cutedsl.moe

Modules:

  • dense –

    Prepared first dense MLP, sharing projection and SiTU decode arithmetic.

  • experts –

    Persistent routed experts: TMA producer, MXFP8 activations, FP4 x MXFP8 FC1/FC2 MMAs,

  • experts_runtime –

    Persistent routed experts for B<=64 (B>8: grouped units of <=8 tokens per expert)

  • front –

    MoE front: the weight-stationary [router shard; down shard] GEMM publishes logits

  • prepared –

    Prepared MoE provider: one model owner serializes its scratch, no tensor pointers

  • routing –

    Top16 routing (threshold, regional candidates and exact composite ranking) and the

  • shared –

    Fused shared expert for B>8: gate/up, SiTU, down and the routed tail's reduce-scatter

  • small_batch –
  • tail –

    Clustered MoE tail (TP8 B1..4, TP16 B1..8): routed finalization/publication,

  • tail_runtime –

    Routed MoE tail: one fused kernel finalizes this rank's top-16 rows, multicasts the