← All problems
#67 MoE Top-K Gating Medium

Implement a GPU program that performs Top-K Gating for Mixture of Experts (MoE) models. Given a logit matrix of shape [M, E] where M is the number of tokens and E is the number of experts, identify the k largest values in each row, extract their indices, and apply softmax to get mixing weights.

For each row i, the operation computes: \[ \begin{align} \text{indices}_i, \text{vals}_i &= \text{TopK}(\text{logits}_i, k) \\ \text{vals}_i &= \text{logits}_i[\text{indices}_i] \\ \text{weights}_i &= \text{Softmax}(\text{vals}_i) \end{align} \]

The selected experts must remain ordered by descending logit value, matching the order returned by topk. The topk_weights array must correspond positionally to topk_indices in that same order.

Implementation Requirements

Example 1:

Input:
  logits = [[1.0, 2.0, 3.0, 4.0],
            [4.0, 3.0, 2.0, 1.0]]
  M = 2, E = 4, k = 2

Output:
  topk_weights = [[0.7311, 0.2689],
                  [0.7311, 0.2689]]
  topk_indices = [[3, 2],
                  [0, 1]]

Explanation:
Row 0: Top-2 values are 4.0 and 3.0 at indices 3 and 2.
       Softmax([4.0, 3.0]) = [0.7311, 0.2689]
Row 1: Top-2 values are 4.0 and 3.0 at indices 0 and 1.
       Softmax([4.0, 3.0]) = [0.7311, 0.2689]

Constraints

Guided walkthrough coming soon. Below is the official starter signature in each language; the clickable line-by-line explanation hasn't been authored yet.
Problem text & starters from the official AlphaGPU/leetgpu-challenges. Solve it on leetgpu.com.