← All problems
#81 INT4 Matmul Medium

Implement a weight-only INT4 quantized matrix multiplication (W4A16), a core kernel used in modern LLM inference. Given a float16 activation matrix x of shape M × K and a weight matrix stored in packed INT4 format, compute the output matrix y = x × WT of shape M × N, where W is the dequantized float16 weight matrix of shape N × K.

Packing format: Each byte of w_q stores two INT4 weights. The high nibble (bits 7–4) holds weight w[n, 2i] and the low nibble (bits 3–0) holds w[n, 2i+1]. INT4 values are stored unsigned in the range [0, 15] with an offset of 8, so the signed weight is nibble − 8, giving values in [−8, 7].

Dequantization: Weights are dequantized group-wise. Each contiguous block of group_size weights along the K dimension shares one float16 scale:

W[n, k] = (w_q_nibble[n, k] - 8) * scales[n, k // group_size]

Implementation Requirements

Example

Input (M = 2, N = 4, K = 4, group_size = 2):

Activations \(x\) (float16, \(2 \times 4\)): \[ \begin{bmatrix} 1.0 & 0.0 & 1.0 & 0.0 \\ 0.0 & 1.0 & 0.0 & 1.0 \end{bmatrix} \] Packed weights \(w\_q\) (uint8, \(4 \times 2\)) with signed INT4 values in brackets: \[ \begin{bmatrix} \texttt{0x99} & \texttt{0x99} \\ \texttt{0xAA} & \texttt{0xAA} \\ \texttt{0x77} & \texttt{0x77} \\ \texttt{0x88} & \texttt{0x88} \end{bmatrix} \;\Rightarrow\; W_{\text{int4}} = \begin{bmatrix} 1 & 1 & 1 & 1 \\ 2 & 2 & 2 & 2 \\ -1 & -1 & -1 & -1 \\ 0 & 0 & 0 & 0 \end{bmatrix} \] Scales (float16, \(4 \times 2\), all entries 0.5): \[ \begin{bmatrix} 0.5 & 0.5 \\ 0.5 & 0.5 \\ 0.5 & 0.5 \\ 0.5 & 0.5 \end{bmatrix} \;\Rightarrow\; W_{\text{dequant}} = \begin{bmatrix} 0.5 & 0.5 & 0.5 & 0.5 \\ 1.0 & 1.0 & 1.0 & 1.0 \\ -0.5 & -0.5 & -0.5 & -0.5 \\ 0.0 & 0.0 & 0.0 & 0.0 \end{bmatrix} \] Output \(y = x \times W^T\) (float16, \(2 \times 4\)): \[ \begin{bmatrix} 1.0 & 2.0 & -1.0 & 0.0 \\ 1.0 & 2.0 & -1.0 & 0.0 \end{bmatrix} \]

Constraints

Guided walkthrough coming soon. Below is the official starter signature in each language; the clickable line-by-line explanation hasn't been authored yet.
Problem text & starters from the official AlphaGPU/leetgpu-challenges. Solve it on leetgpu.com.