← All problems
#113 Layer Normalization Medium

Implement the forward pass of layer normalization for a 2D input tensor. Given an input tensor of shape [N, C] where N is the batch size and C is the number of features, normalize each sample independently across its C features, then apply learnable scale (weight) and shift (bias) parameters. Layer normalization is a core building block of transformer architectures.

For each sample \(i\), layer normalization computes: \[ \begin{align} \mu_i &= \frac{1}{C} \sum_{j=0}^{C-1} x_{i,j} \\ \sigma_i^2 &= \frac{1}{C} \sum_{j=0}^{C-1} (x_{i,j} - \mu_i)^2 \\ y_{i,j} &= \text{weight}_j \cdot \frac{x_{i,j} - \mu_i}{\sqrt{\sigma_i^2 + \varepsilon}} + \text{bias}_j \end{align} \]

Each row is its own normalization group (here N=3, C=6) μ₀, σ₀² over row 0 μ₁, σ₁² over row 1 μ₂, σ₂² over row 2 Statistics are reduced along the feature axis, never across rows. weight and bias are per-feature, shared by every row.

Implementation Requirements

Example

Input:
\(\text{input}\) (N=2, C=4): \[ \begin{bmatrix} 1.0 & 2.0 & 3.0 & 4.0 \\ -1.0 & 0.0 & 0.0 & 1.0 \end{bmatrix} \] \(\text{weight}\): \[ \begin{bmatrix} 1.0 & 1.0 & 1.0 & 1.0 \end{bmatrix} \] \(\text{bias}\): \[ \begin{bmatrix} 0.0 & 0.0 & 0.0 & 0.0 \end{bmatrix} \] \(\varepsilon\) = 1e-5

Output:
\(\text{output}\) (N=2, C=4): \[ \begin{bmatrix} -1.3416 & -0.4472 & 0.4472 & 1.3416 \\ -1.4142 & 0.0 & 0.0 & 1.4142 \end{bmatrix} \]

Constraints

Guided walkthrough coming soon. Below is the official starter signature in each language; the clickable line-by-line explanation hasn't been authored yet.
Problem text & starters from the official AlphaGPU/leetgpu-challenges. Solve it on leetgpu.com.