Implement the forward pass of layer normalization for a 2D input tensor. Given an input tensor of shape [N, C] where N is the batch size and C is the number of features, normalize each sample independently across its C features, then apply learnable scale (weight) and shift (bias) parameters. Layer normalization is a core building block of transformer architectures.
For each sample \(i\), layer normalization computes: \[ \begin{align} \mu_i &= \frac{1}{C} \sum_{j=0}^{C-1} x_{i,j} \\ \sigma_i^2 &= \frac{1}{C} \sum_{j=0}^{C-1} (x_{i,j} - \mu_i)^2 \\ y_{i,j} &= \text{weight}_j \cdot \frac{x_{i,j} - \mu_i}{\sqrt{\sigma_i^2 + \varepsilon}} + \text{bias}_j \end{align} \]
solve function signature must remain unchangedoutput tensor
Input:
\(\text{input}\) (N=2, C=4):
\[
\begin{bmatrix}
1.0 & 2.0 & 3.0 & 4.0 \\
-1.0 & 0.0 & 0.0 & 1.0
\end{bmatrix}
\]
\(\text{weight}\):
\[
\begin{bmatrix}
1.0 & 1.0 & 1.0 & 1.0
\end{bmatrix}
\]
\(\text{bias}\):
\[
\begin{bmatrix}
0.0 & 0.0 & 0.0 & 0.0
\end{bmatrix}
\]
\(\varepsilon\) = 1e-5
Output:
\(\text{output}\) (N=2, C=4):
\[
\begin{bmatrix}
-1.3416 & -0.4472 & 0.4472 & 1.3416 \\
-1.4142 & 0.0 & 0.0 & 1.4142
\end{bmatrix}
\]
N ≤ 65,536C ≤ 4,096eps = 1e-5N = 65,536, C = 512