Forward Propagation on a Computational Graph

The forward pass, in order

Reading the graph

What has to be kept

A computational graph records that $\mathbf{h}$ is the result of applying $\phi$ to $\mathbf{z}$, and that $\mathbf{z}$ is the result of a matrix product of $\mathbf{W}^{(1)}$ with $\mathbf{x}$. What does this structure give the backward pass?

With no bias in the hidden layer, the hidden activation $\mathbf{h}$ satisfies $\mathbf{h} = \phi(\mathbf{z})$ where $\mathbf{z} = \mathbf{W}^{(1)}\mathbf{x}$. The dimension of $\mathbf{z}$ is ____ for $h$ hidden units.