Machine Learning Fundamentals · 免费预览
参数初始化
参数以随机值而非零开始，因为在一层中用零初始化的权重会使所有单元完全相同，该层只能学到一种特征；而该分布的尺度则控制着激活值和梯度是增长还是消失。
1. The failure of zero initialization
The failure of zero initialization
2. The default scaling
The default scaling
3. What goes wrong without the balance
What goes wrong without the balance
4. Why zero initialization fails
A hidden layer of 256 units has every weight initialized to zero and every bias to zero. Training begins with minibatch gradient descent. Which statement describes what happens?
5. The default scale
A weight tensor has input units. The common initialization draws weights from a normal distribution with standard deviation , which equals ____.