Natural Language Processing · 免费预览

循环网络与门控记忆

作为无界上下文的循环隐藏状态、随时间反向传播、梯度消失问题、LSTM 门控，以及通往注意力机制的编码器-解码器桥梁。

1. Recurrent Networks and Gated Memory

Recurrent Networks and Gated Memory

2. Vanishing gradients and gates

Vanishing gradients and gates

3. Recurrence and gates in NumPy

Recurrence and gates in NumPy

4. Shapes of the shared matrices

A simple RNN has input dimension 6, hidden dimension 4, and a 3-word output softmax. What are the shapes of W, U, and V, and which of them are shared across time steps?

5. One recurrent step by hand

With ht−1=[0.76,−0.44]\mathbf{h}_{t-1} = [0.76, -0.44], xt=[−0.20,−0.60]\mathbf{x}_t = [-0.20, -0.60], U=[[−0.26,0.28],[0.15,0.19]]U = [[-0.26, 0.28], [0.15, 0.19]] and W=[[0.02,0.22],[0.14,0.17]]W = [[0.02, 0.22], [0.14, 0.17]], compute the first component of ht=tanh⁡(Uht−1+Wxt)\mathbf{h}_t = \tanh(U\mathbf{h}_{t-1} + W\mathbf{x}_t), rounding to three decimals.

6. Geometric decay backward through time

Backpropagating through a simple RNN, the gradient shrinks by a factor of 0.95 at each of 20 time steps. Compute the fraction of the original gradient that survives, rounding to four decimals (a small scientific-notation value is also fine).

7. Four gates, four jobs

Match each LSTM gate to its role. Which pairing is right?

8. The constant error carousel

Why does the LSTM cell state carry gradients over long spans when the simple RNN hidden state cannot?

进入交互式练习

免费注册 · 即时批改 · 自动安排间隔复习。

免费开始这门课