Natural Language Processing · free preview

Automatic Speech Recognition

Mapping waveforms to text with convolutional front ends, attention encoder-decoders like Whisper, self-supervised HuBERT, the CTC loss, and word error rate.

1. Automatic Speech Recognition

Automatic Speech Recognition

2. Convolutional front ends

Convolutional front ends

3. Encoder-decoder and self-supervised recognition

Encoder-decoder and self-supervised recognition

4. CTC alignment and error rates

CTC alignment and error rates

5. Ranking the settings

Which of these recognition settings should have the highest word error rate for a system trained on standard read speech?

6. Counting encoder frames

Whisper's 30-second audio context yields 3,000 log mel frames at one per 10 ms, and its second convolutional layer uses a stride of 2. How many embeddings per context window leave the two-layer front end?

7. Scoring a transcript

Against a 200200-word reference, a recognizer produces a hypothesis with 1414 substitutions, 66 insertions and 33 deletions. What is its word error rate, to three decimal places?

8. Inventing the targets

In the first stage of HuBERT pretraining, what serves as the gold target at each masked frame?

9. Collapsing a path

A CTC decoder emits the frame-level path [·, a, b, b, ·, b, a] over alphabet {a, b, ·} with · the blank. Apply the collapsing function B, merging consecutive duplicates then deleting blanks. What text does the path decode to?

Practice this interactively

Free account · instant grading · spaced review that schedules itself.

Start this course for free