Lecture 3: Learning LLMs / Decoding

Readings

• Generating Sequences With Recurrent Neural Networks. Graves (2014).
• Recurrent neural network based language model. Mikolov et al. (2010). Interspeech.
• [GPT-1] Improving Language Understanding by Generative Pre-Training. Radford et al. (2018).
• [GPT-2] Language Models are Unsupervised Multitask Learners. Radford et al. (2019).

Unit: Generative models of text