Lecture 2: Transformer LMs

Readings

• The Long Short-Term Memory and Other GatedRNNs. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.10-10.12.
• Attention Is All You Need. Vaswani et al. (2017). NeurIPS.
• The Illustrated Transformer. Alammar (2018).