10-423 / 10-623 / 10-723Generative AI

Spring 2026 · Carnegie Mellon University

Generative AI

10-423 / 10-623 / 10-723

The machine learning and AI techniques driving the recent advances in generative modeling and foundation models.

MWF 2:00–3:20pm (DH 2210) · Lectures Mon/Wed · Recitations occasional Fridays

Lectures

26

Homeworks

6

Quizzes

6

Exams

1

Instructors

Aran Nayebi

Matt Gormley

Example slides

Example slide 1

Example slide 2

Example slide 3

Example slide 4

Example slide 5

Example slide 6

Example slide 7

Example slide 8

Course Modules

About

The Course

Generative models of text

RNN LMs / Autodiff

RNN LMs / Autodiff

Lecture 1: RNN LMs / Autodiff

Transformer LMs

Transformer LMs

Lecture 2: Transformer LMs

Learning LLMs / Decoding

Learning LLMs / Decoding

Lecture 3: Learning LLMs / Decoding

Pre-training, fine-tuning / Modern Transformers

Pre-training, fine-tuning / Modern Transformers

Generative models of images

Computer Vision: CNNs / Encoder-only Transformers / Vision Transformers

Computer Vision: CNNs / Encoder-only Transformers / Vision Transformers

Generative Adversarial Networks (GANs) / PGM

Generative Adversarial Networks (GANs) / PGM

Lecture 6: Generative Adversarial Networks (GANs) / PGM

Diffusion models (Part I)

Diffusion models (Part I)

Lecture 7: Diffusion models (Part I)

Diffusion models (Part II)

Diffusion models (Part II)

Applying and adapting foundation models

Variational Autoencoders (VAEs)

Variational Autoencoders (VAEs)

Lecture 9: Variational Autoencoders (VAEs)

Parameter-efficient fine tuning

Parameter-efficient fine tuning

In-Context Learning / Prompt Engineering / Instruction Fine-tuning / Reinforcement learning with human feedback (RLHF)

In-Context Learning / Prompt Engineering / Instruction Fine-tuning / Reinforcement learning with human feedback (RLHF)

Lecture 11: In-Context Learning / Prompt Engineering / Instruction Fine-tuning / Reinforcement learning with human feedback (RLHF)

Multimodal foundation models

Direct Preference Optimization (DPO) / Text-to-image generation / Latent diffusion model

Direct Preference Optimization (DPO) / Text-to-image generation / Latent diffusion model

Vision-language models

Vision-language models

Lecture 13: Vision-language models

Cross-Attention / Diffusion Transformer / Prompt-to-Prompt

Cross-Attention / Diffusion Transformer / Prompt-to-Prompt

Scaling Up

Querying Transformer / Scaling Laws

Querying Transformer / Scaling Laws

Lecture 15: Querying Transformer / Scaling Laws

Mixture of Experts

Mixture of Experts

Distributed training

Distributed training

Lecture 17: Distributed training

Flash Attention / Efficient decoding strategies

Flash Attention / Efficient decoding strategies

Lecture 18: Flash Attention / Efficient decoding strategies

Advanced Topics

Long Context in LLM

Long Context in LLM

Reasoning Models

Reasoning Models

State Space Models / Hybrid Models

State Space Models / Hybrid Models

Lecture 21: State Space Models / Hybrid Models

Real-world Issues and Considerations / What can go wrong?

Real-world Issues and Considerations / What can go wrong?

Code Generation / Autonomous Agents

Code Generation / Autonomous Agents

Lecture 23: Code Generation / Autonomous Agents

Audio understanding and synthesis

Audio understanding and synthesis

Lecture 24: Audio understanding and synthesis

Generative Models for Videos

Generative Models for Videos

Lecture 25: Generative Models for Videos

Interactive World Models + Science of Alignment

Interactive World Models + Science of Alignment

Announcements

Welcome to 10-423/623/723!
1/13/2026
Lectures begin the first week of the semester. Please read the Syllabus and set up Gradescope.
Spring 2026 Information
1/12/2026
For information about 10-423/623/723 in Spring 2026, click here!