Course: MIT 6.S191: Introduction to Deep Learning

Video Lecture: https://youtu.be/QvkQ1B3FBqA

Course website: IntroToDeepLearning.com

Lecturers: Alexander Amini and Ava Soleimany

Find all my notes for this course in the ML Course Notes repo.

Please note that this is a rough draft of the notes, so you might find mistakes. The visuals and equations are directly obtained from the original slides which you can find on the course website. All of the credit goes to the lecturers. I simply hope that the notes serve as accompanying study material. The lecturers use TensorFlow but I will try my best to convert all codes to PyTorch.

Topics:

• Deep sequence modeling
• Intuition and Applications of RNNs
• Drawbacks of RNNs
• Self-Attention and Transformers

Sequences in the Wild

Sequential data is all around us. There is audio (sequence of sound waves), text (sequence of characters/words), stock prices, DNA sequences, and more.