The residual path
What happens where the skip path rejoins?
29 interactive PyTorch architectures. Follow a guided tour, inspect captured shapes, and take the source back to your own project.
No installation · No Python neededCHOOSE A LEARNING PATH
Start with attention, change one design choice at a time, then connect the pieces in LLaMA.
Follow the path ↗6 lessonsFollow convolution, residual paths, U-shaped skips and image patches through landmark vision designs.
Follow the path ↗5 lessonsExplore alignment, bidirectional pretraining, image–text pairing, low-rank adaptation and reasoning post-training.
Follow the path ↗4 lessonsCompare the GAN game with noise prediction, latent-space denoising and a Transformer diffusion backbone.
Follow the path ↗3 lessonsInspect selective recurrent state, Evoformer sequence/pair coupling and promptable mask decoding.
Follow the path ↗A teaching order, not a claim that each later model adopted every earlier idea.
29 models to explore
What happens where the skip path rejoins?
Where does an image become a classifier input?
How do queries, keys and values meet?
Component · 2019
What does normalization change in one token vector?
Component · 2020
Why does this feed-forward block split into two branches?
Architecture · 2017
How does a decoder form a next-token distribution?
Component · 2020
What happens to the shortcut when normalization moves?
Component · 2023
Which attention heads can share a key/value cache?
Component · 2021
Where does token position enter the attention score?
Inference method · 2022
Can exact attention avoid storing the entire score matrix?
Component · 2024
What must MLA store to avoid reconstructing every key and value?
Component · 2017
How can a token use only part of a larger network?
Training method · 2021
How can two small matrices adapt a frozen linear layer?
Architecture · 2023
Where do RMSNorm, RoPE and SwiGLU meet in an actual decoder block?
Architecture · 2012
How do five convolutions turn an image into a classification vector?
Architecture · 2015
What can flow through a shortcut when a residual branch changes?
Architecture · 2015
How does a segmentation decoder recover features lost during downsampling?
Architecture · 2020
Where does an image become a sequence for a Transformer?
Component · 2014
How does a recurrent decoder decide which source positions to consult?
Architecture · 2018
What changes when the encoder can see both left and right context?
Training method · 2014
Which network is being updated, and what is it trying to change?
Training method · 2020
How does a model learn to reverse corruption?
Architecture · 2021
Why denoise a latent, and where does text enter?
Architecture · 2022
What changes when a Transformer denoises latent patches?
Training method · 2021
Where do the image and text meet?
Component · 2023
How do image features and a user prompt become a mask?
Architecture · 2023
How does the input decide what a recurrent state retains?
Component · 2021
How do sequence and pair representations interact before structure is produced?
Training method · 2025
What changes in the learning process while the base architecture stays the same?
Try another idea, or return to the full collection.
BRING IT BACK TO YOUR PROJECT