Memory tricks that make CNNs, Transformers and LLMs click

Convolutional layer: filters slide across image detecting local features (edges, textures, shapes). Max Pooling: reduces spatial dimensions, provides translation invariance. Fully Connected: flattens and classifies. Key insight: early layers detect edges and corners, deeper layers detect faces and objects. Parameter sharing: same filter applied everywhere -- far fewer parameters than dense network.

RNNs process sequences by maintaining hidden state memory. Vanilla RNN: vanishing gradients over long sequences -- forgets early inputs. LSTM (Long Short-Term Memory): forget gate (what to erase), input gate (what new info to add), output gate (what to expose). GRU: simplified LSTM with two gates -- fewer parameters, comparable performance. Now largely replaced by Transformers for most NLP tasks.

No recurrence -- processes entire sequence in parallel. Self-attention: every token attends to every other. Positional encoding: sine/cosine functions inject position information (without it: dog bites man = man bites dog). Layer normalization + residual connections. Encoder-only (BERT): understanding tasks. Decoder-only (GPT, Claude): generation. Encoder-Decoder (T5): translation and summarization. Foundation of all modern LLMs.

Pretraining: train massive transformer on huge text corpus, predict next token, self-supervised (next token IS the label). Supervised Fine-Tuning (SFT): train on curated instruction-response pairs -- teaches instruction following. RLHF: human raters rank outputs, reward model trained, RL fine-tunes LLM to maximize it -- aligns with human values. Emergent abilities: few-shot learning, chain-of-thought, code generation appear suddenly at scale.

GANs: Generator produces fake data, Discriminator classifies real vs fake, trained adversarially. Challenges: mode collapse (generator produces only a few types), training instability. Diffusion models: forward process adds noise over T steps, reverse process learns to denoise -- generate by starting from pure noise and denoising. Stable Diffusion, DALL-E 3, Midjourney use diffusion. More stable, better diversity, easier text conditioning.

Embeddings map discrete items (words, images, users) to dense vectors where similar items are geometrically close. Word2Vec/GloVe: static -- same word gets same vector regardless of context. BERT: contextual -- same word gets different vector based on context. Cosine similarity measures closeness. Vector databases store embeddings for semantic search. Foundation of RAG: embed documents, retrieve similar chunks as LLM context.

LoRA: instead of updating full weight matrix W, add low-rank update DeltaW = B times A where B is d x r and A is r x k with r much smaller than d. Only A and B trained -- 100-10,000x fewer parameters. At inference: merge back into W -- no latency penalty. QLoRA: combine 4-bit quantization with LoRA -- democratized LLM fine-tuning on consumer hardware.

ViT (Dosovitskiy et al., 2020): split image into 16x16 pixel patches, flatten each, project to embedding, add position embeddings, run through standard Transformer encoder. Global context from layer 1 -- unlike CNNs that build context gradually. Needs more data than CNNs (no inductive biases). SWIN: hierarchical ViT with shifted windows, O(n) complexity, dominant backbone for detection and segmentation.

MoE replaces each dense feed-forward layer with N expert networks, routing each token to only 2-4 of them. Gate network: learned router decides which experts handle each token. Sparse activation: only K of N experts active per token. Mixtral 8x7B: 8 experts, 2 active per token, 47B total params but 13B active compute. Load balancing: auxiliary loss prevents all tokens routing to same expert.

Self-supervised creates supervision from data structure itself -- no human labels. NLP: BERT predicts masked tokens, GPT predicts next token. Computer Vision: SimCLR (match two augmented views of same image), MAE (mask 75% of image patches, reconstruct them). Contrastive learning: pull same-sample representations together, push different samples apart. Enables learning from internet-scale unlabeled data.

Deep RL combines RL with neural networks for large or continuous state spaces. DQN: neural network approximates Q(s,a) for all actions. Experience Replay: store transitions, sample randomly -- breaks temporal correlations. Target Network: frozen network for Q-targets -- stabilizes training. Policy Gradient: directly learn policy pi(a|s). PPO (Proximal Policy Optimization): clips policy updates to prevent destructive large steps -- most widely used DRL algorithm, used in RLHF.

Attention lets the model focus on relevant parts of the input when processing each word. For every word, it computes a Query (what am I looking for?), compares it to Keys (what does each word offer?), and retrieves Values (the actual content) weighted by relevance. High attention score = highlighted in yellow. This lets "it" in "The trophy didn't fit in the suitcase because it was too big" correctly link to "trophy" not "suitcase."

A CNN slides small filters (like magnifying glasses) across the image, detecting features: edges in layer 1, shapes in layer 2, object parts in layer 3, full objects in deeper layers. This is convolution. Then pooling shrinks the map (keeping the strongest signals) to reduce computation. Stacking many convolution+pool layers builds a hierarchy from pixels to objects. The filters are learned — nobody hand-coded "detect an eye."

RNNs process sequence one step at a time, passing a hidden state forward — like reading a book one word at a time, updating your memory. Problem: by the time you reach word 100, word 1 is nearly forgotten (vanishing gradient). Transformers process all words simultaneously using attention — every word sees every other word at once. This parallelism made Transformers dramatically faster to train on GPUs and better at long-range dependencies. Result: RNNs are largely obsolete for NLP.

Pre-training: train a massive model on enormous general data (all of Wikipedia, billions of web pages). It learns language, facts, and reasoning. Fine-tuning: take that pre-trained model and train it a little more on your specific task (medical records, legal documents, customer emails). The chef already knows how to cook — you just teach them your menu. This is why GPT-4, Claude, and BERT work so well with relatively little task-specific data.

As data flows through deep network layers, values can explode (very large) or vanish (very small) — causing unstable training. Batch normalization normalizes the outputs of each layer to have mean=0 and variance=1 before passing to the next. Like adjusting the volume between every song in a playlist so they're all at the same level. Result: faster training, higher learning rates, and less sensitivity to weight initialization.

The Transformer has two stacks. Encoder: reads the input sequence, builds rich contextual representations using self-attention (every word attends to every other word). Decoder: generates the output sequence token by token, attending to both the encoder output and its own previous outputs. Attention mechanism: lets any position in the sequence directly influence any other, regardless of distance. This solved the long-range dependency problem that destroyed RNNs.

Before ResNet (2015), networks deeper than ~20 layers degraded — adding layers made things worse, not better. Residual connections add a shortcut: output = F(x) + x. The network learns the residual (the correction) rather than the full mapping. Even if F(x) learns nothing, the identity x passes through unchanged. This lets gradients flow directly through skip connections, keeping early layers learning. ResNet-152 has 152 layers. Modern LLMs have hundreds of transformer blocks — all using residual connections.

Diffusion models destroyed GANs as the state-of-the-art for image generation. Forward process: gradually add Gaussian noise to an image over 1000 steps until it looks like pure static. Reverse process: train a neural network to predict and remove that noise, step by step. At generation time: start with pure noise, apply the learned denoising process 1000 times — a coherent image emerges. More stable to train than GANs, better quality, and supports text conditioning (Stable Diffusion, DALL-E 2, Midjourney).

Base LLMs (pretrained on internet text) are knowledgeable but unhelpful — they complete text, not follow instructions. RLHF fixes this: Step 1 — collect human feedback on model outputs (which response is better?). Step 2 — train a reward model to predict human preferences. Step 3 — use reinforcement learning (PPO) to fine-tune the LLM to maximize reward. Result: a model that follows instructions, refuses harmful requests, and gives helpful responses. ChatGPT, Claude, and Gemini all use variants of RLHF.