Memory tricks that make AI, ML & Deep Learning click

Artificial Narrow Intelligence (ANI): all current AI — excels at ONE specific task. Cannot transfer skills. ChatGPT writes text but cannot drive a car. Artificial General Intelligence (AGI): hypothetical — human-level ability across ALL tasks. Does not yet exist. Artificial Superintelligence (ASI): theoretical — surpasses human intelligence in every domain. Key point: everything deployed today is Narrow AI.

The three terms are nested, not synonyms. Artificial Intelligence (AI): the broadest category — any technique enabling machines to mimic human intelligence. Machine Learning (ML): a subset of AI where machines learn patterns from data without explicit programming. Deep Learning (DL): a subset of ML using multi-layered neural networks. Key exam point: everything deployed today — ChatGPT, self-driving cars, medical imaging AI — is Narrow AI.

Picture three vivid scenes. Supervised: a TEACHER with the answer key — every example has a correct label. Learns to match inputs to outputs (spam filter, image classifier). Unsupervised: an EXPLORER dropped in an unknown jungle with no map — discovers hidden structure with no labels (clustering, topic modeling). Reinforcement: a DOG trained with treats — takes an action, gets a reward or correction, learns what works (game AI, robotics, RLHF for ChatGPT).

Bias: error from wrong assumptions — model too simple, misses real patterns. A linear model on curved data has high bias (underfitting). Variance: error from sensitivity to training noise — model memorizes training data, fails on new data (overfitting). Goal: find the sweet spot — right complexity for the data. Underfitting fix: more complex model, more features. Overfitting fix: more data, regularization, simpler model.

Visualize a river of regret flowing upstream. Forward pass: data flows forward through the network producing a prediction. Loss: the network measures how wrong it was — this is the regret. Backward pass (backpropagation): the regret flows BACKWARD using the chain rule — assigns blame to each weight. Weight update: each weight is nudged to reduce future regret (gradient descent). Repeat thousands of times. The river of regret gets smaller as the network learns.

Dartmouth Conference (1956): John McCarthy coins 'Artificial Intelligence' — field is born. AI Winters (1970s, 1980s): funding collapses when expectations outstrip results — twice. Deep Learning Breakthrough (2012): AlexNet wins ImageNet by a massive margin using deep CNNs and GPUs — modern AI era begins. LLM Era (2017–present): Transformers → BERT → GPT → ChatGPT → explosion of generative AI. Key names: Turing (test), McCarthy (AI term), Hinton/LeCun/Bengio (deep learning pioneers — 2018 Turing Award).

Token: the unit of text an LLM processes — roughly ¾ of a word. Temperature: controls randomness — 0.0 = deterministic/predictable, 1.0+ = creative/random. Context window: maximum tokens the model can see at once — Claude: 200K, Gemini 1.5: 1M. Hallucination: when an LLM confidently states false information — a fundamental limitation, not a bug. Models predict plausible next tokens, not verified facts.

Fairness: AI should not discriminate based on protected characteristics. Accountability: when AI causes harm, who is responsible? Transparency (Explainability): can we understand why the model made a decision? Ethics: broader societal impact — jobs, privacy, power, autonomy, surveillance. These principles are in tension — no pure algorithmic solution. EU AI Act (2024) is the first comprehensive AI regulation globally.

Classification: predict a category (spam/not spam, cat/dog). Output = discrete label. Regression: predict a number (house price, temperature). Output = continuous value. Clustering: group similar items without labels (customer segments). Unsupervised. Generation: produce new content (text, images, code). LLMs and diffusion models. Knowing the task type immediately narrows which algorithms, loss functions, and evaluation metrics are appropriate.

Linear Algebra: vectors (data points), matrices (weight matrices), dot products (similarity), eigenvalues (PCA). Calculus: derivatives (gradients), chain rule (backpropagation), gradient descent finds minimum of loss function. Matrix Operations: matrix multiplication is the core operation of neural networks — GPUs are optimized for it. Probability: Bayes theorem, probability distributions, expected value — underpins everything from Naive Bayes to diffusion models.

Frameworks: PyTorch (dominant in research and industry), TensorFlow/Keras (Google, production deployment). Hardware: NVIDIA GPUs (CUDA), Google TPUs, Apple Silicon. Open-source models: Hugging Face Hub — 500K+ pretrained models (BERT, Llama, Stable Diffusion). Libraries: scikit-learn (classical ML), NumPy/Pandas (data), Matplotlib (visualization). For beginners: scikit-learn for ML fundamentals, PyTorch + Hugging Face for deep learning.

AI is genuinely excellent at: pattern recognition in images and audio, language generation and translation, game playing, protein structure prediction (AlphaFold), recommendation systems, code generation, speech recognition. AI still struggles with: robust common-sense reasoning, reliable arithmetic and logic, consistent long-term planning, physical world understanding, verified factual accuracy. Key insight: AI excels at interpolating within its training distribution — it struggles when situations require true generalization beyond what it has seen.

Picture a city (AI) — it contains many neighborhoods. One neighborhood is Machine Learning. Inside ML, one specific house is Deep Learning. You cannot have the house without the neighborhood, or the neighborhood without the city. Zoom in = more specialized. Zoom out = broader category. Every time you see these three terms, picture the nested city.

SAIL through AI history: Symbolic AI (1950s–80s — rules-based expert systems), AI Winter (funding dried up twice when promises went unmet), ImageNet Leap (2012 — deep learning crushed image recognition and changed everything), LLM Era (2017+ — Transformers, GPT, ChatGPT, the AI explosion we are living through now). Each era directly caused the next.

Traditional software: a programmer writes every rule explicitly. AI/ML: the system finds rules itself from examples. Traditional: IF temperature > 100 THEN alert. AI: show it 1 million patient records and it learns what danger looks like on its own. The shift from RULES IN to PATTERNS OUT is the foundational idea behind all of machine learning.

Every ML project splits data into sets: Training set — the model learns from this. Validation set — used during development to tune hyperparameters (never trained on). Test set — locked away until the very end to measure real-world performance (touched only once). Mixing them causes data leakage and falsely inflated scores — a classic exam trap worth knowing cold.