References
Curated reading list of foundational and recent papers, books, and resources organised by topic.
Foundational Papers
| Year | Title | Authors | Key Contribution |
| 1943 | A Logical Calculus of Ideas Immanent in Nervous Activity | McCulloch & Pitts | First mathematical model of a neuron |
| 1950 | Computing Machinery and Intelligence | Alan Turing | The Turing Test; framework for machine intelligence |
| 1958 | The Perceptron | Rosenblatt | First trainable neural network |
| 1986 | Learning Representations by Back-propagating Errors | Rumelhart, Hinton, Williams | Backpropagation algorithm |
| 1989 | Multilayer Feedforward Networks are Universal Approximators | Hornik et al. | Universal approximation theorem |
| 1997 | Long Short-Term Memory | Hochreiter & Schmidhuber | LSTM — solved vanishing gradients in RNNs |
| 1998 | Gradient-Based Learning Applied to Document Recognition | LeCun et al. | LeNet; convolutional networks for images |
| 2006 | A Fast Learning Algorithm for Deep Belief Nets | Hinton, Osindero, Teh | Deep learning renaissance |
Deep Learning
| Year | Title | Authors | Key Contribution |
| 2012 | ImageNet Classification with Deep Convolutional Neural Networks | Krizhevsky, Sutskever, Hinton | AlexNet; deep learning on ImageNet |
| 2014 | Generative Adversarial Nets | Goodfellow et al. | GAN framework |
| 2014 | Dropout | Srivastava et al. | Dropout regularisation |
| 2015 | Batch Normalisation | Ioffe & Szegedy | Batch norm — accelerates training |
| 2015 | Deep Residual Learning for Image Recognition | He et al. | ResNet skip connections |
| 2016 | Identity Mappings in Deep Residual Networks | He et al. | Pre-activation ResNet |
| 2017 | Attention Is All You Need | Vaswani et al. | The Transformer architecture |
Language Models
| Year | Title | Authors | Key Contribution |
| 2018 | BERT | Devlin et al. | Bidirectional pre-training |
| 2018 | GPT | Radford et al. | Autoregressive LM pre-training |
| 2019 | GPT-2 | Radford et al. | Large-scale language model; emergent capabilities |
| 2020 | GPT-3 | Brown et al. | Few-shot learning in 175B model |
| 2020 | Scaling Laws for Neural Language Models | Kaplan et al. | Power-law scaling with compute, data, params |
| 2022 | Training Compute-Optimal Large Language Models | Hoffmann et al. | Chinchilla; optimal data-to-parameter ratio |
| 2022 | LLaMA | Touvron et al. (Meta) | Open efficient LLM |
| 2023 | LLaMA 2 | Touvron et al. | Instruction-tuned open LLM |
| 2023 | Mistral 7B | Jiang et al. | Sliding window attention, grouped query attention |
Alignment & Safety
| Year | Title | Authors | Key Contribution |
| 2022 | Training Language Models to Follow Instructions (InstructGPT) | Ouyang et al. | RLHF at scale |
| 2022 | Constitutional AI | Bai et al. (Anthropic) | RLAIF; principle-based alignment |
| 2023 | Direct Preference Optimisation | Rafailov et al. | DPO — simpler alternative to PPO |
| 2024 | The Claude Constitution | Anthropic | Principles for AI safety |
RAG & Retrieval
| Year | Title | Authors | Key Contribution |
| 2020 | Retrieval-Augmented Generation for Knowledge-Intensive NLP | Lewis et al. | The RAG paper |
| 2021 | Dense Passage Retrieval | Karpukhin et al. | DPR — bi-encoder retrieval |
| 2022 | BEIR: Heterogeneous Benchmark for IR | Thakur et al. | Retrieval evaluation benchmark |
| 2023 | RAGAS | Es et al. | Framework for evaluating RAG quality |
AI Agents
| Year | Title | Authors | Key Contribution |
| 2022 | Chain-of-Thought Prompting | Wei et al. | Step-by-step reasoning in LLMs |
| 2022 | ReAct: Synergizing Reasoning and Acting | Yao et al. | ReAct agent framework |
| 2023 | Toolformer | Schick et al. | LLMs learning to use tools |
| 2023 | AgentBench | Liu et al. | Agent evaluation benchmark |
| 2024 | SWE-bench | Jimenez et al. | Coding agent benchmark |
Books
| Title | Authors | Focus |
| Deep Learning | Goodfellow, Bengio, Courville | Comprehensive ML/DL theory |
| Pattern Recognition and Machine Learning | Bishop | Bayesian ML |
| Hands-On Machine Learning | Géron | scikit-learn, Keras, TensorFlow |
| Natural Language Processing with Transformers | Tunstall, von Werra, Wolf | HuggingFace ecosystem |
| Designing Machine Learning Systems | Huyen | Production ML engineering |
| Building LLM Applications | Larson | Applied LLM engineering |
Online Resources
| Resource | URL | Description |
| Anthropic Documentation | docs.anthropic.com | Claude API reference and guides |
| HuggingFace Docs | huggingface.co/docs | Transformers, Datasets, Hub |
| PyTorch Docs | pytorch.org/docs | Framework reference |
| Papers With Code | paperswithcode.com | Papers + implementations + benchmarks |
| Arxiv AI | arxiv.org/list/cs.AI | Preprint server for AI research |