annotated_deep_learning_paper_implementations
🧑🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optim
What is annotated_deep_learning_paper_implementations?
🧑🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
Overview of annotated_deep_learning_paper_implementations
What it is
This is a collection of simple PyTorch implementations of neural networks and related algorithms, documented with explanations. The website renders these as side-by-side formatted notes. The repository is actively maintained and new implementations are added almost weekly.
Key features
- 60+ implementations/tutorials of deep learning papers with side-by-side notes.
- Covers transformers (original, XL, switch, feedback, ViT, etc.), optimizers (Adam, AdaBelief, Sophia, etc.), GANs (CycleGAN, StyleGAN2, etc.), reinforcement learning (PPO, DQN), capsule networks, distillation, and more.
- Includes implementations for diffusion models (DDPM, DDIM, latent diffusion, stable diffusion), graph neural networks (GAT, GATv2), and normalization layers.
- Provides language model sampling techniques (greedy, temperature, top-k, nucleus).
- Features scalable training/inference with Zero3 memory optimizations.
Use cases
Researchers and practitioners can use these implementations to understand deep learning papers through annotated code. The side-by-side notes help in learning the algorithms. It is suitable for studying transformers, optimizers, GANs, reinforcement learning, and other neural network architectures.
Getting started
bashpip install labml-nn
License and pricing
The license is MIT. The project is open source.
Based on the project's public README and repository metadata.