🧑‍🏫 59 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
13039 1371 236 67

Twitter Sponsor Deep Learning Paper Implementations

This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations,

The website renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better.


We are actively maintaining this repo and adding new implementations almost weekly. Twitter for updates.

Paper Implementations

✨ Transformers

✨ Eleuther GPT-NeoX

✨ Diffusion models

✨ Generative Adversarial Networks

✨ Recurrent Highway Networks


✨ HyperNetworks - HyperLSTM

✨ ResNet

✨ ConvMixer

✨ Capsule Networks

✨ U-Net

✨ Sketch RNN

✨ Graph Neural Networks

✨ Counterfactual Regret Minimization (CFR)

Solving games with incomplete information such as poker with CFR.

✨ Reinforcement Learning

✨ Optimizers

✨ Normalization Layers

✨ Distillation

✨ Adaptive Computation

✨ Uncertainty

✨ Activations

✨ Langauge Model Sampling Techniques

✨ Scalable Training/Inference

Highlighted Research Paper PDFs


pip install labml-nn


If you use this for academic research, please cite it using the following BibTeX entry.

 author = {Varuna Jayasiri, Nipun Wijerathne},
 title = { Annotated Paper Implementations},
 year = {2020},
 url = {},

Other Projects

🚀 Trending Research Papers

This shows the most popular research papers on social media. It also aggregates links to useful resources like paper explanations videos and discussions.


This is a library that let's you monitor deep learning model training and hardware usage from your mobile phone. It also comes with a bunch of other tools to help write deep learning code efficiently.