NeuralNetworks-Zero-To-Hero

Neural Networks: Zero to Hero is a course on deep learning fundamentals by the renowned AI researcher and educator Andrej Karpathy. This repository contains my personal lecture notes and exercise solutions for the course, which covers a wide range of topics such as neural networks, backpropagation, wavenet, GPT & more.

You can find the lecture notes and exercise here: https://anubhavmaity.github.io/NeuralNetworks-Zero-To-Hero/

If you want to play with lecture notes and exercise in Google Colab then head over to the Contents Section.

The youtube lectures of the course can be found here: https://karpathy.ai/zero-to-hero.html

Table of Contents

Course Overview

The “Neural Networks: Zero to Hero” course covers a range of deep learning topics, including:

- Neural Networks and Backward propogation
- Language Modeling
- MultiLayer Perceptron
- Activations & Gradients
- Batch normalizations
- Wavenet
- GPT
- And more!

Contents

This repository contains my personal notes and exercise solutions for each lecture in the course. The lecture notes provide a detailed summary of each lecture, including key concepts, equations, diagrams, and examples. The exercise solutions include code and written explanations, helping you to deepen your understanding of the material and apply it in practice.

Lecture 1: The spelled-out intro to neural networks and backpropagation: building micrograd

Style Colab Github
Lecture Note Open In Colab Open In GitHub
Exercise Open In Colab Open In GitHub

Lecture 2: The spelled-out intro to language modeling: building makemore

Style Colab Github
Lecture Note Open In Colab Open In GitHub
Exercise Open In Colab Open In GitHub

Lecture 3: Building makemore Part 2: MLP

Style Colab Github
Lecture Note Open In Colab Open In GitHub
Exercise Open In Colab Open In GitHub

Lecture 4: Building makemore Part 3: Activations & Gradients, BatchNorm

Style Colab Github
Lecture Note Open In Colab Open In GitHub
Exercise Open In Colab Open In GitHub

Lecture 5: Building makemore Part 4: Becoming a Backprop Ninja

Style Colab Github
Lecture Note & Exercise Open In Colab Open In GitHub

Lecture 6: Building makemore Part 5: Building a WaveNet

Style Colab Github
Lecture Note Open In Colab Open In GitHub
Exercise Open In Colab Open In GitHub

Usage

You can use this repository to review and reference the lecture notes, as well as to deepen your understanding of the concepts through the exercise solutions. You can also modify and build upon the code to suit your own needs.

If you use this repository in your project, please attribute it to me (Anubhav Maity) and include a link to the original repository:

  • Repository: https://github.com/anubhavmaity/NeuralNetworks-Zero-To-Hero/
  • Author: Anubhav Maity

Contributing

Contributions to this repository are welcome! If you notice any errors or have suggestions for improving the content, feel free to submit a pull request or open an issue.

The docs are generated from Jupyter Notebooks using nbdev. To make any changes in the doc, you can make the changes in the Jupyter Notebook and the Github Action will generate the docs from the notebook for you.

LICENSE

This repository is licensed under the Apache License 2.0. See the LICENSE file in the repo for details.