Deep Learning Illustrated, Part 2: How Does a Neural Network Learn? | by Shreya Rao | Feb, 2024

An illustrated and intuitive guide on how Neural Networks learn

Welcome to Part 2 of the Deep Learning Illustrated series. In the previous article (definitely read that first!), we covered how a neural network works and how a trained neural network makes predictions.

In this article, we’ll delve into the training process and explore how a neural network learns.

📣 If you haven’t read my previous articles, I highly recommend you start with my series of articles covering the basics of machine learning, specifically the one on Gradient Descent because you’ll find that a lot of the material covered there is relevant here.

Machine Learning Starter Pack

Let’s say we want to create a neural network that predicts the daily revenue of ice cream sales using the features temperature and day of the week.

This is the (synthetic) training dataset we’re using:

To build a neural network, as we learned in the previous article, we need to first decide on its architecture. This includes determining the number of hidden layers, the number of neurons in each layer, and the activation function of each neuron.

Let’s say we decided our architecture is: 1 hidden layer with 2 neurons, and 1 output neuron, all using the rectifier activation function.

Terminology segue: In the previous article, we learned about using subscripts to differentiate between different weights. We’re sticking with the same convention here, and in addition, we‘ll use superscripts to…