Learn practical skills, build real-world projects, and advance your career

Training Deep Neural Networks on a GPU with PyTorch

Part 4 of "PyTorch: Zero to GANs"

This post is the fourth in a series of tutorials on building deep learning models with PyTorch, an open source neural networks library. Check out the full series:

  1. PyTorch Basics: Tensors & Gradients
  2. Linear Regression & Gradient Descent
  3. Image Classfication using Logistic Regression
  4. Training Deep Neural Networks on a GPU
  5. Coming soon.. (CNNs, RNNs, GANs etc.)

In the previous tutorial, we trained a logistic regression model to identify handwritten digits from the MNIST dataset with an accuracy of around 86%.

alt

However, we also noticed that it's quite difficult to improve the accuracy beyond 87%, due to the limited power of the model. In this post, we'll try to improve upon it using a feedforward neural network.

!pip install jovian
Collecting jovian Downloading https://files.pythonhosted.org/packages/96/3c/472d7af5c9724ae4832537bbd3101d28247eabe4c1ce07cf147fcafa1093/jovian-0.1.89-py3-none-any.whl (42kB) 100% |████████████████████████████████| 51kB 2.4MB/s ta 0:00:011 Requirement already satisfied: requests in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jovian) (2.22.0) Collecting pyyaml (from jovian) Downloading https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz (265kB) 100% |████████████████████████████████| 266kB 5.6MB/s ta 0:00:011 Collecting uuid (from jovian) Downloading https://files.pythonhosted.org/packages/ce/63/f42f5aa951ebf2c8dac81f77a8edcc1c218640a2a35a03b9ff2d4aa64c3d/uuid-1.30.tar.gz Requirement already satisfied: idna<2.9,>=2.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->jovian) (2.8) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->jovian) (3.0.4) Requirement already satisfied: certifi>=2017.4.17 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->jovian) (2019.3.9) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests->jovian) (1.25.3) Building wheels for collected packages: pyyaml, uuid Building wheel for pyyaml (setup.py) ... done Stored in directory: /home/jovyan/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030 Building wheel for uuid (setup.py) ... done Stored in directory: /home/jovyan/.cache/pip/wheels/2a/80/9b/015026567c29fdffe31d91edbe7ba1b17728db79194fca1f21 Successfully built pyyaml uuid Installing collected packages: pyyaml, uuid, jovian Successfully installed jovian-0.1.89 pyyaml-5.1.2 uuid-1.30
import jovian
jovian.commit()
[jovian] Saving notebook..

System Setup

If you want to follow along and run the code as you read, you can clone this notebook, install the required dependencies using conda, and start Jupyter by running the following commands on the terminal:

pip install jovian --upgrade    # Install the jovian library 
jovian clone fdaae0bf32cf4917a931ac415a5c31b0  # Download notebook
cd 04-feedforward-nn            # Enter the created directory 
jovian install                  # Install the dependencies
conda activate 04-feedfoward-nn # Activate virtual env
jupyter notebook                # Start Jupyter

On older versions of conda, you might need to run source activate 04-feedfoward-nn to activate the virtual environment. For a more detailed explanation of the above steps, check out the System setup section in the first notebook.