(Some of) PyTorch for Dummies (by a dummy, abbreviated)

PyTorch is a popular Python library for working with tensors and neural networks. A lot of the magic of PyTorch is built around its automatic and efficient calculation of gradients whenever you run any tensor operations. This is noteworthy for fitting your neural network to your dataset through backpropogation.

Backpropogation involves calculating gradients on your network's loss function with respect to all the networks's weights individually for the purpose of iteratively adjusting those weights to improve the network's performance (and reduce the loss from the loss function as much as possible).

More explicitly and loosely speaking (through contrived and imperfect metaphors!), in the training of your network, your network will:

take a mock exam by making its best guesses (predict target values for your training data)
look at the answers and calculate a metric for how poorly it did (your loss function)
figure out what it studied too much or not enough for (backpropogation w.r.t. each attribute)
adjust how much it focuses on each exam section (adjusting weights up or down based on calculated gradients)
take the exam again and again, until they've come as close as possible to the answer key

Frankly, that's a mouthful for a dummy like me - and maybe for you if you're also new to deep learning! We will cover some of the more basic functions that allow you to better understand the above, at a conceptual level.

torch.from_numpy - takes an array and turns it into a tensor object, PyTorch's most fundamental class
tensor.numpy - takes a tensor and turns it back into an array
torch.nn.Linear - creates a linear model, from PyTorch's submodule nn
torch.nn.Functional.mse_loss - imports the popular loss function mean squared error, pre-defined
loss_fn.backward - calls on PyTorch to do all the heavy lifting of calculating gradients

Before we begin, let's install and import PyTorch

# Uncomment and run the appropriate command for your operating system, if required

# Linux / Binder
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

# Windows
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

# MacOS
# !pip install numpy torch torchvision torchaudio

# Import torch and other required modules
import torch
import numpy as np
import jovian

Function 1 - torch.from_numpy

There is a lot of overlap in functionality between Numpy and PyTorch. Sometimes it'll make sense to use one and sometimes it'll make sense to use the other. Fortunately, PyTorch was designed with this in mind and creating a tensor object from a Numpy array is incredibly simple. Note: The two share the same memory location so editing the array will update the tensor and vice versa.

# Example 1 - creating an array and a tensor from the array.
array1a = np.array([5, 15, 20.])
tensor1a = torch.from_numpy(array1a)

# Printing the type and value of both the array and tensor.
print(type(array1a), array1a, array1a.dtype)
print(type(tensor1a), tensor1a)

<class 'numpy.ndarray'> [ 5. 15. 20.] float64
<class 'torch.Tensor'> tensor([ 5., 15., 20.], dtype=torch.float64)