Tensor Operations

Subtitle

PyTorch is an Open Source Machine Learning library used for applications such as Natural Language Processing, Computer Vision etc. The library torch is imported to utilise the API PyTorch provides.

backward()
fill_diagonal()
function 3
function 4
function 5

# Import torch and other required modules
import torch
import numpy as np

Function 1 - backward(gradient=None, retain_graph=None, create_graph=True)

The backward() function creates gradient tensors with respect to the variables which have the required_grad bool set to true. Since the function accumulates gradients in the graph leaves, the gradients of the leaves might have to be set to zero before finding the gradient again.

# Example 1 - working
t = torch.tensor(3.)
w = torch.tensor(4., requires_grad=True)
b = torch.tensor(7., requires_grad=True)
y = w*t + b
y, w, t, b

(tensor(19., grad_fn=<AddBackward0>),
 tensor(4., requires_grad=True),
 tensor(3.),
 tensor(7., requires_grad=True))

A simple demonstration with just 3 variables, and setting two of them to required_grad = True.