What is the use of requires_grad in Tensors?

@aakashns Sir please explain requires_grad set to True. again …??? and its uses…??

requires_grad was difficult to understand, could you pl explain with simple practical use cases to start with?

@lakshaysethia @iamsusmita
if you set requires_grad to True to any tensor, then PyTorch will automatically track and calculate gradients for that tensor.
Now, why is this needed? Any Machine learning model will learn to predict better using an algorithm knows as Backpropagation, during backpropagation you need gradients to updates with new weights. This is done with the .backward() method, so during this operation tensors with requires_grad set to True will be used along with the tensor used to call backward() with to calculate the gradients.

Example:

import torch
x = torch.tensor(1.0, requires_grad = True)
z = x ** 3 # z=x^3
z.backward() #Computes the gradient 
print(x.grad.data) # this is dz/dx
5 Likes

When requires_grad is set to True for a variable, pytorch tracks every operation on it and when you finally use the backward() method for a tensor which came from operation of some variables you can see the derivative of that final variable w.r.t the variables you set requires grad to True.

import torch

x = torch.tensor(5)
w  = torch.tensor(3, requires_grad=True)
b = torch.tensor(2, requires_grad=True)

y = w*x + b  # y is 3*5 + 2 = 17
y.backward()

w.grad() # This returns the derivative of y w.r.t w which is equal to x=5
b.grad() # This returns the derivative of y w.r.t b which is  1
x.grad() # This will return None as we haven't set requires_grad to True for the tensor

This is useful as our neural network “learns” by using this derivatives to adjust the weights.
I hope this helps

5 Likes

Every Tensor has a flag: requires_grad that allows for fine grained exclusion of subgraphs from gradient computation and can increase efficiency.

requires_grad

If there’s a single input to an operation that requires gradient, its output will also require gradient. Conversely, only if all inputs don’t require gradient, the output also won’t require it. Backward computation is never performed in the subgraphs, where all Tensors didn’t require gradients.
hope this works if not jump to this link for the official doc https://pytorch.org/docs/stable/notes/autograd.html

3 Likes