Learn data science and machine learning by building real-world projects on Jovian

Deep Learning with PyTorch

Assignment1 - All about torch.Tensor

PyTorch is an optimized tensor library for deep learning using GPUs and CPUs.

The torch package contains data structures for multi-dimensional tensors and mathematical operations. Additionally, it provides many utilities for efficient serializing of Tensors and arbitrary types, and other useful utilities.

A torch.Tensor is a multi-dimensional matrix containing elements of a single data type. Torch defines nine CPU tensor types and nine GPU tensor types.

The following torch.Tensor functions are experimented as part of this assignment:

  • torch.abs() : Computes the element-wise absolute value of the given input tensor
  • torch.bitwise_and() : Computes the bitwise AND of input and other.
  • torch.reshape() : Returns a tensor with the same data and number of elements as input, but with the specified shape.
  • torch.unbind() : Removes a tensor dimension.
  • torch.take() : Returns a new tensor with the elements of input at the given indices.
In [1]:
# Import torch and other required modules
import torch

Function 1 - torch.abs()

Computes the element-wise absolute value of the given input tensor.

Parameters:

  • input(Tensor) : the input tensor
  • output(Tensor, Optional) : the output tensor
In [2]:
# Example 1
torch.abs(torch.tensor([-1, -2, 3]))
Out[2]:
tensor([1, 2, 3])
In [3]:
# Example 2
torch.abs(torch.tensor([-1, -2, -(-3)]))
Out[3]:
tensor([1, 2, 3])
In [4]:
# Example 3
torch.abs(torch.tensor([1, 2, 3]))
Out[4]:
tensor([1, 2, 3])

Use this function when their is need for absolute values. For ex: during computation of loss of a model etc.

Function 2 - torch.bitwise_and()

Computes the bitwise AND of input and other. The input tensor must be of integral or Boolean types. For bool tensors, it computes the logical AND.

Parameters:

  • input(Tensor) : the input tensor
  • other : the second input tensor
  • output(Tensor, Optional) : the output tensor
In [5]:
# Example 1
torch.bitwise_and(torch.tensor([True, True, False]), torch.tensor([False, True, False]))
Out[5]:
tensor([False,  True, False])

The input and other are bool tensors in this case.

In [6]:
# Example 2
torch.bitwise_and(torch.tensor([1, 2, 3], dtype=torch.int8), torch.tensor([1, 2, 3], dtype=torch.int8))
Out[6]:
tensor([1, 2, 3], dtype=torch.int8)
In [7]:
# Example 3
torch.bitwise_and(torch.tensor([-1, 2, 3], dtype=torch.int8), torch.tensor([1, 0, 3], dtype=torch.int8))
Out[7]:
tensor([1, 0, 3], dtype=torch.int8)

The input and other are integral tensors in this case. Presence of 0 in any position results in a 0 in the output.

Use this function to find bitwise and.

Function 3 - torch.reshape()

Returns a tensor with the same data and number of elements as input, but with the specified shape. When possible, the returned tensor will be a view of input. Otherwise, it will be a copy. Contiguous inputs and inputs with compatible strides can be reshaped without copying, but you should not depend on the copying vs. viewing behavior.

Parameters:

  • input(Tensor) : the input tensor to be reshaped
  • shape(tuple of python:ints) : the new shape
In [8]:
# Example 1
a = torch.arange(4.)
torch.reshape(a, (2, 2))
Out[8]:
tensor([[0., 1.],
        [2., 3.]])
In [9]:
# Example 2
b = torch.tensor([[0, 1], [2, 3]])
torch.reshape(b, (-1,))
Out[9]:
tensor([0, 1, 2, 3])
In [10]:
# Example 3
c = torch.tensor([[0, 1], [2, 3]])
#torch.reshape(a,(2,3)) # this is invalid for input size 4
torch.reshape(c,(4,1))
Out[10]:
tensor([[0],
        [1],
        [2],
        [3]])

Use this function when a tensor needs to be reshaped to perform operations such as multiplication etc.

Function 4 - torch.unbind()

Removes a tensor dimension. Returns a tuple of all slices along a given dimension, already without it.

Parameters:

  • input(Tensor) : the input tensor to unbind
  • dim(int) : dimension to remove
In [11]:
# Example 1
torch.unbind(torch.tensor([[1, 2, 3],[4, 5, 6],[7, 8, 9]]))
Out[11]:
(tensor([1, 2, 3]), tensor([4, 5, 6]), tensor([7, 8, 9]))
In [12]:
# Example 2
torch.unbind(torch.tensor([[1, 2, 3]]))
Out[12]:
(tensor([1, 2, 3]),)
In [13]:
# Example 3
torch.unbind(torch.tensor([[1, 2],[4, 5]]))
Out[13]:
(tensor([1, 2]), tensor([4, 5]))

Use this function when dimension of tensor needs to be reduced.

Function 5 - torch.take()

Returns a new tensor with the elements of input at the given indices. The input tensor is treated as if it were viewed as a 1-D tensor. The result takes the same shape as the indices.

Parameters:

  • input(Tensor) : the input tensor
  • indices (LongTensor) : the indices into tensor
In [14]:
# Example 1
src = torch.tensor([[4, 3, 5],[6, 7, 8]])
torch.take(src, torch.tensor([0, 2, 5]))
Out[14]:
tensor([4, 5, 8])
In [15]:
# Example 2
src = torch.tensor([10,11,12])
torch.take(src, torch.tensor([0,1]))
Out[15]:
tensor([10, 11])
In [16]:
# Example 2
src = torch.tensor([10,11,12])
#torch.take(src, torch.tensor([3])) # this example results in a error
torch.take(src, torch.tensor([2]))
Out[16]:
tensor([12])

Use this function when values are to be retrieved using indices

Conclusion

We have experimented with few pytorch functions that manipulate tensors to generate the desired results.

Reference Links

Provide links to your references and other interesting articles about tensors

In [17]:
!pip install jovian --upgrade --quiet
In [18]:
import jovian
In [ ]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Please enter your API key ( from https://jovian.ml/ ): API KEY:
In [ ]: