Jovian
⭐️
Sign In

First 5 things to start learning PyTorch Tensors

lets begin our TorchAdventure! with this 11 basic functions distributed in the following sections:

  • Creating tensors: Empty() and Zeros()
  • measuring tensors: Size() and Shape(), Sum() and Dimensions
  • changing and copying tensors: Reshape(), view() and randm()
  • modifying tensors: Unsqueeze()
  • comparing tensors: Element-Wise Equality: Eq

Before we begin, let's install and import PyTorch

In [ ]:
# Uncomment and run the appropriate command for your operating system, if required

# Linux / Binder
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

# Windows
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

# MacOS
# !pip install numpy torch torchvision torchaudio
In [2]:
# Import torch and other required modules
import torch

Function 1 - Empty and Zeros - how to initialize tensors

We need to start workng with the basics pytorch functions, and the first thing is create our matrix

In [4]:

# Creates a 3 x 2 matrix which is empty
a = torch.empty(3, 2)
print(a)
tensor([[1.5842e-35, 0.0000e+00], [4.4842e-44, 0.0000e+00], [ nan, 0.0000e+00]])

Here is how PyTorch is allocating memory for this tensor. Whatever, it will not erase anything previous content in the memory.

by default, when you initializes a tensor is used the float32 dtype. you can review it here: https://pytorch.org/docs/stable/generated/torch.set_default_tensor_type.html#torch.set_default_tensor_type

But you can also start working woth torch.zeros

In [6]:
# Applying the zeros function and 
# storing the resulting tensor
 
a = torch.zeros([3, 5]) 
print("a = ", a) 
  
b = torch.zeros([2, 4]) 
print("b = ", b) 
  
c = torch.zeros([4, 1]) 
print("c = ", c) 
  
d = torch.zeros([4, 4, 2]) 
print("d = ", d) 
a = tensor([[0., 0., 0., 0., 0.], [0., 0., 0., 0., 0.], [0., 0., 0., 0., 0.]]) b = tensor([[0., 0., 0., 0.], [0., 0., 0., 0.]]) c = tensor([[0.], [0.], [0.], [0.]]) d = tensor([[[0., 0.], [0., 0.], [0., 0.], [0., 0.]], [[0., 0.], [0., 0.], [0., 0.], [0., 0.]], [[0., 0.], [0., 0.], [0., 0.], [0., 0.]], [[0., 0.], [0., 0.], [0., 0.], [0., 0.]]])

this tensor is filled with zeros, so PyTorch allocates memory and zero-initializes the tensor elements inside

You cannot change the way a tensor is created, if you create a zeros tensor, make sure is not referenced to any other.

In [34]:
# correctly initialized
a = torch.empty(3,3)
print(a)

#also correct
b = torch.empty(1,2,3)

print(b)
tensor([[2.4258e-35, 0.0000e+00, 1.5975e-43], [1.3873e-43, 1.4574e-43, 6.4460e-44], [1.4153e-43, 1.5274e-43, 1.5695e-43]]) tensor([[[2.3564e-35, 0.0000e+00, 1.4013e-45], [1.4574e-43, 6.4460e-44, 1.4153e-43]]])
In [35]:
# incorrect reference, you must create a new one
c = torch.zeros(b,1) 
print("c = ", c) 
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-35-c044e2995879> in <module>() 1 # incorrect reference, you must create a new one ----> 2 c = torch.zeros(b,1) 3 print("c = ", c) TypeError: zeros() received an invalid combination of arguments - got (Tensor, int), but expected one of: * (tuple of ints size, *, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad) * (tuple of ints size, *, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)

lets review the zero method closely:

Our documentation says:

Syntax: torch.zeros(size, out=None)

Parameters:
size: a sequence of integers defining the shape of the output tensor
out (Tensor, optional): the output tensor

Return type: A tensor filled with scalar value 0, of same shape as size.

torch.zeros and torch.empty are the first functions to start working with pytorch tensors and learning a little bit of matrix and vectors

Let's save our work using Jovian before continuing.

In [36]:
!pip install jovian --upgrade --quiet
|████████████████████████████████| 71kB 3.1MB/s eta 0:00:011 Building wheel for uuid (setup.py) ... done
In [37]:
import jovian
In [104]:
jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')
[jovian] Detected Colab notebook... [jovian] Uploading colab notebook to Jovian... [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/cortezb-claro/first-5-things-to-start-learning-pytorch-tensors

Function 2 - Tensor Size, Shape and Dimension Operations

Lets understand dimensions in Pytorch.

Now lets create some tensors and determine the size of every one

In [45]:
import torch 
# Create a tensor from data
c = torch.tensor([[3.2 , 1.6, 2], [1.3, 2.5 , 6.9]])
print(c)

print(c.size())
tensor([[3.2000, 1.6000, 2.0000], [1.3000, 2.5000, 6.9000]]) torch.Size([2, 3])

Lets see torch.shape and take a closer look at how size is given here

Now in the next example lets use shape functions to get the size of a tensor

In [47]:
x = torch.tensor([
     [1, 2, 3],
     [4, 5, 6]
   ])
x.shape
torch.Size([2, 3])
Out[47]:
torch.Size([2, 3])

Shape and Size give us the same correct dimensions of the tensor.

in this case we have a 3D-tensor (with 3 dimensions)

Dimension 0 Dimension 1 and Dimension 2

lets create a new tensor:

In [49]:
y = torch.tensor([
     [
       [1, 2, 3],
       [4, 5, 6]
     ],
     [
       [1, 2, 3],
       [4, 5, 6]
     ],
     [
       [1, 2, 3],
       [4, 5, 6]
     ]
   ])

y.shape
Out[49]:
torch.Size([3, 2, 3])

Lets se how we can make operations using a 3d tensor now for every dimension layer and see how it behaves

In [57]:
sum1 = torch.sum(y, dim=0)
print(sum1)

sum2 = torch.sum(y, dim=1)
print(sum2)

sum3 = torch.sum(y, dim=2)
print(sum3)
tensor([[ 3, 6, 9], [12, 15, 18]]) tensor([[5, 7, 9], [5, 7, 9], [5, 7, 9]]) tensor([[ 6, 15], [ 6, 15], [ 6, 15]])

we can see now a 3d tensor is more complicated as we advance, and we can perform custom operations within every dimension

its limited right now to 3 dims so we cannot perform this:

In [56]:
sum2 = torch.sum(y, dim=3)
print(sum1)

--------------------------------------------------------------------------- IndexError Traceback (most recent call last) <ipython-input-56-7caf723c6102> in <module>() ----> 1 sum2 = torch.sum(y, dim=3) 2 print(sum1) IndexError: Dimension out of range (expected to be in range of [-3, 2], but got 3)

to accomplish adding more layers of dimensions we can review the unsqueeze functions --> .unsqueeze() but for now leta go to the next basic function

In [58]:
jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')
[jovian] Detected Colab notebook... [jovian] Uploading colab notebook to Jovian... [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/cortezb-claro/5-basic-functions-to-learn-pytorch-tensors-d6973

Function 3 - Reshape, View and Random

there is a function in NunPy called ndarray.reshape() for reshaping an array.

Now in pytorch, there is torch.view(tensor) for the same purpose, but at the same time, there is also a torch.reshape(tensor).

lets figure out the differences between them and when you should use either of them.

First of all we are going to use a new functions for randomize our tensor.

In [69]:
import torch 
x = torch.randn(5, 3)

print(x)
tensor([[-0.5793, 0.6999, 1.7417], [-0.9810, 0.0626, 0.4100], [-0.6519, -0.0595, -1.2156], [-0.3973, -0.3103, 1.6253], [ 0.2775, -0.0045, -0.2985]])

this is basic usage of torch.randm, so now lets use View from another variable "y" and describe all elements

In [74]:
# Return a view of the x, but only having 
# one dimension and max number of elements
y = x.view(5 * 3)
 
#lets see the size of every tensor
print("lets see the size of every tensor")
print('Size of x:', x.size())
print('Size of y:', y.size())
 
#and the elements of very tensor to compare
print("and the elements of very tensor to compare")
print("X:", x)
print("Y:", y)
 

lets see the size of every tensor Size of x: torch.Size([5, 3]) Size of y: torch.Size([15]) and the elements of very tensor to compare X: tensor([[-0.5793, 0.6999, 1.7417], [-0.9810, 0.0626, 0.4100], [-0.6519, -0.0595, -1.2156], [-0.3973, -0.3103, 1.6253], [ 0.2775, -0.0045, -0.2985]]) Y: tensor([-0.5793, 0.6999, 1.7417, -0.9810, 0.0626, 0.4100, -0.6519, -0.0595, -1.2156, -0.3973, -0.3103, 1.6253, 0.2775, -0.0045, -0.2985])

take a look at Y tensor, it only has 1 dimension. so Viewing another tensor may be difficult for some operations

Now lets use Reshape to replicate the exact dimensions of X

In [61]:
# Get back the original tensor with reshape()
z = y.reshape(5, 3)
print(z)

tensor([[-0.2927, 0.0329, 0.8485], [ 1.9581, 0.8313, -0.1529], [-0.2330, -0.1887, 1.8206], [ 1.5252, 1.0909, 0.0547], [-0.1231, -0.4238, -0.6724]])

we cannot only reshape the original, we can also change the dimensions with some limited actions related to the maximum elements:

first lets reshape to 1 more dimension

In [77]:
# Get back the original tensor with reshape()
z = y.reshape(15)
print(z)

#reshaping 15 elements, 1 dim
z = y.reshape(3*5)
print(z)

#reshaping in different order, 3 dimensions
z = y.reshape(3,5)
print(z)

#Reshaping with more dimensions but its 15 elements always
z = y.reshape(3,5,1)
print(z)



tensor([-0.5793, 0.6999, 1.7417, -0.9810, 0.0626, 0.4100, -0.6519, -0.0595, -1.2156, -0.3973, -0.3103, 1.6253, 0.2775, -0.0045, -0.2985]) tensor([-0.5793, 0.6999, 1.7417, -0.9810, 0.0626, 0.4100, -0.6519, -0.0595, -1.2156, -0.3973, -0.3103, 1.6253, 0.2775, -0.0045, -0.2985]) tensor([[-0.5793, 0.6999, 1.7417, -0.9810, 0.0626], [ 0.4100, -0.6519, -0.0595, -1.2156, -0.3973], [-0.3103, 1.6253, 0.2775, -0.0045, -0.2985]]) tensor([[[-0.5793], [ 0.6999], [ 1.7417], [-0.9810], [ 0.0626]], [[ 0.4100], [-0.6519], [-0.0595], [-1.2156], [-0.3973]], [[-0.3103], [ 1.6253], [ 0.2775], [-0.0045], [-0.2985]]])

Now lets reshape exceeding the number of elements in the tensor:

In [78]:
z = y.reshape(16)
print(z)
--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-78-c7ae174fce73> in <module>() ----> 1 z = y.reshape(16) 2 print(z) RuntimeError: shape '[16]' is invalid for input of size 15

it will fail also for z = y.reshape(3*6) or putting more elements that does not exist in the tensor.

Now lets keep going to the next section.

In [109]:
jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')
[jovian] Detected Colab notebook... [jovian] Uploading colab notebook to Jovian... [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/cortezb-claro/first-5-things-to-start-learning-pytorch-tensors-2047c

Function 4 - Unsqueeze()

Mainly, it allows us to add more dimensions at specific index you define.

lets take a look:

In [84]:
import torch

#dim=1, that is (3)
x = torch.tensor([1, 2, 3])
print('x: ', x)
print('x.size: ', x.size())

#x1 becomes a matrix of (3,1)
x1 = torch.unsqueeze(x, 1)
print('x1: ', x1)
print('x1.size: ', x1.size())



x: tensor([1, 2, 3]) x.size: torch.Size([3]) x1: tensor([[1], [2], [3]]) x1.size: torch.Size([3, 1]) x2: tensor([[1, 2, 3]]) x2.size: torch.Size([1, 3])

Our initial tensor is Tensor([1,2,3]), and the output size is [3].

And then we proceed with adding 1 dimenson with unsqueeze operation, namely torch.unsqueeze(x, 1), the size of x1 is [3,1].

In [85]:
#x2 becomes a matrix of (1,3)
x2 = torch.unsqueeze(x, 0)
print('x2: ', x2)
print('x2.size: ', x2.size())
x2: tensor([[1, 2, 3]]) x2.size: torch.Size([1, 3])

When we perform torch.unsqueeze(x, 0), the size of x2 is [1,3].

In [89]:
# Example 3 - breaking (to illustrate when it breaks)


print(x.unsqueeze())

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-89-5a320a828907> in <module>() 2 3 ----> 4 print(x.unsqueeze()) TypeError: unsqueeze() missing 1 required positional arguments: "dim"

We must specified the dimension correctly, although we are just adding 1 dim, it is necessary to put like this: x.unqueeze(0)

In [ ]:
jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')

Function 5 - Torch Eq (Element Wise equality)

This function is under comparison category and it computes equality in element-wise and returns a boolean tensor. True if equal, False otherwise.

Lets review how we can operate with different sizes of tensors:

In [103]:
 # Example 1 - working 
x1 = torch.tensor([[1, 2], [3, 4.]])
x2 = torch.tensor([[2, 2], [2, 5]])
x3 = torch.randn(3,5)

#size x1 and z2
print(x1.size())
print(x2.size())

# tensors 1 and 2
print(x1)
print(x2)


#size x3
print(x3.size())

#tensors 3
print(x3)

torch.eq(x1,x2)
torch.Size([2, 2]) torch.Size([2, 2]) tensor([[1., 2.], [3., 4.]]) tensor([[2, 2], [2, 5]]) torch.Size([3, 5]) tensor([[-1.3040, -0.4658, -0.5269, 0.7409, 0.9135], [ 1.0780, 2.0584, -0.9629, -1.1412, -0.3105], [ 0.3613, -1.4196, 2.1145, 0.3649, 0.2037]])
Out[103]:
tensor([[False,  True],
        [False, False]])

x1 and x3 have the same size, but x3 is [3,5], has bigger size.

comparing x1 and x2 is Ok.

In [97]:
# Example 2 - working (with broadcasting)
x4 = torch.tensor([[1, 2], [3, 4]])
print(x4.size())
x5 = torch.tensor([2, 5])
print(x5.size())
torch.eq(x4, x5)
torch.Size([2, 2]) torch.Size([2])
Out[97]:
tensor([[False, False],
        [False, False]])

we can also compare, different sizes only if the seconf value, in this case x5, that has size of [2] is broadcastable with the frist one thst is [2,2]

In [98]:
# Example 3 - breaking (to illustrate when it breaks)
x6 = torch.tensor([[0, 2, 4], [3, 4, 5]])
print(x6.size())
x7 = torch.tensor([[2, 3], [2, 4]])
print(x7.size())

torch.eq(x6, x3)
torch.Size([2, 3]) torch.Size([2, 2])
--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-98-ac2ad1ecd5b0> in <module>() 5 print(x7.size()) 6 ----> 7 torch.eq(x6, x3) RuntimeError: The size of tensor a (3) must match the size of tensor b (5) at non-singleton dimension 1

finally, we cant compare different sizes if the second arguments shape is not broadcastable with the first argument.

Closing comments about when to use this function

In [ ]:
jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')

Conclusion

we review 5 basic topics covering more than 10 PyTorch functions.

in the next post ill talk about Linear Regresion.

Reference Links

Provide links to your references and other interesting articles about tensors

In [110]:
jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')
[jovian] Detected Colab notebook... [jovian] Uploading colab notebook to Jovian... [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/cortezb-claro/first-5-things-to-start-learning-pytorch-tensors-e9ba3
In [ ]: