Learn data science and machine learning by building real-world projects on Jovian

Sign up to execute **first-5-things-to-start-learning-pytorch-tensors-ead72** and 150,000+ data science projects. Build your own projects and share them online!

Created 6 months ago

lets begin our TorchAdventure! with this 11 basic functions distributed in the following sections:

- Creating tensors: Empty() and Zeros()
- measuring tensors: Size() and Shape(), Sum() and Dimensions
- changing and copying tensors: Reshape(), view() and randm()
- modifying tensors: Unsqueeze()
- comparing tensors: Element-Wise Equality: Eq

Before we begin, let's install and import PyTorch

In [ ]:

```
# Uncomment and run the appropriate command for your operating system, if required
# Linux / Binder
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
# Windows
# !pip install numpy torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
# MacOS
# !pip install numpy torch torchvision torchaudio
```

In [2]:

```
# Import torch and other required modules
import torch
```

We need to start workng with the basics pytorch functions, and the first thing is create our matrix

In [4]:

```
# Creates a 3 x 2 matrix which is empty
a = torch.empty(3, 2)
print(a)
```

```
tensor([[1.5842e-35, 0.0000e+00],
[4.4842e-44, 0.0000e+00],
[ nan, 0.0000e+00]])
```

Here is how PyTorch is allocating memory for this tensor. Whatever, it will not erase anything previous content in the memory.

by default, when you initializes a tensor is used the float32 dtype. you can review it here: https://pytorch.org/docs/stable/generated/torch.set_default_tensor_type.html#torch.set_default_tensor_type

But you can also start working woth torch.zeros

In [6]:

```
# Applying the zeros function and
# storing the resulting tensor
a = torch.zeros([3, 5])
print("a = ", a)
b = torch.zeros([2, 4])
print("b = ", b)
c = torch.zeros([4, 1])
print("c = ", c)
d = torch.zeros([4, 4, 2])
print("d = ", d)
```

```
a = tensor([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]])
b = tensor([[0., 0., 0., 0.],
[0., 0., 0., 0.]])
c = tensor([[0.],
[0.],
[0.],
[0.]])
d = tensor([[[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.]],
[[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.]],
[[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.]],
[[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.]]])
```

this tensor is filled with zeros, so PyTorch allocates memory and zero-initializes the tensor elements inside

You cannot change the way a tensor is created, if you create a zeros tensor, make sure is not referenced to any other.

In [34]:

```
# correctly initialized
a = torch.empty(3,3)
print(a)
#also correct
b = torch.empty(1,2,3)
print(b)
```

```
tensor([[2.4258e-35, 0.0000e+00, 1.5975e-43],
[1.3873e-43, 1.4574e-43, 6.4460e-44],
[1.4153e-43, 1.5274e-43, 1.5695e-43]])
tensor([[[2.3564e-35, 0.0000e+00, 1.4013e-45],
[1.4574e-43, 6.4460e-44, 1.4153e-43]]])
```

In [35]:

```
# incorrect reference, you must create a new one
c = torch.zeros(b,1)
print("c = ", c)
```

```
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-35-c044e2995879> in <module>()
1 # incorrect reference, you must create a new one
----> 2 c = torch.zeros(b,1)
3 print("c = ", c)
TypeError: zeros() received an invalid combination of arguments - got (Tensor, int), but expected one of:
* (tuple of ints size, *, tuple of names names, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
* (tuple of ints size, *, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
```

lets review the zero method closely:

Our documentation says:

```
Syntax: torch.zeros(size, out=None)
Parameters:
size: a sequence of integers defining the shape of the output tensor
out (Tensor, optional): the output tensor
Return type: A tensor filled with scalar value 0, of same shape as size.
```

torch.zeros and torch.empty are the first functions to start working with pytorch tensors and learning a little bit of matrix and vectors

Let's save our work using Jovian before continuing.

In [36]:

`!pip install jovian --upgrade --quiet`

```
|████████████████████████████████| 71kB 3.1MB/s eta 0:00:011
Building wheel for uuid (setup.py) ... done
```

In [37]:

`import jovian`

In [104]:

`jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')`

```
[jovian] Detected Colab notebook...
[jovian] Uploading colab notebook to Jovian...
[jovian] Capturing environment..
[jovian] Committed successfully! https://jovian.ai/cortezb-claro/first-5-things-to-start-learning-pytorch-tensors
```

Lets understand dimensions in Pytorch.

Now lets create some tensors and determine the size of every one

In [45]:

```
import torch
# Create a tensor from data
c = torch.tensor([[3.2 , 1.6, 2], [1.3, 2.5 , 6.9]])
print(c)
print(c.size())
```

```
tensor([[3.2000, 1.6000, 2.0000],
[1.3000, 2.5000, 6.9000]])
torch.Size([2, 3])
```

Lets see torch.shape and take a closer look at how size is given here

Now in the next example lets use shape functions to get the size of a tensor

In [47]:

```
x = torch.tensor([
[1, 2, 3],
[4, 5, 6]
])
x.shape
torch.Size([2, 3])
```

Out[47]:

`torch.Size([2, 3])`

Shape and Size give us the same correct dimensions of the tensor.

in this case we have a 3D-tensor (with 3 dimensions)

Dimension 0 Dimension 1 and Dimension 2

lets create a new tensor:

In [49]:

```
y = torch.tensor([
[
[1, 2, 3],
[4, 5, 6]
],
[
[1, 2, 3],
[4, 5, 6]
],
[
[1, 2, 3],
[4, 5, 6]
]
])
y.shape
```

Out[49]:

`torch.Size([3, 2, 3])`

Lets se how we can make operations using a 3d tensor now for every dimension layer and see how it behaves

In [57]:

```
sum1 = torch.sum(y, dim=0)
print(sum1)
sum2 = torch.sum(y, dim=1)
print(sum2)
sum3 = torch.sum(y, dim=2)
print(sum3)
```

```
tensor([[ 3, 6, 9],
[12, 15, 18]])
tensor([[5, 7, 9],
[5, 7, 9],
[5, 7, 9]])
tensor([[ 6, 15],
[ 6, 15],
[ 6, 15]])
```

we can see now a 3d tensor is more complicated as we advance, and we can perform custom operations within every dimension

its limited right now to 3 dims so we cannot perform this:

In [56]:

```
sum2 = torch.sum(y, dim=3)
print(sum1)
```

```
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-56-7caf723c6102> in <module>()
----> 1 sum2 = torch.sum(y, dim=3)
2 print(sum1)
IndexError: Dimension out of range (expected to be in range of [-3, 2], but got 3)
```

to accomplish adding more layers of dimensions we can review the unsqueeze functions --> .unsqueeze() but for now leta go to the next basic function

In [58]:

`jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')`

```
[jovian] Detected Colab notebook...
[jovian] Uploading colab notebook to Jovian...
[jovian] Capturing environment..
[jovian] Committed successfully! https://jovian.ai/cortezb-claro/5-basic-functions-to-learn-pytorch-tensors-d6973
```

there is a function in NunPy called ndarray.reshape() for reshaping an array.

Now in pytorch, there is torch.view(tensor) for the same purpose, but at the same time, there is also a torch.reshape(tensor).

lets figure out the differences between them and when you should use either of them.

First of all we are going to use a new functions for randomize our tensor.

In [69]:

```
import torch
x = torch.randn(5, 3)
print(x)
```

```
tensor([[-0.5793, 0.6999, 1.7417],
[-0.9810, 0.0626, 0.4100],
[-0.6519, -0.0595, -1.2156],
[-0.3973, -0.3103, 1.6253],
[ 0.2775, -0.0045, -0.2985]])
```

this is basic usage of torch.randm, so now lets use View from another variable "y" and describe all elements

In [74]:

```
# Return a view of the x, but only having
# one dimension and max number of elements
y = x.view(5 * 3)
#lets see the size of every tensor
print("lets see the size of every tensor")
print('Size of x:', x.size())
print('Size of y:', y.size())
#and the elements of very tensor to compare
print("and the elements of very tensor to compare")
print("X:", x)
print("Y:", y)
```

```
lets see the size of every tensor
Size of x: torch.Size([5, 3])
Size of y: torch.Size([15])
and the elements of very tensor to compare
X: tensor([[-0.5793, 0.6999, 1.7417],
[-0.9810, 0.0626, 0.4100],
[-0.6519, -0.0595, -1.2156],
[-0.3973, -0.3103, 1.6253],
[ 0.2775, -0.0045, -0.2985]])
Y: tensor([-0.5793, 0.6999, 1.7417, -0.9810, 0.0626, 0.4100, -0.6519, -0.0595,
-1.2156, -0.3973, -0.3103, 1.6253, 0.2775, -0.0045, -0.2985])
```

take a look at Y tensor, it only has 1 dimension. so Viewing another tensor may be difficult for some operations

Now lets use Reshape to replicate the exact dimensions of X

In [61]:

```
# Get back the original tensor with reshape()
z = y.reshape(5, 3)
print(z)
```

```
tensor([[-0.2927, 0.0329, 0.8485],
[ 1.9581, 0.8313, -0.1529],
[-0.2330, -0.1887, 1.8206],
[ 1.5252, 1.0909, 0.0547],
[-0.1231, -0.4238, -0.6724]])
```

we cannot only reshape the original, we can also change the dimensions with some limited actions related to the maximum elements:

first lets reshape to 1 more dimension

In [77]:

```
# Get back the original tensor with reshape()
z = y.reshape(15)
print(z)
#reshaping 15 elements, 1 dim
z = y.reshape(3*5)
print(z)
#reshaping in different order, 3 dimensions
z = y.reshape(3,5)
print(z)
#Reshaping with more dimensions but its 15 elements always
z = y.reshape(3,5,1)
print(z)
```

```
tensor([-0.5793, 0.6999, 1.7417, -0.9810, 0.0626, 0.4100, -0.6519, -0.0595,
-1.2156, -0.3973, -0.3103, 1.6253, 0.2775, -0.0045, -0.2985])
tensor([-0.5793, 0.6999, 1.7417, -0.9810, 0.0626, 0.4100, -0.6519, -0.0595,
-1.2156, -0.3973, -0.3103, 1.6253, 0.2775, -0.0045, -0.2985])
tensor([[-0.5793, 0.6999, 1.7417, -0.9810, 0.0626],
[ 0.4100, -0.6519, -0.0595, -1.2156, -0.3973],
[-0.3103, 1.6253, 0.2775, -0.0045, -0.2985]])
tensor([[[-0.5793],
[ 0.6999],
[ 1.7417],
[-0.9810],
[ 0.0626]],
[[ 0.4100],
[-0.6519],
[-0.0595],
[-1.2156],
[-0.3973]],
[[-0.3103],
[ 1.6253],
[ 0.2775],
[-0.0045],
[-0.2985]]])
```

Now lets reshape exceeding the number of elements in the tensor:

In [78]:

```
z = y.reshape(16)
print(z)
```

```
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-78-c7ae174fce73> in <module>()
----> 1 z = y.reshape(16)
2 print(z)
RuntimeError: shape '[16]' is invalid for input of size 15
```

it will fail also for z = y.reshape(3*6) or putting more elements that does not exist in the tensor.

Now lets keep going to the next section.

In [109]:

`jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')`

```
[jovian] Detected Colab notebook...
[jovian] Uploading colab notebook to Jovian...
[jovian] Capturing environment..
[jovian] Committed successfully! https://jovian.ai/cortezb-claro/first-5-things-to-start-learning-pytorch-tensors-2047c
```

Mainly, it allows us to add more dimensions at specific index you define.

lets take a look:

In [84]:

```
import torch
#dim=1, that is (3)
x = torch.tensor([1, 2, 3])
print('x: ', x)
print('x.size: ', x.size())
#x1 becomes a matrix of (3,1)
x1 = torch.unsqueeze(x, 1)
print('x1: ', x1)
print('x1.size: ', x1.size())
```

```
x: tensor([1, 2, 3])
x.size: torch.Size([3])
x1: tensor([[1],
[2],
[3]])
x1.size: torch.Size([3, 1])
x2: tensor([[1, 2, 3]])
x2.size: torch.Size([1, 3])
```

Our initial tensor is Tensor([1,2,3]), and the output size is [3].

And then we proceed with adding 1 dimenson with unsqueeze operation, namely torch.unsqueeze(x, 1), the size of x1 is [3,1].

In [85]:

```
#x2 becomes a matrix of (1,3)
x2 = torch.unsqueeze(x, 0)
print('x2: ', x2)
print('x2.size: ', x2.size())
```

```
x2: tensor([[1, 2, 3]])
x2.size: torch.Size([1, 3])
```

When we perform torch.unsqueeze(x, 0), the size of x2 is [1,3].

In [89]:

```
# Example 3 - breaking (to illustrate when it breaks)
print(x.unsqueeze())
```

```
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-89-5a320a828907> in <module>()
2
3
----> 4 print(x.unsqueeze())
TypeError: unsqueeze() missing 1 required positional arguments: "dim"
```

We must specified the dimension correctly, although we are just adding 1 dim, it is necessary to put like this: x.unqueeze(0)

In [ ]:

`jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')`

This function is under comparison category and it computes equality in element-wise and returns a boolean tensor. True if equal, False otherwise.

Lets review how we can operate with different sizes of tensors:

In [103]:

```
# Example 1 - working
x1 = torch.tensor([[1, 2], [3, 4.]])
x2 = torch.tensor([[2, 2], [2, 5]])
x3 = torch.randn(3,5)
#size x1 and z2
print(x1.size())
print(x2.size())
# tensors 1 and 2
print(x1)
print(x2)
#size x3
print(x3.size())
#tensors 3
print(x3)
torch.eq(x1,x2)
```

```
torch.Size([2, 2])
torch.Size([2, 2])
tensor([[1., 2.],
[3., 4.]])
tensor([[2, 2],
[2, 5]])
torch.Size([3, 5])
tensor([[-1.3040, -0.4658, -0.5269, 0.7409, 0.9135],
[ 1.0780, 2.0584, -0.9629, -1.1412, -0.3105],
[ 0.3613, -1.4196, 2.1145, 0.3649, 0.2037]])
```

Out[103]:

```
tensor([[False, True],
[False, False]])
```

x1 and x3 have the same size, but x3 is [3,5], has bigger size.

comparing x1 and x2 is Ok.

In [97]:

```
# Example 2 - working (with broadcasting)
x4 = torch.tensor([[1, 2], [3, 4]])
print(x4.size())
x5 = torch.tensor([2, 5])
print(x5.size())
torch.eq(x4, x5)
```

```
torch.Size([2, 2])
torch.Size([2])
```

Out[97]:

```
tensor([[False, False],
[False, False]])
```

we can also compare, different sizes only if the seconf value, in this case x5, that has size of [2] is broadcastable with the frist one thst is [2,2]

In [98]:

```
# Example 3 - breaking (to illustrate when it breaks)
x6 = torch.tensor([[0, 2, 4], [3, 4, 5]])
print(x6.size())
x7 = torch.tensor([[2, 3], [2, 4]])
print(x7.size())
torch.eq(x6, x3)
```

```
torch.Size([2, 3])
torch.Size([2, 2])
```

```
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-98-ac2ad1ecd5b0> in <module>()
5 print(x7.size())
6
----> 7 torch.eq(x6, x3)
RuntimeError: The size of tensor a (3) must match the size of tensor b (5) at non-singleton dimension 1
```

finally, we cant compare different sizes if the second arguments shape is not broadcastable with the first argument.

Closing comments about when to use this function

In [ ]:

`jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')`

we review 5 basic topics covering more than 10 PyTorch functions.

in the next post ill talk about Linear Regresion.

Provide links to your references and other interesting articles about tensors

- Official documentation for tensor operations: https://pytorch.org/docs/stable/torch.html
- Squeeze and unsqueeze functions here: https://www.programmersought.com/article/65705814717/
- Understanding dimensions in Pytorch: https://towardsdatascience.com/understanding-dimensions-in-pytorch-6edf9972d3be

In [110]:

`jovian.commit(project='First-5-things-to-start-learning-PyTorch-Tensors.ipynb')`

```
[jovian] Detected Colab notebook...
[jovian] Uploading colab notebook to Jovian...
[jovian] Capturing environment..
[jovian] Committed successfully! https://jovian.ai/cortezb-claro/first-5-things-to-start-learning-pytorch-tensors-e9ba3
```

In [ ]:

` `