Learn data science and machine learning by building real-world projects on Jovian

## PyTorch Introductory - Assignment 1

#### A brief walkthrough with Shripal ... ... to understand 5 of the functions that can be applied to tensors - torch.Tensor objects.

• function 1 - new_empty()
• function 3 - sqrt()
• function 4 - bernoulli()
• function 5 - sin()
In :
``````# Import torch and other required modules
import torch``````

### Function 1 - torch.Tensor.new_empty()

• returns a new empty Tensor, with specified size (from arguments) with uninitialized elements and attributes (torch.dtype and torch.device) same as that of the mentioned tensor.
In :
``````# Example 1 - working

t1 = torch.zeros(())
print("t1 =", t1, "\nt1.dtype = ", t1.dtype, ";   t1.device = ", t1.device)
print()

t2 = t1.new_empty((2,4))
print("t1 =", t1)
print("t2 =", t2, "\nt2.dtype = ", t2.dtype, ";   t2.device = ", t2.device)

``````
```t1 = tensor(0.) t1.dtype = torch.float32 ; t1.device = cpu t1 = tensor(0.) t2 = tensor([[7.5510e-34, 4.5623e-41, 7.5510e-34, 4.5623e-41], [4.4842e-44, 0.0000e+00, 8.9683e-44, 0.0000e+00]]) t2.dtype = torch.float32 ; t2.device = cpu ```

By default, the dtype of `t1` is `torch.float32` so, dtype of `t2` is also `torch.float32` since it is not explicitly mentioned to something else.

Same is for `device = cpu`

In :
``````# Example 2 - working

t3 = t1.new_empty((2,3), dtype=torch.int8)
print("t3 =", t3, "\nt3.dtype = ", t3.dtype, ";   t3.device = ", t3.device)
print("t1 =", t1, "\nt1.dtype = ", t1.dtype, ";   t1.device = ", t1.device)``````
```t3 = tensor([[ 112, 118, -110], [ 70, 101, 85]], dtype=torch.int8) t3.dtype = torch.int8 ; t3.device = cpu t1 = tensor(0.) t1.dtype = torch.float32 ; t1.device = cpu ```

When explicitly mentioned, the `dtype` of the returned `t3` is set to `torch.int8`
While, that of `t1` is still `torch.float32`

In :
``````# Example 3 - breaking - not working when notebook is run on CPU only mode, without any accelerator/GPU.

t4 = t1.new_empty((2,5), dtype=torch.int8, device=torch.device("cuda:0"))
print("t3 =", t3, "\nt3.dtype = ", t3.dtype, ";   t3.device = ", t3.device)
print("t1 =", t1, "\nt1.dtype = ", t1.dtype, ";   t1.device = ", t1.device)``````
```--------------------------------------------------------------------------- AssertionError Traceback (most recent call last) <ipython-input-4-1274903458ea> in <module> 1 # Example 3 - breaking - not working when notebook is run on CPU only mode, without any accelerator/GPU. 2 ----> 3 t4 = t1.new_empty((2,5), dtype=torch.int8, device=torch.device("cuda:0")) 4 print("t3 =", t3, "\nt3.dtype = ", t3.dtype, "; t3.device = ", t3.device) 5 print("t1 =", t1, "\nt1.dtype = ", t1.dtype, "; t1.device = ", t1.device) /srv/conda/envs/notebook/lib/python3.7/site-packages/torch/cuda/__init__.py in _lazy_init() 147 raise RuntimeError( 148 "Cannot re-initialize CUDA in forked subprocess. " + msg) --> 149 _check_driver() 150 if _cudart is None: 151 raise AssertionError( /srv/conda/envs/notebook/lib/python3.7/site-packages/torch/cuda/__init__.py in _check_driver() 52 Found no NVIDIA driver on your system. Please check that you 53 have an NVIDIA GPU and installed a driver from ---> 54 http://www.nvidia.com/Download/index.aspx""") 55 else: 56 # TODO: directly link to the alternative bin that needs install AssertionError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx```

Providing the keyword argument `device` = `torch.device("cuda:0")` throws a RunTimeError since there is hardware incompatibility.

• it adds the tensor with another tensor or a scalar object
• it is required that the provided (adder) tensor be of the same size as that of the (addend) tensor,
or, that it can be broadcasted to be the same as the size of the (addend) tensor.
In :
``````# Example 1 - working

t5 = torch.ones((2,3))
print(t5)
print()

print(t6)
``````
```tensor([[1., 1., 1.], [1., 1., 1.]]) tensor([[6., 6., 6.], [6., 6., 6.]]) ```
• Above example demonstrates how to add a Python `scalar` to a `torch.Tensor`
In :
``````# Example 2 - working

t7 = torch.ones((2,3), dtype=torch.int)
print(t7)
print()

t8 = torch.tensor([[8.99, 8.99, 8.99]], dtype=torch.float64)
print("t7.shape =", t7.shape)
print("t8.shape =", t8.shape)
print()

print(t9)
``````
```tensor([[1, 1, 1], [1, 1, 1]], dtype=torch.int32) t7.shape = torch.Size([2, 3]) t8.shape = torch.Size([1, 3]) tensor([[9.9900, 9.9900, 9.9900], [9.9900, 9.9900, 9.9900]], dtype=torch.float64) ```

Above example shows that

• the `torch.Size` of `t7` and `t8` tensors are different, but it was automatically broadcasted to match each other and then added to form `t9` tensor.
• also, the `dtype` of `t7` is `torch.int` but it was automatically typecasted to `torch.float64` since `t8` has the higher dtype; and `t9` gets the `dtype` = `torch.float64`
In :
``````# Example 3 - breaking - when shape doesn't match & can't be matched even using broadcasting

t10 = t7.add(torch.tensor([1, 2, 3, 4]))``````
```--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-7-482ebf4c5394> in <module> 1 # Example 3 - breaking - when shape doesn't match & can't be matched even using broadcasting 2 ----> 3 t10 = t7.add(torch.tensor([1, 2, 3, 4])) RuntimeError: The size of tensor a (3) must match the size of tensor b (4) at non-singleton dimension 1```
• The size of the tensors don't match! Moreover, they are not even broadcast-able to match.
Thus, it failed.

### Function 3 - torch.sqrt()

• returns a tensor with the same size as the original.
• but, all the elements are the square roots of those in the original tensor.
In :
``````# Example 1 - working

t11 = torch.tensor([[100., 64],
[   9, 49]])
t12 = t11.sqrt()
t12
``````
Out:
``````tensor([[10.,  8.],
[ 3.,  7.]])``````
• the function returns a tensor with the square roots of all the elements in the `t11` tensor
In :
``````# Example 2 - working

t13 = torch.randn(2,3)
print(t13, "\n")
t14 = torch.sqrt(t13)
print(t14)
``````
```tensor([[-0.3703, -0.5108, -1.0514], [-0.4055, -0.5981, 1.7822]]) tensor([[ nan, nan, nan], [ nan, nan, 1.3350]]) ```
• the square root of a negative number is a complex mathematical entity.
Hence, it successfully returns a tensor with `nan`'s wherever a real square root is not possible.
In :
``````# Example 3 - breaking - sqrt() doesn't work for tensors with dtype=torch.int

t15 = torch.ones((2,2), dtype=torch.int)
t16 = torch.sqrt(t15)
``````
```--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-10-b80610a36a8a> in <module> 2 3 t15 = torch.ones((2,2), dtype=torch.int) ----> 4 t16 = torch.sqrt(t15) RuntimeError: sqrt_vml_cpu not implemented for 'Int'```
• `torch.sqrt()` doesn't work for Short, Int, Long `dtype` tensors

### Function 4 - torch.bernoulli()

• Draws binary random numbers (0 or 1) from a Bernoulli distribution.
• The i-th element of the output tensor will draw a value 1 according to the i-th probability value given in input.
##### output_i ∼ Bernoulli( p = input_i )
In :
``````# Example 1 - working

t17 = torch.empty(2, 3).uniform_(0, 1)  # generates a uniform random matrix with range [0, 1]
print(t17)

t18 = torch.bernoulli(t17)
print(t18)
``````
```tensor([[0.1917, 0.8109, 0.3409], [0.8578, 0.1723, 0.5195]]) tensor([[0., 1., 0.], [1., 0., 1.]]) ```
• the input tensor `t17` provides input values for the Bernoulli() function.
• then generates the output tensor `t18` with output values (1 or 0) based on the corresponding input probability of generating a 1 (otherwise 0)
In :
``````# Example 2 - working

t19 = torch.tensor([[0.1, 0.8, 0.9], [0.7, 0.2, 0.5]])

t20 = torch.bernoulli(input=t19, out=torch.rand(2,2))
print(t20)

``````
```tensor([[1., 1., 1.], [0., 0., 0.]]) ```

As per the pytorch documentation, `torch.bernoulli()` can also be run using the above method.
` ` is same as before, but ` ` takes another tensor and is used to compute the bernoulli function on the input.

In :
``````# Example 3 - breaking - the input tensor should always contain elements ranging from 0 through 1.

t21 = torch.randn(2,3)
print(t21)

t22 = torch.bernoulli(t21)
``````
```tensor([[-0.5437, -1.5096, 0.5433], [ 0.2914, -1.5694, -0.2815]]) ```
```--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-13-f6fbe500ac77> in <module> 4 print(t21) 5 ----> 6 t22 = torch.bernoulli(t21) RuntimeError: Expected p_in >= 0 && p_in <= 1 to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)```
• `torch.bernoulli()` function takes input tensor as a probability of getting a 1 (otherwise 0) and hence, input tensor elements should always be a number ranging from 0 through 1.

I checked the documentation as well as tried to go through a few other references to find a better understanding of the poorly documented arguments - `*` and `generator` in the function - `torch.bernoulli()`

It'd be great if you can help me understand that better!

Here is the official github issues page related to this: https://github.com/pytorch/pytorch/issues/21567

### Function 5 - torch.sin()

• returns a tensor that contains elementwise (trigonometric) sine values of the input tensor.
In :
``!pip install matplotlib``
```Requirement already satisfied: matplotlib in /srv/conda/envs/notebook/lib/python3.7/site-packages (3.2.1) Requirement already satisfied: cycler>=0.10 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib) (0.10.0) Requirement already satisfied: python-dateutil>=2.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib) (2.8.1) Requirement already satisfied: kiwisolver>=1.0.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib) (1.2.0) Requirement already satisfied: numpy>=1.11 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib) (1.18.1) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib) (2.4.7) Requirement already satisfied: six in /srv/conda/envs/notebook/lib/python3.7/site-packages (from cycler>=0.10->matplotlib) (1.14.0) ```
In :
``````import numpy as np
pi = np.pi

import matplotlib.pyplot as plt
%matplotlib inline
``````
In :
``````# Example 1 - working

t23 = torch.tensor([[0   , pi    , 2*pi],
[pi/2, 3*pi/2, pi/6]])
print(t23)
print()

t24 = t23.sin()
print(t24)
print()

t25 = t24.round()
print("Rounding off t24 for simplicity:\n", t25)``````
```tensor([[0.0000, 3.1416, 6.2832], [1.5708, 4.7124, 0.5236]]) tensor([[ 0.0000e+00, -8.7423e-08, 1.7485e-07], [ 1.0000e+00, -1.0000e+00, 5.0000e-01]]) Rounding off t24 for simplicity: tensor([[ 0., -0., 0.], [ 1., -1., 0.]]) ```

In :
``````# Example 2 - working

np26 = np.arange(-pi, pi+pi/6, pi/6)
t26 = torch.tensor(np26)
print(t26)
print()

t27 = t26.sin()
print(t27)

plt.bar(t26, t27, color='purple')``````
```tensor([-3.1416e+00, -2.6180e+00, -2.0944e+00, -1.5708e+00, -1.0472e+00, -5.2360e-01, -8.8818e-16, 5.2360e-01, 1.0472e+00, 1.5708e+00, 2.0944e+00, 2.6180e+00, 3.1416e+00], dtype=torch.float64) tensor([-1.2246e-16, -5.0000e-01, -8.6603e-01, -1.0000e+00, -8.6603e-01, -5.0000e-01, -8.8818e-16, 5.0000e-01, 8.6603e-01, 1.0000e+00, 8.6603e-01, 5.0000e-01, 1.8988e-15], dtype=torch.float64) ```
Out:
``<BarContainer object of 13 artists>`` • in the above demonstration, first we got a range of equidistant elements from to π with a step-size = π/6
• then we created a tensor from that `numpy.ndarray`.
• then, applied function `torch.sin()` on that tensor.
• then we passed both the tensors into a `matplotlib.pyplot` function as arguments to plot a graph of sine curve.
In :
``````# Example 3 - breaking - torch.sin() function doesn't apply on tensors with dtypes other than torch.float32 or torch.float64

t29 = torch.zeros((2,3), dtype=torch.int)
t30 = torch.sin(t29)
``````
```--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-18-7f866c15b079> in <module> 2 3 t29 = torch.zeros((2,3), dtype=torch.int) ----> 4 t30 = torch.sin(t29) RuntimeError: sin_vml_cpu not implemented for 'Int'```
• `torch.sin()` function doesn't apply on tensors with dtypes other than `torch.float32` or `torch.float64`

### Conclusion

To summarize, in this notebook, we covered a brief description and a few examples of executing 5 selected functions on torch.Tensor object.

the functions selected and discussed in this notebook are:

• new_empty()
• torch.sqrt()
• torch.bernoulli()
• torch.sin()

``!pip install jovian --upgrade --quiet``
``import jovian``
``jovian.commit()``
```[jovian] Attempting to save notebook.. ```
`` ``