Jovian
⭐️
Sign In

Assignment 2 - Numpy Array Operations

Linear Algebra routines in Numpy

Numpy is the fundamental package for scientific and numerical computing in Python. Its knowledge is useful for understanding other libraries in Python, for example, Pandas and matplotlib, which bulid upon Numpy. The five numpy functions which I will explain in this notebook are:

  • inner(a,b)
  • kron(a,b)
  • linalg.cholesky(a)
  • linalg.det(a)
  • linalg.solve(a,b)
In [1]:
!pip install jovian --upgrade -q
In [2]:
import jovian
In [3]:
jovian.commit(project='numpy-array-operations')
[jovian] Attempting to save notebook.. [jovian] Updating notebook "adithyan-ramesh/numpy-array-operations" on https://jovian.ai/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/adithyan-ramesh/numpy-array-operations

Let's begin by importing Numpy and listing out the functions covered in this notebook.

In [4]:
import numpy as np
In [5]:
a=np.array([[1, 2, 3],[2, 2 , 4]])
In [6]:
b=np.array([[2,2,2.4],[3,7,4.5]])
In [7]:
# List of functions explained 
#function1 = np.inner(a,b)
#function2 = np.kron(a,b)
#function3 = np.linalg.cholesky(a)
#function4 = linalg.det(a)
#function5 = linalg.solve(a,b)

Function 1 - np.inner(a,b)

Quite intuitive, it merely calculates the inner product of two matrices. The inner product is a generalization of the dot product, and dot product of two arrays could be done using the np.dot() function in numpy.

In [8]:
# Example 1- working
arr1 = np.array([[1, 2], 
        [3, 4.]])

arr2 = np.array([[5, 6, 7], 
        [8, 9, 10]])

np.inner(arr1, arr2.reshape(3,2))
Out[8]:
array([[17., 23., 29.],
       [39., 53., 67.]])

In the above example, we have two arrays arr1 and arr2, where the shape of the two are (2,2) and (2,3) respectively. Now, one thing to note is if the arrays being passed to inner() are of different shapes then the last dimensions of both the arrays must be the same. In this case, the last dim of arr1 i.e. 2 is not the same as the last dimension of arr2 i.e. 3. Hence, I reshaped arr2 to (3,2) to run the function properly without errors.

In [9]:
# Example 2 - working
a=np.array([1, 2,1])
b=np.array([1,0,2])
np.inner(a,b)
Out[9]:
3

Here, I demonstrate that if we have two vectors, then the inner product is simply the dot product of the two vectors and its very simple to see from the above example. The answer, we would do by hand, would be : 1x1 + 2x0 + 1x2 = 3

In [10]:
# Example 3 - breaking (to illustrate when it breaks)
arr1 = [[1, 2], 
        [3, 4.]]

arr2 = [[5, 6, 7], 
        [8, 9, 10]]

np.inner(arr1, arr2)
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-10-c1d07ff52293> in <module> 6 [8, 9, 10]] 7 ----> 8 np.inner(arr1, arr2) <__array_function__ internals> in inner(*args, **kwargs) ValueError: shapes (2,2) and (3,2) not aligned: 2 (dim 1) != 3 (dim 0)

As explained before, we must have the last dimensions of multidimensional arrays to be equal while performing the inner product of the two arrays. Here, it wasn't equal, and after running we get the ValueError saying that the shapes of the argument arrays is not aligned.

In [11]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "adithyan-ramesh/numpy-array-operations" on https://jovian.ai/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/adithyan-ramesh/numpy-array-operations

Function 2 - np.kron(a,b)

This function computes and returns the Kronecker product of the two arrays passed to it. Don't confuse this with usual matrix multiplication.

In [12]:
# Example 1 - working
a=np.array([1,2,3])
b=np.array([2,2,2])
np.kron(a,b)
Out[12]:
array([2, 2, 2, 4, 4, 4, 6, 6, 6])

The most important thing to note here is the shape of the output array. The Kronecker product produces the output, say arr1 has shape (2,2) and arr2 has shape (2,3) then output has shape (2x2,2x3)= (4,6) i.e. just multiply the corresponding dimensions of the arrays to figure out the shape of the output array. In the above example, the two arrays are simply vectors and the result is a (9,) array.

In [13]:
# Example 2 - working
a=np.array([[1, 2, 3], [2, 4, 3]])
b=np.array([[[0,1], [2, 3], [ 4 , 6]]])
np.kron(a,b)
Out[13]:
array([[[ 0,  1,  0,  2,  0,  3],
        [ 2,  3,  4,  6,  6,  9],
        [ 4,  6,  8, 12, 12, 18],
        [ 0,  2,  0,  4,  0,  3],
        [ 4,  6,  8, 12,  6,  9],
        [ 8, 12, 16, 24, 12, 18]]])

One thing which is demonstrated above is that, if the number of dimensions of the argument arrays does not match, although the function wants it to be same, but no worries, it will prepend the smallest ones with 1 and execute the product. Here, 'a' is (2,3) and 'b' is (1,3,2) and internally the shape of 'a' becomes (1,2,3) and the shape of output array then becomes (1x1, 3x2, 2x3)=(1,6,6) and it is clear from the output.

In [14]:
# Example 3 - breaking (to illustrate when it breaks)
## This is a product between two matrices so I think it will not break anywhere for any two given matrices

This is a very flexible function, and is useful for mathematical operations and research tasks.

In [15]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "adithyan-ramesh/numpy-array-operations" on https://jovian.ai/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/adithyan-ramesh/numpy-array-operations

Function 3 - np.linalg.cholesky

It returns the Cholesky decomposition of a square matrix

In [18]:
# Example 1 - working
a=np.array([[1,2],[1,2]])
np.linalg.cholesky(a)

Out[18]:
array([[1., 0.],
       [1., 1.]])

The function returns the Cholesky decomposition of the matrix a

In [16]:
# Example 2 - working
a=np.array([[2, 1.j],[-1.j,2]])
np.linalg.cholesky(a)
Out[16]:
array([[ 1.41421356+0.j        ,  0.        +0.j        ],
       [-0.        -0.70710678j,  1.22474487+0.j        ]])

Here, we have a complex matrix, which is posititve definite and the function returns the Cholesky decomposition of such a matrix.

In [17]:
# Example 3 - breaking (to illustrate when it breaks)
a=np.array([[[1,2,3,0], [3,3.4,4.5,9], [5,6,0,8], [1,1,1,2]]])
np.linalg.cholesky(a)
--------------------------------------------------------------------------- LinAlgError Traceback (most recent call last) <ipython-input-17-edb518e67d76> in <module> 1 # Example 3 - breaking (to illustrate when it breaks) 2 a=np.array([[[1,2,3,0], [3,3.4,4.5,9], [5,6,0,8], [1,1,1,2]]]) ----> 3 np.linalg.cholesky(a) <__array_function__ internals> in cholesky(*args, **kwargs) ~\Anaconda3\lib\site-packages\numpy\linalg\linalg.py in cholesky(a) 753 t, result_t = _commonType(a) 754 signature = 'D->D' if isComplexType(t) else 'd->d' --> 755 r = gufunc(a, signature=signature, extobj=extobj) 756 return wrap(r.astype(result_t, copy=False)) 757 ~\Anaconda3\lib\site-packages\numpy\linalg\linalg.py in _raise_linalgerror_nonposdef(err, flag) 98 99 def _raise_linalgerror_nonposdef(err, flag): --> 100 raise LinAlgError("Matrix is not positive definite") 101 102 def _raise_linalgerror_eigenvalues_nonconvergence(err, flag): LinAlgError: Matrix is not positive definite

Here, I entered a random array into the function, and it returned a LinAlgError which says that 'Matrix is not positive definite'. Now, the cholesky function requires that the matrix which we pass in as an argument must have to be positive definite and Hermitian. It does not make efforts to check for hermiticity but it checks for positive-definiteness of the matrix, hence this error. So make sure to enter a positive definite matrix only to avoid errors. Ofcourse, python provides us with the try and except block so if you are unsure whether the matrix is positive definite or not, go forward and use the try and except block.

This function is generally used for solving system of linear equations very efficiently. We encounter system of linear equations very often in machine learning.

In [18]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "adithyan-ramesh/numpy-array-operations" on https://jovian.ai/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/adithyan-ramesh/numpy-array-operations

Function 4 - np.linalg.det(a)

It computes the determinant of the array being passed to it.

In [19]:
# Example 1 - working
a=np.array([[1,2],[3,4]])
np.linalg.det(a)
Out[19]:
-2.0000000000000004

The array is a 2x2 matrix so determinant is (ad-bc), i.e. ((a[0,0]x a[1,1]) - (a[0,1]x a[1,0]))

In [20]:
# Example 2 - working
a=np.array([[1,1],[1,1]])
np.linalg.det(a)
Out[20]:
0.0

An example of a simple matrix where the determinant is zero. Special importance is that for a matrix whose determinant is zero is called a Singular matrix and for such a matrix we cannot evaluate its inverse.

In [21]:
# Example 3 - breaking (to illustrate when it breaks)
a=np.array([[[1,2,1],[2,2,0], [3,3.3,1],[1,1,1]]])
np.linalg.det(a)
--------------------------------------------------------------------------- LinAlgError Traceback (most recent call last) <ipython-input-21-7b27efac09f2> in <module> 1 # Example 3 - breaking (to illustrate when it breaks) 2 a=np.array([[[1,2,1],[2,2,0], [3,3.3,1],[1,1,1]]]) ----> 3 np.linalg.det(a) <__array_function__ internals> in det(*args, **kwargs) ~\Anaconda3\lib\site-packages\numpy\linalg\linalg.py in det(a) 2111 a = asarray(a) 2112 _assert_stacked_2d(a) -> 2113 _assert_stacked_square(a) 2114 t, result_t = _commonType(a) 2115 signature = 'D->D' if isComplexType(t) else 'd->d' ~\Anaconda3\lib\site-packages\numpy\linalg\linalg.py in _assert_stacked_square(*arrays) 211 m, n = a.shape[-2:] 212 if m != n: --> 213 raise LinAlgError('Last 2 dimensions of the array must be square') 214 215 def _assert_finite(*arrays): LinAlgError: Last 2 dimensions of the array must be square

Here a (1,4,3) shaped matrix is chosen. Remember, for multidimensional arrays, the last two dimensions must be equal. Hence , we have a LinAlgError here so we cannot evaluate the determinant here.

This determinant function is central to linear algebra and is quite important in many calculations. So having a knowledge of this will certainly help.

In [22]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "adithyan-ramesh/numpy-array-operations" on https://jovian.ai/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/adithyan-ramesh/numpy-array-operations

Function 5 - np.linalg.solve(a,b)

Solves a system of linear equations or a linear matrix equation.

In [23]:
# Example 1 - working
a=np.array([[1,2],[2,3]])
b=np.array([1,2])

np.linalg.solve(a,b.reshape(2,1))
Out[23]:
array([[1.],
       [0.]])

Here, the matrix 'a' is called the coefficient matrix. What we get is basically the solution of AX = B type of system of equations. The matrix 'X' contains the variables whose values are to be solved and the np.linalg.solve() gives this matrix as the output. I used the reshape function on matrix 'b' because I had to make sure the shape of that matrix is compatible for multiplication with the inverse of matrix 'a' which is a 2x2 matrix.

In [24]:
# Example 2 - working
a=np.array([[1,1],[2,4]])
b=np.array([1,1])

np.linalg.solve(a, b.reshape(2,1))

Out[24]:
array([[ 1.5],
       [-0.5]])

Another example of solving the system of linear equations.

In [25]:
# Example 3 - breaking (to illustrate when it breaks)
a=np.array([[1,3],[2,6]])
b=np.array([1,2])

np.linalg.solve(a,b.reshape(2,1))
--------------------------------------------------------------------------- LinAlgError Traceback (most recent call last) <ipython-input-25-d06b99e2c6ab> in <module> 3 b=np.array([1,2]) 4 ----> 5 np.linalg.solve(a,b.reshape(2,1)) <__array_function__ internals> in solve(*args, **kwargs) ~\Anaconda3\lib\site-packages\numpy\linalg\linalg.py in solve(a, b) 397 signature = 'DD->D' if isComplexType(t) else 'dd->d' 398 extobj = get_linalg_error_extobj(_raise_linalgerror_singular) --> 399 r = gufunc(a, b, signature=signature, extobj=extobj) 400 401 return wrap(r.astype(result_t, copy=False)) ~\Anaconda3\lib\site-packages\numpy\linalg\linalg.py in _raise_linalgerror_singular(err, flag) 95 96 def _raise_linalgerror_singular(err, flag): ---> 97 raise LinAlgError("Singular matrix") 98 99 def _raise_linalgerror_nonposdef(err, flag): LinAlgError: Singular matrix

This is a very popular reason why we cannot find a solution to the system of linear equations, here, the matrix 'a' is singular , which means its determinant is zero, as I had explained in my above function of np.linalg.det(). Remember to have a non-singular coefficient matrix, otherwise embrace yourself to encounter the "LinAlgError: Singular matrix" error !

This function is widely used to solve the system of linear equations and has wide spectrum of applicability in various fields of science and mathematics.

In [26]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "adithyan-ramesh/numpy-array-operations" on https://jovian.ai/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ai/adithyan-ramesh/numpy-array-operations

Conclusion

In this notebook I covered just a small fraction i.e. only five Linear Algebra functions which Numpy has to offer. There are tons of functions available on the Numpy documentation page. It is really amazing to see so many functions available for us to use and make our life so much easy. Refer to it and make it one of your best friends !

Reference Links

Provide links to your references and other interesting articles about Numpy arrays:

In [ ]:
jovian.commit()
[jovian] Attempting to save notebook..
In [ ]: