Jovian
⭐️
Sign In

Data Analysis with Python: Zero to Pandas - Course Project Guidelines

(remove this cell before submission)

Important links:

This is the starter notebook for the course project for Data Analysis with Python: Zero to Pandas. You will pick a real-world dataset of your choice and apply the concepts learned in this course to perform exploratory data analysis. Use this starter notebook as an outline for your project . Focus on documentation and presentation - this Jupyter notebook will also serve as a project report, so make sure to include detailed explanations wherever possible using Markdown cells.

Evaluation Criteria

Your submission will be evaluated using the following criteria:

  • Dataset must contain at least 3 columns and 150 rows of data
  • You must ask and answer at least 4 questions about the dataset
  • Your submission must include at least 4 visualizations (graphs)
  • Your submission must include explanations using markdown cells, apart from the code.
  • Your work must not be plagiarized i.e. copy-pasted for somewhere else.

Follow this step-by-step guide to work on your project.

Step 1: Select a real-world dataset

Here's some sample code for downloading the US Elections Dataset:

import opendatasets as od
dataset_url = 'https://www.kaggle.com/tunguz/us-elections-dataset'
od.download('https://www.kaggle.com/tunguz/us-elections-dataset')

You can find a list of recommended datasets here: https://jovian.ml/forum/t/recommended-datasets-for-course-project/11711

Step 2: Perform data preparation & cleaning

  • Load the dataset into a data frame using Pandas
  • Explore the number of rows & columns, ranges of values etc.
  • Handle missing, incorrect and invalid data
  • Perform any additional steps (parsing dates, creating additional columns, merging multiple dataset etc.)

Step 3: Perform exploratory analysis & visualization

  • Compute the mean, sum, range and other interesting statistics for numeric columns
  • Explore distributions of numeric columns using histograms etc.
  • Explore relationship between columns using scatter plots, bar charts etc.
  • Make a note of interesting insights from the exploratory analysis

Step 4: Ask & answer questions about the data

  • Ask at least 4 interesting questions about your dataset
  • Answer the questions either by computing the results using Numpy/Pandas or by plotting graphs using Matplotlib/Seaborn
  • Create new columns, merge multiple dataset and perform grouping/aggregation wherever necessary
  • Wherever you're using a library function from Pandas/Numpy/Matplotlib etc. explain briefly what it does

Step 5: Summarize your inferences & write a conclusion

  • Write a summary of what you've learned from the analysis
  • Include interesting insights and graphs from previous sections
  • Share ideas for future work on the same topic using other relevant datasets
  • Share links to resources you found useful during your analysis

Step 6: Make a submission & share your work

(Optional) Step 7: Write a blog post

Example Projects

Refer to these projects for inspiration:

NOTE: Remove this cell containing the instructions before making your submission. You can do using the "Edit > Delete Cells" menu option.

Project Title - IPL Data Analysis(2008-2019)

Indian Premier League (IPL) is a Twenty20 cricket format league in India. It is usually played in April and May every year. The league was founded by Board of Control for Cricket India (BCCI) in 2008. There have been twelve seasons of the IPL tournament. The current IPL title holders are the Mumbai Indians, who won the 2019 season.

The dataset has been downloaded from Kaggle Dataset. The dataset contains two csv files but only one csv file i.e matches.csv has been used in this project for the analysis. The libraries for data analysis and visualization used in this project are Numpy, Pandas, Matplotlib and Seaborn. These libraries are covered in the course by Jovian.ml named as Data Analysis with Python: Zero to Pandas.

As a first step, let's upload our Jupyter notebook to Jovian.ml.

How to run the code

This is an executable Jupyter notebook hosted on Jovian.ml, a platform for sharing data science projects. You can run and experiment with the code in a couple of ways: using free online resources (recommended) or on your own computer.

Option 1: Running using free online resources (1-click, recommended)

The easiest way to start executing this notebook is to click the "Run" button at the top of this page, and select "Run on Binder". This will run the notebook on mybinder.org, a free online service for running Jupyter notebooks. You can also select "Run on Colab" or "Run on Kaggle".

Option 2: Running on your computer locally
  1. Install Conda by following these instructions. Add Conda binaries to your system PATH, so you can use the conda command on your terminal.

  2. Create a Conda environment and install the required libraries by running these commands on the terminal:

conda create -n zerotopandas -y python=3.8 
conda activate zerotopandas
pip install jovian jupyter numpy pandas matplotlib seaborn opendatasets --upgrade
  1. Press the "Clone" button above to copy the command for downloading the notebook, and run it on the terminal. This will create a new directory and download the notebook. The command will look something like this:
jovian clone notebook-owner/notebook-id
  1. Enter the newly created directory using cd directory-name and start the Jupyter notebook.
jupyter notebook

You can now access Jupyter's web interface by clicking the link that shows up on the terminal or by visiting http://localhost:8888 on your browser. Click on the notebook file (it has a .ipynb extension) to open it.

Downloading the Dataset

TODO - add some explanation here

Instructions for downloading the dataset (delete this cell)

In [1]:
!pip install jovian opendatasets --upgrade --quiet

Let's begin by downloading the data, and listing the files within the dataset.

In [2]:
# Change this
dataset_url = 'https://www.kaggle.com/ramjidoolla/ipl-data-se' 
In [3]:
import opendatasets as od
od.download('https://www.kaggle.com/ramjidoolla/ipl-data-set')
Please provide your Kaggle credentials to download this dataset. Learn more: http://bit.ly/kaggle-creds Your Kaggle username: akankshabisht Your Kaggle Key: ········
100%|██████████| 1.24M/1.24M [00:00<00:00, 143MB/s]
Downloading ipl-data-set.zip to ./ipl-data-set

The dataset has been downloaded and extracted.

In [8]:
# Change this
data_dir = './ipl-data-set'
In [9]:
import os
os.listdir(data_dir)
Out[9]:
['deliveries.csv',
 'teams.csv',
 'teamwise_home_and_away.csv',
 'most_runs_average_strikerate.csv',
 'Players.xlsx',
 'matches.csv']

Let us save and upload our work to Jovian before continuing.

In [10]:
project_name = "IPL_Data_Analysis(2008-2019)" # change this (use lowercase letters and hyphens only)
In [11]:
!pip install jovian --upgrade -q
In [12]:
import jovian
In [13]:
jovian.commit(project=project_name)
[jovian] Attempting to save notebook.. [jovian] Please enter your API key ( from https://jovian.ml/ ): API KEY: ········ [jovian] Creating a new project "akankshabisht9897/IPL_Data_Analysis(2008-2019)" [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ml/akankshabisht9897/ipl-data-analysis-2008-2019

Data Preparation and Cleaning

In this step we will perform the following process:

  • Install and import the upgraded version of all libraries to be used in the project
  • Analyse the dataset available with us
  • Find the missing data, if possible
  • Reason of missing data
In [17]:
#Installing the upgraded version of all libraries
!pip install numpy pandas matplotlib seaborn --upgrade --quiet
In [16]:
#Importing the libraries
import numpy as np
import pandas as pd
import matplotlib
import matplotlib.pyplot as plt
import seaborn as sns
In [18]:
#Global Settings
matplotlib.rcParams['font.size'] = 14
matplotlib.rcParams['figure.figsize'] = (15, 7.5)
matplotlib.rcParams['figure.facecolor'] = '#00000000'
sns.set_style("darkgrid")
In [21]:
#Reading the matches.csv into matches_df dataframe
matches_df = pd.read_csv(data_dir +"/matches.csv")
In [22]:
#show matches_df
matches_df
Out[22]:
In [23]:
#Getting the information about the matches dataframe
matches_df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 756 entries, 0 to 755 Data columns (total 18 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 id 756 non-null int64 1 Season 756 non-null object 2 city 749 non-null object 3 date 756 non-null object 4 team1 756 non-null object 5 team2 756 non-null object 6 toss_winner 756 non-null object 7 toss_decision 756 non-null object 8 result 756 non-null object 9 dl_applied 756 non-null int64 10 winner 752 non-null object 11 win_by_runs 756 non-null int64 12 win_by_wickets 756 non-null int64 13 player_of_match 752 non-null object 14 venue 756 non-null object 15 umpire1 754 non-null object 16 umpire2 754 non-null object 17 umpire3 119 non-null object dtypes: int64(4), object(14) memory usage: 106.4+ KB
In [ ]:
matches_df.d()
In [24]:
#Dropping the column of umpires as we will not be using it for any data analysis
matches_df.drop(columns=['umpire1','umpire2','umpire3'], inplace=True)
In [25]:
#Finding the shape of the matches dataframe
matches_df.shape
Out[25]:
(756, 15)
In [26]:
#Finding the columns of matches dataframe
matches_df.columns
Out[26]:
Index(['id', 'Season', 'city', 'date', 'team1', 'team2', 'toss_winner',
       'toss_decision', 'result', 'dl_applied', 'winner', 'win_by_runs',
       'win_by_wickets', 'player_of_match', 'venue'],
      dtype='object')
In [27]:
matches_df.team1.unique()
Out[27]:
array(['Sunrisers Hyderabad', 'Mumbai Indians', 'Gujarat Lions',
       'Rising Pune Supergiant', 'Royal Challengers Bangalore',
       'Kolkata Knight Riders', 'Delhi Daredevils', 'Kings XI Punjab',
       'Chennai Super Kings', 'Rajasthan Royals', 'Deccan Chargers',
       'Kochi Tuskers Kerala', 'Pune Warriors', 'Rising Pune Supergiants',
       'Delhi Capitals'], dtype=object)
In [ ]:
 
In [ ]:
 
In [ ]:
 
In [ ]:
 
In [ ]:
 
In [ ]:
 
In [ ]:
 
In [12]:
import jovian
In [ ]:
jovian.commit()
[jovian] Attempting to save notebook..

Exploratory Analysis and Visualization

TODO - write some explanation here.

Instructions (delete this cell)

  • Compute the mean, sum, range and other interesting statistics for numeric columns
  • Explore distributions of numeric columns using histograms etc.
  • Explore relationship between columns using scatter plots, bar charts etc.
  • Make a note of interesting insights from the exploratory analysis

Let's begin by importingmatplotlib.pyplot and seaborn.

In [6]:
#Finding the number of IPL matches played till 2019
matches_df.id.count()
--------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-6-abc4e90edb6d> in <module> 1 #Finding the number of IPL matches played till 2019 ----> 2 matches_df.id.count() NameError: name 'matches_df' is not defined

TODO - Explore one or more columns by plotting a graph below, and add some explanation about it

In [5]:
#Finding the number of teams who played IPL till 2019
print(matches_df.team1.unique())
print('{} different teams played the IPL from 2008-2019'.format(matches_df.team1.unique().shape[0]))
--------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-5-3163980f1256> in <module> 1 #Finding the number of teams who played IPL till 2019 ----> 2 print(matches_df.team1.unique()) 3 print('{} different teams played the IPL from 2008-2019'.format(matches_df.team1.unique().shape[0])) NameError: name 'matches_df' is not defined
In [ ]:
 
In [ ]:
 

TODO - Explore one or more columns by plotting a graph below, and add some explanation about it

In [ ]:
 
In [ ]:
 

TODO - Explore one or more columns by plotting a graph below, and add some explanation about it

In [ ]:
 
In [ ]:
 

TODO - Explore one or more columns by plotting a graph below, and add some explanation about it

In [ ]:
 
In [ ]:
 

TODO - Explore one or more columns by plotting a graph below, and add some explanation about it

In [ ]:
 
In [ ]:
 

Let us save and upload our work to Jovian before continuing

In [ ]:
import jovian
In [ ]:
jovian.commit()

Asking and Answering Questions

TODO - write some explanation here.

Instructions (delete this cell)

  • Ask at least 5 interesting questions about your dataset
  • Answer the questions either by computing the results using Numpy/Pandas or by plotting graphs using Matplotlib/Seaborn
  • Create new columns, merge multiple dataset and perform grouping/aggregation wherever necessary
  • Wherever you're using a library function from Pandas/Numpy/Matplotlib etc. explain briefly what it does
Q1: TODO - ask a question here and answer it below
In [ ]:
 
In [ ]:
 
In [ ]:
 
Q2: TODO - ask a question here and answer it below
In [ ]:
 
In [ ]:
 
In [ ]:
 
Q3: TODO - ask a question here and answer it below
In [ ]:
 
In [ ]:
 
In [ ]:
 
Q4: TODO - ask a question here and answer it below
In [ ]:
 
In [ ]:
 
In [ ]:
 
Q5: TODO - ask a question here and answer it below
In [ ]:
 
In [ ]:
 
In [ ]:
 

Let us save and upload our work to Jovian before continuing.

In [ ]:
import jovian
In [ ]:
jovian.commit()

Inferences and Conclusion

TODO - Write some explanation here: a summary of all the inferences drawn from the analysis, and any conclusions you may have drawn by answering various questions.

In [ ]:
import jovian
In [ ]:
jovian.commit()

References and Future Work

TODO - Write some explanation here: ideas for future projects using this dataset, and links to resources you found useful.

Submission Instructions (delete this cell)

(Optional) Write a blog post

In [ ]:
import jovian
In [ ]:
jovian.commit()
In [ ]: