Jovian
⭐️
Sign In
NotebookFilesCollaborators
Version 18

Data Analysis with Python: Zero to Pandas - Course Project Guidelines

(remove this cell before submission)

Important links:

This is the starter notebook for the course project for Data Analysis with Python: Zero to Pandas. You will pick a real-world dataset of your choice and apply the concepts learned in this course to perform exploratory data analysis. Use this starter notebook as an outline for your project . Focus on documentation and presentation - this Jupyter notebook will also serve as a project report, so make sure to include detailed explanations wherever possible using Markdown cells.

Evaluation Criteria

Your submission will be evaluated using the following criteria:

  • Dataset must contain at least 3 columns and 150 rows of data
  • You must ask and answer at least 4 questions about the dataset
  • Your submission must include at least 4 visualizations (graphs)
  • Your submission must include explanations using markdown cells, apart from the code.
  • Your work must not be plagiarized i.e. copy-pasted for somewhere else.

Follow this step-by-step guide to work on your project.

Step 1: Select a real-world dataset

Here's some sample code for downloading the US Elections Dataset:

import opendatasets as od
dataset_url = 'https://www.kaggle.com/tunguz/us-elections-dataset'
od.download('https://www.kaggle.com/tunguz/us-elections-dataset')

You can find a list of recommended datasets here: https://jovian.ml/forum/t/recommended-datasets-for-course-project/11711

Step 2: Perform data preparation & cleaning

  • Load the dataset into a data frame using Pandas
  • Explore the number of rows & columns, ranges of values etc.
  • Handle missing, incorrect and invalid data
  • Perform any additional steps (parsing dates, creating additional columns, merging multiple dataset etc.)

Step 3: Perform exploratory analysis & visualization

  • Compute the mean, sum, range and other interesting statistics for numeric columns
  • Explore distributions of numeric columns using histograms etc.
  • Explore relationship between columns using scatter plots, bar charts etc.
  • Make a note of interesting insights from the exploratory analysis

Step 4: Ask & answer questions about the data

  • Ask at least 4 interesting questions about your dataset
  • Answer the questions either by computing the results using Numpy/Pandas or by plotting graphs using Matplotlib/Seaborn
  • Create new columns, merge multiple dataset and perform grouping/aggregation wherever necessary
  • Wherever you're using a library function from Pandas/Numpy/Matplotlib etc. explain briefly what it does

Step 5: Summarize your inferences & write a conclusion

  • Write a summary of what you've learned from the analysis
  • Include interesting insights and graphs from previous sections
  • Share ideas for future work on the same topic using other relevant datasets
  • Share links to resources you found useful during your analysis

Step 6: Make a submission & share your work

(Optional) Step 7: Write a blog post

Example Projects

Refer to these projects for inspiration:

NOTE: Remove this cell containing the instructions before making your submission. You can do using the "Edit > Delete Cells" menu option.

Project Title - change this

TODO - Write some introduction about your project here: describe the dataset, where you got it from, what you're trying to do with it, and which tools & techniques you're using. You can also mention about the course Data Analysis with Python: Zero to Pandas, and what you've learned from it.

How to run the code

This is an executable Jupyter notebook hosted on Jovian.ml, a platform for sharing data science projects. You can run and experiment with the code in a couple of ways: using free online resources (recommended) or on your own computer.

Option 1: Running using free online resources (1-click, recommended)

The easiest way to start executing this notebook is to click the "Run" button at the top of this page, and select "Run on Binder". This will run the notebook on mybinder.org, a free online service for running Jupyter notebooks. You can also select "Run on Colab" or "Run on Kaggle".

Option 2: Running on your computer locally

  1. Install Conda by following these instructions. Add Conda binaries to your system PATH, so you can use the conda command on your terminal.

  2. Create a Conda environment and install the required libraries by running these commands on the terminal:

conda create -n zerotopandas -y python=3.8 
conda activate zerotopandas
pip install jovian jupyter numpy pandas matplotlib seaborn opendatasets --upgrade
  1. Press the "Clone" button above to copy the command for downloading the notebook, and run it on the terminal. This will create a new directory and download the notebook. The command will look something like this:
jovian clone notebook-owner/notebook-id
  1. Enter the newly created directory using cd directory-name and start the Jupyter notebook.
jupyter notebook

You can now access Jupyter's web interface by clicking the link that shows up on the terminal or by visiting http://localhost:8888 on your browser. Click on the notebook file (it has a .ipynb extension) to open it.

Downloading the Dataset

TODO - add some explanation here

Instructions for downloading the dataset (delete this cell)

In [1]:
!pip install jovian opendatasets --upgrade --quiet

Let's begin by downloading the data, and listing the files within the dataset.

In [2]:
# Change this
dataset_url = 'https://www.kaggle.com/rishidamarla/video-game-sales'
In [6]:
import opendatasets as od
od.download(dataset_url)
Please provide your Kaggle credentials to download this dataset. Learn more: http://bit.ly/kaggle-creds Your Kaggle username: akariiiii Your Kaggle Key: ········
100%|██████████| 476k/476k [00:00<00:00, 85.7MB/s]
Downloading video-game-sales.zip to ./video-game-sales

The dataset has been downloaded and extracted.

In [7]:
# Change this
data_dir = './video-game-sales'
In [8]:
import os
os.listdir(data_dir)
Out[8]:
['Video_Games.csv']

Let us save and upload our work to Jovian before continuing.

In [9]:
project_name = "data-analysis-of-video-game-sales" # change this (use lowercase letters and hyphens only)
In [10]:
!pip install jovian --upgrade -q
In [11]:
import jovian
In [12]:
jovian.commit(project=project_name)
[jovian] Attempting to save notebook.. [jovian] Please enter your API key ( from https://jovian.ml/ ): API KEY: ········ [jovian] Updating notebook "indexkyou/data-analysis-of-video-game-sales" on https://jovian.ml/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ml/indexkyou/data-analysis-of-video-game-sales
Out[12]:
'https://jovian.ml/indexkyou/data-analysis-of-video-game-sales'

Data Preparation and Cleaning

TODO - Write some explanation here.

Instructions (delete this cell):

  • Load the dataset into a data frame using Pandas
  • Explore the number of rows & columns, ranges of values etc.
  • Handle missing, incorrect and invalid data
  • Perform any additional steps (parsing dates, creating additional columns, merging multiple dataset etc.)
In [13]:
import pandas as pd
In [55]:
game_sales_df = pd.read_csv('./video-game-sales/Video_Games.csv')
In [56]:
game_sales_df
Out[56]:
In [57]:
game_sales_df.columns
Out[57]:
Index(['Name', 'Platform', 'Year_of_Release', 'Genre', 'Publisher', 'NA_Sales',
       'EU_Sales', 'JP_Sales', 'Other_Sales', 'Global_Sales', 'Critic_Score',
       'Critic_Count', 'User_Score', 'User_Count', 'Developer', 'Rating'],
      dtype='object')
In [58]:
game_sales_df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 16719 entries, 0 to 16718 Data columns (total 16 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Name 16717 non-null object 1 Platform 16719 non-null object 2 Year_of_Release 16450 non-null float64 3 Genre 16717 non-null object 4 Publisher 16665 non-null object 5 NA_Sales 16719 non-null float64 6 EU_Sales 16719 non-null float64 7 JP_Sales 16719 non-null float64 8 Other_Sales 16719 non-null float64 9 Global_Sales 16719 non-null float64 10 Critic_Score 8137 non-null float64 11 Critic_Count 8137 non-null float64 12 User_Score 10015 non-null object 13 User_Count 7590 non-null float64 14 Developer 10096 non-null object 15 Rating 9950 non-null object dtypes: float64(9), object(7) memory usage: 2.0+ MB

Look at the info we can see that not every game is rating and get critic score. We should take a closer look at the description.

In [61]:
game_sales_df.describe()
Out[61]:

So we have around 16450 game titles that was sold between 1980 and 2020. NA seems like the biggest market to sell game.

In [62]:
import jovian
In [ ]:
jovian.commit()
[jovian] Attempting to save notebook..

Exploratory Analysis and Visualization

At first look the dataframe is already sorted by Global_Sales. But for better viewer we should try creating a few graph.

Instructions (delete this cell)

  • Compute the mean, sum, range and other interesting statistics for numeric columns
  • Explore distributions of numeric columns using histograms etc.
  • Explore relationship between columns using scatter plots, bar charts etc.
  • Make a note of interesting insights from the exploratory analysis

Let's begin by importingmatplotlib.pyplot and seaborn.

In [28]:
import seaborn as sns
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

sns.set_style('darkgrid')
matplotlib.rcParams['font.size'] = 13
matplotlib.rcParams['figure.figsize'] = (36, 20)
matplotlib.rcParams['figure.facecolor'] = '#00000000'

First, We should see the total sales of games each year. It helps us know when video game is declined and when is it popular.

In [29]:
sns.countplot('Year_of_Release', data = game_sales_df)
plt.title('Total game sale each year')
plt.show()

So in 2008 and 2009 video game is really popular. Let see which game is the best seller in 2008 and 2009

In [30]:
list_games_2008 = game_sales_df.loc[game_sales_df['Year_of_Release'] == 2008]
list_games_2008.sort_values('Global_Sales',ascending = False).head(10)
Out[30]:
In [31]:
list_games_2009 = game_sales_df.loc[game_sales_df['Year_of_Release'] == 2009]
list_games_2009.sort_values('Global_Sales',ascending = False).head(10)
Out[31]:

In 2008 and 2009, the most popular game is from Wii platform. That's pretty interesting let see the devide graph for platform (We should combine two dataframe as well)

In [32]:
combine_list = list_games_2008.append(list_games_2009)
platform_counts = combine_list.Platform.value_counts()
plt.figure(figsize=(24,12))
plt.pie(platform_counts, labels=platform_counts.index, autopct='%1.1f%%', startangle=180);

Overall DS and Wii did the best in these year. Nitendo was really good back in time. We should properly check how Nintendo is doing up until now.

In [49]:
nitendo_df = game_sales_df.loc[game_sales_df['Publisher'] == 'Nintendo']
nitendo_df = nitendo_df.groupby('Year_of_Release').count()
nitendo_df = nitendo_df['Global_Sales']

plt.title("Nintendo sales")
sns.barplot(nitendo_df.index, nitendo_df);
In [41]:
Out[41]:
In [43]:
sns.catplot(x = "Year_of_Release", y = "Global_Sales", kind = "point",
            data = publisher_df,
            hue = "Publisher",
            ci = None,
            edgecolor=None,
            height=10, 
            aspect=36/12)
plt.show()
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-43-df44de01ffa1> in <module> ----> 1 sns.catplot(x = "Year_of_Release", y = "Global_Sales", kind = "point", 2 data = publisher_df, 3 ci = None, 4 edgecolor=None, 5 height=10, /srv/conda/envs/notebook/lib/python3.8/site-packages/seaborn/categorical.py in catplot(x, y, hue, data, row, col, col_wrap, estimator, ci, n_boot, units, seed, order, hue_order, row_order, col_order, kind, height, aspect, orient, color, palette, legend, legend_out, sharex, sharey, margin_titles, facet_kws, **kwargs) 3721 # facets to ensure representation of all data in the final plot 3722 p = _CategoricalPlotter() -> 3723 p.establish_variables(x_, y_, hue, data, orient, order, hue_order) 3724 order = p.group_names 3725 hue_order = p.hue_names /srv/conda/envs/notebook/lib/python3.8/site-packages/seaborn/categorical.py in establish_variables(self, x, y, hue, data, orient, order, hue_order, units) 150 if isinstance(var, str): 151 err = "Could not interpret input '{}'".format(var) --> 152 raise ValueError(err) 153 154 # Figure out the plotting orientation ValueError: Could not interpret input 'Year_of_Release'

Ok this look a bit complicated let narrow down it to 2007 to 2016.

In [ ]:
new_data = game_sales_df.loc[game_sales_df['Year_of_Release'] > 2007]
sns.catplot(x = "Year_of_Release", y = "Global_Sales", kind = "point",
            data = new_data,
            hue = "Platform",
            palette='Set1',
            ci = None,
            edgecolor=None,
            height=5, 
            aspect=36/12)
plt.show()

Look at the graph, we can see that sadly video game is declined.

TODO - Explore one or more columns by plotting a graph below, and add some explanation about it

In [ ]:
In [ ]:

Let us save and upload our work to Jovian before continuing

In [106]:
import jovian
In [107]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "indexkyou/data-analysis-of-video-game-sales" on https://jovian.ml/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ml/indexkyou/data-analysis-of-video-game-sales
Out[107]:
'https://jovian.ml/indexkyou/data-analysis-of-video-game-sales'

Asking and Answering Questions

TODO - write some explanation here.

Instructions (delete this cell)

  • Ask at least 5 interesting questions about your dataset
  • Answer the questions either by computing the results using Numpy/Pandas or by plotting graphs using Matplotlib/Seaborn
  • Create new columns, merge multiple dataset and perform grouping/aggregation wherever necessary
  • Wherever you're using a library function from Pandas/Numpy/Matplotlib etc. explain briefly what it does
Q1: TODO - ask a question here and answer it below
In [ ]:
In [ ]:
In [ ]:
Q2: TODO - ask a question here and answer it below
In [ ]:
In [ ]:
In [ ]:
Q3: TODO - ask a question here and answer it below
In [ ]:
In [ ]:
In [ ]:
Q4: TODO - ask a question here and answer it below
In [ ]:
In [ ]:
In [ ]:
Q5: TODO - ask a question here and answer it below
In [ ]:
In [ ]:
In [ ]:

Let us save and upload our work to Jovian before continuing.

In [28]:
import jovian
In [29]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "aakashns/zerotopandas-course-project-starter" on https://jovian.ml/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ml/aakashns/zerotopandas-course-project-starter
Out[29]:
'https://jovian.ml/aakashns/zerotopandas-course-project-starter'

Inferences and Conclusion

TODO - Write some explanation here: a summary of all the inferences drawn from the analysis, and any conclusions you may have drawn by answering various questions.

In [30]:
import jovian
In [31]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "aakashns/zerotopandas-course-project-starter" on https://jovian.ml/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ml/aakashns/zerotopandas-course-project-starter
Out[31]:
'https://jovian.ml/aakashns/zerotopandas-course-project-starter'

References and Future Work

TODO - Write some explanation here: ideas for future projects using this dataset, and links to resources you found useful.

Submission Instructions (delete this cell)

(Optional) Write a blog post

In [32]:
import jovian
In [35]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "aakashns/zerotopandas-course-project-starter" on https://jovian.ml/ [jovian] Uploading notebook.. [jovian] Capturing environment.. [jovian] Committed successfully! https://jovian.ml/aakashns/zerotopandas-course-project-starter
Out[35]:
'https://jovian.ml/aakashns/zerotopandas-course-project-starter'
In [ ]: