Jovian
⭐️
Sign In
NotebookFilesCollaborators
Version 15

Data Analysis with Python: Zero to Pandas - Course Project Guidelines

(remove this cell before submission)

Important links:

This is the starter notebook for the course project for Data Analysis with Python: Zero to Pandas. You will pick a real-world dataset of your choice and apply the concepts learned in this course to perform exploratory data analysis. Use this starter notebook as an outline for your project . Focus on documentation and presentation - this Jupyter notebook will also serve as a project report, so make sure to include detailed explanations wherever possible using Markdown cells.

Evaluation Criteria

Your submission will be evaluated using the following criteria:

  • Dataset must contain at least 3 columns and 150 rows of data
  • You must ask and answer at least 4 questions about the dataset
  • Your submission must include at least 4 visualizations (graphs)
  • Your submission must include explanations using markdown cells, apart from the code.
  • Your work must not be plagiarized i.e. copy-pasted for somewhere else.

Follow this step-by-step guide to work on your project.

Step 1: Select a real-world dataset

Here's some sample code for downloading the US Elections Dataset:

import opendatasets as od
dataset_url = 'https://www.kaggle.com/tunguz/us-elections-dataset'
od.download('https://www.kaggle.com/tunguz/us-elections-dataset')

You can find a list of recommended datasets here: https://jovian.ml/forum/t/recommended-datasets-for-course-project/11711

Step 2: Perform data preparation & cleaning

  • Load the dataset into a data frame using Pandas
  • Explore the number of rows & columns, ranges of values etc.
  • Handle missing, incorrect and invalid data
  • Perform any additional steps (parsing dates, creating additional columns, merging multiple dataset etc.)

Step 3: Perform exploratory analysis & visualization

  • Compute the mean, sum, range and other interesting statistics for numeric columns
  • Explore distributions of numeric columns using histograms etc.
  • Explore relationship between columns using scatter plots, bar charts etc.
  • Make a note of interesting insights from the exploratory analysis

Step 4: Ask & answer questions about the data

  • Ask at least 4 interesting questions about your dataset
  • Answer the questions either by computing the results using Numpy/Pandas or by plotting graphs using Matplotlib/Seaborn
  • Create new columns, merge multiple dataset and perform grouping/aggregation wherever necessary
  • Wherever you're using a library function from Pandas/Numpy/Matplotlib etc. explain briefly what it does

Step 5: Summarize your inferences & write a conclusion

  • Write a summary of what you've learned from the analysis
  • Include interesting insights and graphs from previous sections
  • Share ideas for future work on the same topic using other relevant datasets
  • Share links to resources you found useful during your analysis

Step 6: Make a submission & share your work

(Optional) Step 7: Write a blog post

Example Projects

Refer to these projects for inspiration:

NOTE: Remove this cell containing the instructions before making your submission. You can do using the "Edit > Delete Cells" menu option.

Project Title - change this

TODO - Write some introduction about your project here: describe the dataset, where you got it from, what you're trying to do with it, and which tools & techniques you're using. You can also mention about the course Data Analysis with Python: Zero to Pandas, and what you've learned from it.

How to run the code

This is an executable Jupyter notebook hosted on Jovian.ml, a platform for sharing data science projects. You can run and experiment with the code in a couple of ways: using free online resources (recommended) or on your own computer.

Option 1: Running using free online resources (1-click, recommended)

The easiest way to start executing this notebook is to click the "Run" button at the top of this page, and select "Run on Binder". This will run the notebook on mybinder.org, a free online service for running Jupyter notebooks. You can also select "Run on Colab" or "Run on Kaggle".

Option 2: Running on your computer locally

  1. Install Conda by following these instructions. Add Conda binaries to your system PATH, so you can use the conda command on your terminal.

  2. Create a Conda environment and install the required libraries by running these commands on the terminal:

conda create -n zerotopandas -y python=3.8 
conda activate zerotopandas
pip install jovian jupyter numpy pandas matplotlib seaborn opendatasets --upgrade
  1. Press the "Clone" button above to copy the command for downloading the notebook, and run it on the terminal. This will create a new directory and download the notebook. The command will look something like this:
jovian clone notebook-owner/notebook-id
  1. Enter the newly created directory using cd directory-name and start the Jupyter notebook.
jupyter notebook

You can now access Jupyter's web interface by clicking the link that shows up on the terminal or by visiting http://localhost:8888 on your browser. Click on the notebook file (it has a .ipynb extension) to open it.

Downloading the Dataset

TODO - add some explanation here

In [42]:
!pip install jovian opendatasets --upgrade --quiet

Let's begin by downloading the data, and listing the files within the dataset.

The dataset has been downloaded and extracted.

In [43]:
import pandas as pd
import numpy as np
In [44]:
from urllib.request import urlretrieve

urlretrieve('https://hub.jovian.ml/wp-content/uploads/2020/09/countries.csv', 
            'countries.csv')
Out[44]:
('countries.csv', <http.client.HTTPMessage at 0x7f36fd1b52b0>)

Let us save and upload our work to Jovian before continuing.

In [45]:
project_name = "course-project-starter-pro" # change this (use lowercase letters and hyphens only)
In [46]:
!pip install jovian --upgrade -q
In [47]:
import jovian
In [48]:
jovian.commit(project=project_name)
[jovian] Attempting to save notebook.. [jovian] Updating notebook "prapathade1111/course-project-starter-pro" on https://jovian.ai [jovian] Uploading notebook.. [jovian] Uploading additional files... [jovian] Committed successfully! https://jovian.ai/prapathade1111/course-project-starter-pro
Out[48]:
'https://jovian.ai/prapathade1111/course-project-starter-pro'

Data Preparation and Cleaning

we have downlaoded dataset from website. Next step is uploading csv file in dataframe

In [49]:
countries_df = pd.read_csv('countries.csv')
In [50]:
countries_df
Out[50]:
In [51]:
countries_df.shape
Out[51]:
(210, 6)
In [52]:
countries_df.count()
Out[52]:
location                      210
continent                     210
population                    210
life_expectancy               207
hospital_beds_per_thousand    164
gdp_per_capita                183
dtype: int64
In [53]:
countries_df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 210 entries, 0 to 209 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 location 210 non-null object 1 continent 210 non-null object 2 population 210 non-null float64 3 life_expectancy 207 non-null float64 4 hospital_beds_per_thousand 164 non-null float64 5 gdp_per_capita 183 non-null float64 dtypes: float64(4), object(2) memory usage: 10.0+ KB
In [54]:
countries_df.describe()
Out[54]:
In [55]:
import jovian
In [56]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "prapathade1111/course-project-starter-pro" on https://jovian.ai [jovian] Uploading notebook.. [jovian] Uploading additional files... [jovian] Committed successfully! https://jovian.ai/prapathade1111/course-project-starter-pro
Out[56]:
'https://jovian.ai/prapathade1111/course-project-starter-pro'

Exploratory Analysis and Visualization

  • we Compute the mean, sum & interesting statistics for numeric columns
  • We explore distributions of numeric columns using histograms
  • we also Explore relationship between columns using scatter plot & bar charts
In [57]:
countries_df.groupby('continent').mean()
Out[57]:
In [58]:
countries_df['population'].sum()
Out[58]:
7757980095.0
In [59]:
countries_df['life_expectancy'].sum()
Out[59]:
15220.68
In [60]:
countries_df['hospital_beds_per_thousand'].sum()
Out[60]:
494.078
In [61]:
countries_df['gdp_per_capita'].sum()
Out[61]:
3565921.9689999996
In [62]:
countries_df['gdp_per_capita'].mean()
Out[62]:
19485.912398907098

Let's begin by importingmatplotlib.pyplot and seaborn.

In [63]:
import seaborn as sns
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

sns.set_style('darkgrid')
matplotlib.rcParams['font.size'] = 14
matplotlib.rcParams['figure.figsize'] = (9, 5)
matplotlib.rcParams['figure.facecolor'] = '#00000000'

ploted bar graph for population of continents also analyzing data with histogram

In [64]:
plt.figure(figsize=(12,6))
plt.xticks(rotation=75)
plt.title('population of continents')
sns.barplot(x=countries_df['continent'], y=countries_df['population']);
In [65]:
plt.figure(figsize=(12, 6))
plt.title('population of continents')
plt.xlabel('population')
plt.ylabel('counts')

plt.hist(countries_df['population'], bins=(10), color='purple');

ploted bar graph for 'life_expectancy of continents' also analyzing data with histogram

In [66]:
plt.figure(figsize=(12,6))
plt.xticks(rotation=75)
plt.title('life_expectancy of continents')
sns.barplot(x=countries_df['continent'], y=countries_df['life_expectancy']);
In [67]:
plt.figure(figsize=(12, 6))
plt.title('life_expectancy of continents')
plt.xlabel('life_expectancy')
plt.ylabel('counts')

plt.hist(countries_df['life_expectancy'], bins=(10), color='red');

ploted bar graph for 'hospital_beds_per_thousand of continents' also analyzing data with histogram

In [68]:
plt.figure(figsize=(12,6))
plt.xticks(rotation=75)
plt.title('hospital_beds_per_thousand of continents')
sns.barplot(x=countries_df['continent'], y=countries_df['hospital_beds_per_thousand']);
In [69]:
plt.figure(figsize=(12, 6))
plt.title('hospital_beds_per_thousand of continents')
plt.xlabel('hospital_beds_per_thousand')
plt.ylabel('counts')

plt.hist(countries_df['hospital_beds_per_thousand'], bins=(10), color='red');

ploted bar graph for 'gdp_per_capita of continents' also analyzing data with histogram

In [70]:
plt.figure(figsize=(12,6))
plt.xticks(rotation=75)
plt.title('gdp_per_capita of continents')
sns.barplot(x=countries_df['continent'], y=countries_df['gdp_per_capita']);
In [71]:
plt.figure(figsize=(12, 6))
plt.title('gdp_per_capita of continents')
plt.xlabel('gdp_per_capita')
plt.ylabel('counts')

plt.hist(countries_df['gdp_per_capita'], bins=(10), color='Blue');

Ploted scatterplot between population & life_expectancy also analyzing data with histogram

In [72]:
sns.scatterplot( x=countries_df['population'], y=countries_df['life_expectancy'], hue=countries_df['continent'])
Out[72]:
<AxesSubplot:xlabel='population', ylabel='life_expectancy'>

Ploted scatterplot between population & gdp_per_capita also analyzing data with histogram

In [73]:
sns.scatterplot( x=countries_df['population'], y=countries_df['gdp_per_capita'], hue=countries_df['continent'])
Out[73]:
<AxesSubplot:xlabel='population', ylabel='gdp_per_capita'>

Ploted scatterplot between life_expectancy & gdp_per_capita also analyzing data with histogram

In [74]:
sns.scatterplot( x=countries_df['life_expectancy'], y=countries_df['gdp_per_capita'], hue=countries_df['continent'])
Out[74]:
<AxesSubplot:xlabel='life_expectancy', ylabel='gdp_per_capita'>

Let us save and upload our work to Jovian before continuing

In [75]:
import jovian
In [76]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "prapathade1111/course-project-starter-pro" on https://jovian.ai [jovian] Uploading notebook.. [jovian] Uploading additional files... [jovian] Committed successfully! https://jovian.ai/prapathade1111/course-project-starter-pro
Out[76]:
'https://jovian.ai/prapathade1111/course-project-starter-pro'

Asking and Answering Questions

TODO - write some explanation here.

Q: Add a new column in countries_df to record the overall GDP per country (product of population & per capita GDP).

In [77]:
countries_df['gdp'] = countries_df['population']*countries_df['gdp_per_capita']
In [78]:
countries_df
Out[78]:

Q: Create a data frame showing the total population of each continent.

In [79]:
continent_populations_df = countries_df.groupby(["continent"]).sum().reset_index()[["continent","population"]]
In [80]:
continent_populations_df
Out[80]:

Q:Create a dataframe containing 10 countries with the lowest GDP per capita, among the counties with population greater than 100 million.

In [81]:
Low_GDP = countries_df.sort_values(by='gdp', ascending=True)
In [82]:
Mill = Low_GDP.groupby('population')
In [83]:
Mill.head(10)
Out[83]:

Q: Create graph showing the life_expectancy of each continent.

In [84]:
continent_populations_df = countries_df.groupby(["continent"]).sum().reset_index()[["continent","life_expectancy"]]
In [85]:
continent_populations_df
Out[85]:
In [86]:
sns.barplot(x=continent_populations_df['continent'] , y=continent_populations_df['life_expectancy'])
Out[86]:
<AxesSubplot:xlabel='continent', ylabel='life_expectancy'>

Q. Find the correlation between population & gdp

In [87]:
sns.scatterplot( x=countries_df['population'], y=countries_df['gdp'], hue=countries_df['continent'])
Out[87]:
<AxesSubplot:xlabel='population', ylabel='gdp'>

Let us save and upload our work to Jovian before continuing.

In [88]:
import jovian
In [89]:
jovian.commit()
[jovian] Attempting to save notebook.. [jovian] Updating notebook "prapathade1111/course-project-starter-pro" on https://jovian.ai [jovian] Uploading notebook.. [jovian] Uploading additional files... [jovian] Committed successfully! https://jovian.ai/prapathade1111/course-project-starter-pro
Out[89]:
'https://jovian.ai/prapathade1111/course-project-starter-pro'

Inferences and Conclusion

  1. Life expectency of Europe is very high
  2. Asia is highest populated country
  3. life_expectancy & gdp_per_capita having exponetial curve ,very low for africa & high for asia
In [90]:
import jovian
In [ ]:
jovian.commit()
[jovian] Attempting to save notebook..
In [ ]: