adamchan-215/pandas-practice-assignment - Jovian
Learn data science and machine learning by building real-world projects on Jovian

Assignment 3 - Pandas Data Analysis Practice

This assignment is a part of the course "Data Analysis with Python: Zero to Pandas"

In this assignment, you'll get to practice some of the concepts and skills covered this tutorial: https://jovian.ml/aakashns/python-pandas-data-analysis

As you go through this notebook, you will find a ??? in certain places. To complete this assignment, you must replace all the ??? with appropriate values, expressions or statements to ensure that the notebook runs properly end-to-end.

Some things to keep in mind:

  • Make sure to run all the code cells, otherwise you may get errors like NameError for undefined variables.
  • Do not change variable names, delete cells or disturb other existing code. It may cause problems during evaluation.
  • In some cases, you may need to add some code cells or new statements before or after the line of code containing the ???.
  • Since you'll be using a temporary online service for code execution, save your work by running jovian.commit at regular intervals.
  • Questions marked (Optional) will not be considered for evaluation, and can be skipped. They are for your learning.

You can make submissions on this page: https://jovian.ml/learn/data-analysis-with-python-zero-to-pandas/assignment/assignment-3-pandas-practice

If you are stuck, you can ask for help on the community forum: https://jovian.ml/forum/t/assignment-3-pandas-practice/11225/3 . You can get help with errors or ask for hints, describe your approach in simple words, link to documentation, but please don't ask for or share the full working answer code on the forum.

How to run the code and save your work

The recommended way to run this notebook is to click the "Run" button at the top of this page, and select "Run on Binder". This will run the notebook on mybinder.org, a free online service for running Jupyter notebooks.

Before staring the assignment, let's save a snapshot of the assignment to your Jovian.ml profile, so that you can access it later, and continue your work.

import jovian
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Creating a new project "adamchan-215/pandas-practice-assignment" [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment
# Run the next line to install Pandas
!pip install pandas --upgrade
Collecting pandas Downloading pandas-1.3.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB) |████████████████████████████████| 11.7 MB 5.6 MB/s eta 0:00:01 Requirement already satisfied, skipping upgrade: python-dateutil>=2.7.3 in /opt/conda/lib/python3.8/site-packages (from pandas) (2.8.1) Requirement already satisfied, skipping upgrade: pytz>=2017.3 in /opt/conda/lib/python3.8/site-packages (from pandas) (2020.1) Requirement already satisfied, skipping upgrade: numpy>=1.17.3 in /opt/conda/lib/python3.8/site-packages (from pandas) (1.19.2) Requirement already satisfied, skipping upgrade: six>=1.5 in /opt/conda/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas) (1.15.0) Installing collected packages: pandas Attempting uninstall: pandas Found existing installation: pandas 1.1.3 Uninstalling pandas-1.1.3: Successfully uninstalled pandas-1.1.3 Successfully installed pandas-1.3.1
import pandas as pd

In this assignment, we're going to analyze an operate on data from a CSV file. Let's begin by downloading the CSV file.

from urllib.request import urlretrieve

urlretrieve('https://hub.jovian.ml/wp-content/uploads/2020/09/countries.csv', 
            'countries.csv')
('countries.csv', <http.client.HTTPMessage at 0x7f24ce51c9d0>)

Let's load the data from the CSV file into a Pandas data frame.

countries_df = pd.read_csv('countries.csv')
countries_df

Q1: How many countries does the dataframe contain?

Hint: Use the .shape method.

num_countries = countries_df['location'].shape[0]
print('There are {} countries in the dataset'.format(num_countries))
There are 210 countries in the dataset
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q2: Retrieve a list of continents from the dataframe?

Hint: Use the .unique method of a series.

continents = countries_df.continent.unique()
continents
array(['Asia', 'Europe', 'Africa', 'North America', 'South America',
       'Oceania'], dtype=object)
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q3: What is the total population of all the countries listed in this dataset?

total_population = countries_df.population.sum()
print('The total population is {}.'.format(int(total_population)))
The total population is 7757980095.
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q: (Optional) What is the overall life expectancy across in the world?

Hint: You'll need to take a weighted average of life expectancy using populations as weights.

countries_df
weights = countries_df.population / total_population
overall_life_expectancy = (countries_df.life_expectancy * weights).sum()
print("The overall life expectancy across in the world is {}".format(overall_life_expectancy))
The overall life expectancy across in the world is 72.72165193409664
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q4: Create a dataframe containing 10 countries with the highest population.

Hint: Chain the sort_values and head methods.

most_populous_df = countries_df.sort_values("population", ascending = False).head(10)
most_populous_df
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q5: Add a new column in countries_df to record the overall GDP per country (product of population & per capita GDP).

countries_df['gdp'] = countries_df.population * countries_df.gdp_per_capita
countries_df
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q: (Optional) Create a dataframe containing 10 countries with the lowest GDP per capita, among the counties with population greater than 100 million.

countries_df[countries_df['population'] > 100e6].sort_values('gdp_per_capita', ascending = True).head(10)
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q6: Create a data frame that counts the number countries in each continent?

Hint: Use groupby, select the location column and aggregate using count.

country_counts_df = countries_df.groupby('continent')[['location']].count()
country_counts_df
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q7: Create a data frame showing the total population of each continent.

Hint: Use groupby, select the population column and aggregate using sum.

continent_populations_df = countries_df.groupby('continent')[['population']].sum()
continent_populations_df
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Let's download another CSV file containing overall Covid-19 stats for various countires, and read the data into another Pandas data frame.

urlretrieve('https://hub.jovian.ml/wp-content/uploads/2020/09/covid-countries-data.csv', 
            'covid-countries-data.csv')
('covid-countries-data.csv', <http.client.HTTPMessage at 0x7f24ce2f9070>)
covid_data_df = pd.read_csv('covid-countries-data.csv')
covid_data_df

Q8: Count the number of countries for which the total_tests data is missing.

Hint: Use the .isna method.

total_tests_missing = covid_data_df['total_tests'].isna().sum()
print("The data for total tests is missing for {} countries.".format(int(total_tests_missing)))
The data for total tests is missing for 122 countries.
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Let's merge the two data frames, and compute some more metrics.

Q9: Merge countries_df with covid_data_df on the location column.

*Hint: Use the .merge method on countries_df.

combined_df = countries_df.merge(covid_data_df, on = 'location')
combined_df
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q10: Add columns tests_per_million, cases_per_million and deaths_per_million into combined_df.

combined_df['tests_per_million'] = combined_df['total_tests'] * 1e6 / combined_df['population']
combined_df['cases_per_million'] = combined_df['total_cases'] * 1e6 / combined_df['population']
combined_df['deaths_per_million'] = combined_df['total_deaths'] * 1e6 / combined_df['population']
combined_df
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q11: Create a dataframe with 10 countires that have highest number of tests per million people.

highest_tests_df = combined_df.sort_values("tests_per_million", ascending = False).head(10)
highest_tests_df
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q12: Create a dataframe with 10 countires that have highest number of positive cases per million people.

highest_cases_df = combined_df.sort_values("cases_per_million", ascending = False).head(10)
highest_cases_df
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Q13: Create a dataframe with 10 countires that have highest number of deaths cases per million people?

highest_deaths_df = combined_df.sort_values("deaths_per_million", ascending = False).head(10)
highest_deaths_df
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

(Optional) Q: Count number of countries that feature in both the lists of "highest number of tests per million" and "highest number of cases per million".

common_countries = highest_tests_df.merge(highest_cases_df, on = 'location')
print("The number of common countries in both list is", common_countries.location.count())
The number of common countries in both list is 2
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

(Optional) Q: Count number of countries that feature in both the lists "20 countries with lowest GDP per capita" and "20 countries with the lowest number of hospital beds per thousand population". Only consider countries with a population higher than 10 million while creating the list.

lowest_GDP_df = combined_df[combined_df['population'] > 10e6].sort_values("gdp_per_capita").head(20)
lowest_GDP_df
lowest_hospital_beds_df = combined_df[combined_df['population'] > 10e6].sort_values("hospital_beds_per_thousand").head(20)
lowest_hospital_beds_df
common_df = lowest_GDP_df.merge(lowest_hospital_beds_df, on = 'location')
common_df
print("There are {} countries featuring in both lists.".format(common_df.location.count()))
There are 14 countries featuring in both lists.
import jovian
jovian.commit(project='pandas-practice-assignment', environment=None)
[jovian] Updating notebook "adamchan-215/pandas-practice-assignment" on https://jovian.ai [jovian] Committed successfully! https://jovian.ai/adamchan-215/pandas-practice-assignment

Submission

Congratulations on making it this far! You've reached the end of this assignment, and you just completed your first real-world data analysis problem. It's time to record one final version of your notebook for submission.

Make a submission here by filling the submission form: https://jovian.ml/learn/data-analysis-with-python-zero-to-pandas/assignment/assignment-3-pandas-practice

Also make sure to help others on the forum: https://jovian.ml/forum/t/assignment-3-pandas-practice/11225/2

jovian.submit(assignment="zero-to-pandas-a3")