Insurance cost prediction using linear regression

In this assignment we're going to use information like a person's age, sex, BMI, no. of children and smoking habit to predict the price of yearly medical bills. This kind of model is useful for insurance companies to determine the yearly insurance premium for a person. The dataset for this problem is taken from: https://www.kaggle.com/mirichoi0218/insurance

We will create a model with the following steps:

Download and explore the dataset
Prepare the dataset for training
Create a linear regression model
Train the model to fit the data
Make predictions using the trained model

This assignment builds upon the concepts from the first 2 lectures. It will help to review these Jupyter notebooks:

PyTorch basics: https://jovian.ml/aakashns/01-pytorch-basics
Linear Regression: https://jovian.ml/aakashns/02-linear-regression
Logistic Regression: https://jovian.ml/aakashns/03-logistic-regression
Linear regression (minimal): https://jovian.ml/aakashns/housing-linear-minimal
Logistic regression (minimal): https://jovian.ml/aakashns/mnist-logistic-minimal

As you go through this notebook, you will find a ??? in certain places. Your job is to replace the ??? with appropriate code or values, to ensure that the notebook runs properly end-to-end . In some cases, you'll be required to choose some hyperparameters (learning rate, batch size etc.). Try to experiment with the hypeparameters to get the lowest loss.

# Uncomment and run the commands below if imports fail
#install numpy pytorch torchvision cpuonly -c pytorch -y
# !pip install matplotlib --upgrade --quiet
!pip install numpy --upgrade --quiet
!pip install jovian --upgrade --quiet
!pip install pytorch --upgrade --quiet
!pip install torchvision

  ERROR: Command errored out with exit status 1:
   command: /srv/conda/envs/notebook/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-nn5ntzsu/pytorch/setup.py'"'"'; __file__='"'"'/tmp/pip-install-nn5ntzsu/pytorch/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-_0fewy_t
       cwd: /tmp/pip-install-nn5ntzsu/pytorch/
  Complete output (5 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-nn5ntzsu/pytorch/setup.py", line 15, in <module>
      raise Exception(message)
  Exception: You tried to install "pytorch". The package named for PyTorch is "torch"
  ----------------------------------------
  ERROR: Failed building wheel for pytorch
    ERROR: Command errored out with exit status 1:
     command: /srv/conda/envs/notebook/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-nn5ntzsu/pytorch/setup.py'"'"'; __file__='"'"'/tmp/pip-install-nn5ntzsu/pytorch/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-bdcvzc6_/install-record.txt --single-version-externally-managed --compile --install-headers /srv/conda/envs/notebook/include/python3.7m/pytorch
         cwd: /tmp/pip-install-nn5ntzsu/pytorch/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-nn5ntzsu/pytorch/setup.py", line 11, in <module>
        raise Exception(message)
    Exception: You tried to install "pytorch". The package named for PyTorch is "torch"
    ----------------------------------------
ERROR: Command errored out with exit status 1: /srv/conda/envs/notebook/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-nn5ntzsu/pytorch/setup.py'"'"'; __file__='"'"'/tmp/pip-install-nn5ntzsu/pytorch/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-bdcvzc6_/install-record.txt --single-version-externally-managed --compile --install-headers /srv/conda/envs/notebook/include/python3.7m/pytorch Check the logs for full command output.
Collecting torchvision
  Downloading torchvision-0.6.0-cp37-cp37m-manylinux1_x86_64.whl (6.6 MB)
     |████████████████████████████████| 6.6 MB 4.9 MB/s eta 0:00:01     |███████████                     | 2.3 MB 4.9 MB/s eta 0:00:01
Collecting pillow>=4.1.1
  Downloading Pillow-7.1.2-cp37-cp37m-manylinux1_x86_64.whl (2.1 MB)
     |████████████████████████████████| 2.1 MB 46.1 MB/s eta 0:00:01
Requirement already satisfied: numpy in /srv/conda/envs/notebook/lib/python3.7/site-packages (from torchvision) (1.18.5)
Collecting torch==1.5.0
  Downloading torch-1.5.0-cp37-cp37m-manylinux1_x86_64.whl (752.0 MB)
     |████████████████████████████████| 752.0 MB 8.0 kB/s  eta 0:00:01   |                                | 1.9 MB 52.6 MB/s eta 0:00:15     |█▏                              | 26.9 MB 52.6 MB/s eta 0:00:14     |███▏                            | 74.3 MB 34.6 MB/s eta 0:00:20     |████▋                           | 108.0 MB 66.3 MB/s eta 0:00:10     |█████▊                          | 134.4 MB 42.7 MB/s eta 0:00:15     |██████                          | 139.2 MB 42.7 MB/s eta 0:00:15     |███████▍                        | 174.9 MB 42.7 MB/s eta 0:00:14     |████████▉                       | 206.6 MB 50.7 MB/s eta 0:00:11     |███████████████████████▍        | 549.6 MB 45.7 MB/s eta 0:00:05     |█████████████████████████▎      | 592.8 MB 44.4 MB/s eta 0:00:04     |███████████████████████████████▏| 731.9 MB 47.9 MB/s eta 0:00:01
Collecting future
  Downloading future-0.18.2.tar.gz (829 kB)
     |████████████████████████████████| 829 kB 38.3 MB/s eta 0:00:01
Building wheels for collected packages: future
  Building wheel for future (setup.py) ... done
  Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491058 sha256=1b2909c3b3c40df7d42f4ee3a75253d87d9839412acfe7dc103b347f66f3e007
  Stored in directory: /home/jovyan/.cache/pip/wheels/56/b0/fe/4410d17b32f1f0c3cf54cdfb2bc04d7b4b8f4ae377e2229ba0
Successfully built future
Installing collected packages: pillow, future, torch, torchvision
Successfully installed future-0.18.2 pillow-7.1.2 torch-1.5.0 torchvision-0.6.0

!pip install pandas

Collecting pandas
  Downloading pandas-1.0.4-cp37-cp37m-manylinux1_x86_64.whl (10.1 MB)
     |████████████████████████████████| 10.1 MB 4.5 MB/s eta 0:00:01    |█▋                              | 512 kB 4.5 MB/s eta 0:00:03
Collecting pytz>=2017.2
  Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB)
     |████████████████████████████████| 510 kB 24.7 MB/s eta 0:00:01
Requirement already satisfied: numpy>=1.13.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pandas) (1.18.5)
Requirement already satisfied: python-dateutil>=2.6.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pandas) (2.8.1)
Requirement already satisfied: six>=1.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas) (1.15.0)
Installing collected packages: pytz, pandas
Successfully installed pandas-1.0.4 pytz-2020.1

!pip install matplotlib --upgrade --quiet

import torch
import jovian
import torchvision
import torch.nn as nn
import pandas as pd
import matplotlib.pyplot as plt
import torch.nn.functional as F
from torchvision.datasets.utils import download_url
from torch.utils.data import DataLoader, TensorDataset, random_split