Jovian
⭐️
Sign In

Image Classification of 250 Bird Species dataset

In this project, I trained a ResNet-n (n=9) neural networks architecture with a different layers to classify a diverse set of 250 Bird Species from the Kaggle dataset. In this project, I used the 250 Birds Species Dataset, which consists of 250 bird species. 35215 training images, 1250 test images(5 X 250 species) and 1250 validation images(5 X 250 species). All images are 224 X 224 X 3 color images in jpg format. Also includes a “consolidated” image set that combines the training, test and validation images into a single data set.

Let's start by installing jovian library for link my notebook with my profile.

In [114]:
# Jovian Commit Essentials
# Please retain and execute this cell without modifying the contents for `jovian.commit` to work
!pip install jovian --upgrade -q
import jovian
# jovian.utils.colab.set_colab_file_id('1lgTTsO6CuYS49m24tKeRN6uSpkipyE3Q')
jovian.utils.colab.set_colab_file_id('1VYHHrXA8mAi2fQi5tTmjiTLCa0bc_WjL')

Mounting the google drive

In [ ]:
from google.colab import drive
drive.mount('/content/gdrive')
Mounted at /content/gdrive

Setting the default directory as pwd

In [ ]:
import os
os.environ['KAGGLE_CONFIG_DIR'] = "/content/gdrive/My Drive/Kaggle"
In [ ]:
# Changing the working Directory
%cd /content/gdrive/My Drive/Kaggle
/content/gdrive/My Drive/Kaggle
In [ ]:
!ls
consolidated kaggle.json test train valid

Since I already downloaded the dataset and unzip the same. So I'm not unzipping again and again.

In [ ]:
!kaggle datasets download -d gpiosenka/100-bird-species
In [ ]:
#unzipping the zip files and deleting the zip files
!unzip \*.zip  && rm *.zip
In [ ]:
!rm *.json

Importing some required libraries.

In [ ]:
import torch
import torchvision
import tarfile
import torch.nn as nn
import numpy as np
import torch.nn.functional as F
from torchvision.datasets.utils import download_url
from torchvision.datasets import ImageFolder
from torch.utils.data import DataLoader
import torchvision.transforms as tt
from torch.utils.data import random_split
from torchvision.utils import make_grid
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

matplotlib.rcParams['figure.facecolor'] = '#ffffff'
In [ ]:
from PIL import Image
from pathlib import Path
import pandas as pd
import math
import cv2
from scipy import signal
In [ ]:
import keras
from keras.models import Sequential
from keras.layers import Dense,Conv2D,MaxPool2D,Dropout,BatchNormalization,Flatten,Activation
from keras.preprocessing import image 
from keras.preprocessing.image import ImageDataGenerator
import matplotlib.pyplot as plt
from keras.utils import plot_model
import seaborn as sns
In [115]:
project_name= 'DL-Project-Assignment'

Preparing the Dataset

Data set of 250 bird species. 35215 training images, 1250 test images(5 X 250 species) and 1250 validation images(5 X 250 species). All images ahaving 224 X 224 X 3 dimension which shows that it is color images with jpg format. There is a "consolidated" image set that groups the training, test and validation images into a single data set.

In [ ]:
print(os.listdir('.'))
['kaggle.json', 'consolidated', 'test', 'train', 'valid']
In [ ]:
# Look into the 250 Birds Species data directory
data_dir = './'
print(os.listdir(data_dir))
['kaggle.json', 'consolidated', 'test', 'train', 'valid']

Let's check inside some of the folders, from the training set and another from the test set. For analytic exploration, let verify the number of images for each class in the training set as well as in the test set.

In [ ]:
classes = os.listdir(data_dir + "/train")
print(len(classes))
print(classes)
250 ['AFRICAN CROWNED CRANE', 'AFRICAN FIREFINCH', 'ALBATROSS', 'ALEXANDRINE PARAKEET', 'AMERICAN AVOCET', 'AMERICAN BITTERN', 'AMERICAN COOT', 'AMERICAN GOLDFINCH', 'AMERICAN KESTREL', 'AMERICAN PIPIT', 'AMERICAN REDSTART', 'ANHINGA', 'ANNAS HUMMINGBIRD', 'ANTBIRD', 'ARARIPE MANAKIN', 'ASIAN CRESTED IBIS', 'BALD EAGLE', 'BALI STARLING', 'BALTIMORE ORIOLE', 'BANANAQUIT', 'BANDED BROADBILL', 'BAR-TAILED GODWIT', 'BARN OWL', 'BARN SWALLOW', 'BARRED PUFFBIRD', 'BAY-BREASTED WARBLER', 'BEARDED BARBET', 'BELTED KINGFISHER', 'BIRD OF PARADISE', 'BLACK FRANCOLIN', 'BLACK SKIMMER', 'BLACK SWAN', 'BLACK THROATED WARBLER', 'BLACK VULTURE', 'BLACK-CAPPED CHICKADEE', 'BLACK-NECKED GREBE', 'BLACK-THROATED SPARROW', 'BLACKBURNIAM WARBLER', 'BLUE GROUSE', 'BLUE HERON', 'BOBOLINK', 'BROWN NOODY', 'BROWN THRASHER', 'CACTUS WREN', 'CALIFORNIA CONDOR', 'CALIFORNIA GULL', 'CALIFORNIA QUAIL', 'CANARY', 'CAPE MAY WARBLER', 'CAPUCHINBIRD', 'CARMINE BEE-EATER', 'CASPIAN TERN', 'CASSOWARY', 'CHARA DE COLLAR', 'CHIPPING SPARROW', 'CHUKAR PARTRIDGE', 'CINNAMON TEAL', 'COCK OF THE ROCK', 'COCKATOO', 'COMMON FIRECREST', 'COMMON GRACKLE', 'COMMON HOUSE MARTIN', 'COMMON LOON', 'COMMON POORWILL', 'COMMON STARLING', 'COUCHS KINGBIRD', 'CRESTED AUKLET', 'CRESTED CARACARA', 'CRESTED NUTHATCH', 'CROW', 'CROWNED PIGEON', 'CUBAN TODY', 'CURL CRESTED ARACURI', 'D-ARNAUDS BARBET', 'DARK EYED JUNCO', 'DOWNY WOODPECKER', 'EASTERN BLUEBIRD', 'EASTERN MEADOWLARK', 'EASTERN ROSELLA', 'EASTERN TOWEE', 'ELEGANT TROGON', 'ELLIOTS PHEASANT', 'EMPEROR PENGUIN', 'EMU', 'EURASIAN GOLDEN ORIOLE', 'EURASIAN MAGPIE', 'EVENING GROSBEAK', 'FIRE TAILLED MYZORNIS', 'FLAME TANAGER', 'FLAMINGO', 'FRIGATE', 'GAMBELS QUAIL', 'GILA WOODPECKER', 'GILDED FLICKER', 'GLOSSY IBIS', 'GO AWAY BIRD', 'GOLD WING WARBLER', 'GOLDEN CHEEKED WARBLER', 'GOLDEN CHLOROPHONIA', 'GOLDEN EAGLE', 'GOLDEN PHEASANT', 'GOLDEN PIPIT', 'GOULDIAN FINCH', 'GRAY CATBIRD', 'GRAY PARTRIDGE', 'GREAT POTOO', 'GREATOR SAGE GROUSE', 'GREEN JAY', 'GREY PLOVER', 'GUINEA TURACO', 'GUINEAFOWL', 'GYRFALCON', 'HARPY EAGLE', 'HAWAIIAN GOOSE', 'HELMET VANGA', 'HOATZIN', 'HOODED MERGANSER', 'HOOPOES', 'HORNBILL', 'HORNED GUAN', 'HORNED SUNGEM', 'HOUSE FINCH', 'HOUSE SPARROW', 'IMPERIAL SHAQ', 'INCA TERN', 'INDIAN BUSTARD', 'INDIAN PITTA', 'INDIGO BUNTING', 'JABIRU', 'JAVA SPARROW', 'JAVAN MAGPIE', 'KAKAPO', 'KILLDEAR', 'KING VULTURE', 'KIWI', 'KOOKABURRA', 'LARK BUNTING', 'LEARS MACAW', 'LILAC ROLLER', 'LONG-EARED OWL', 'MALABAR HORNBILL', 'MALACHITE KINGFISHER', 'MALEO', 'MALLARD DUCK', 'MANDRIN DUCK', 'MARABOU STORK', 'MASKED BOOBY', 'MASKED LAPWING', 'MIKADO PHEASANT', 'MOURNING DOVE', 'MYNA', 'NICOBAR PIGEON', 'NORTHERN BALD IBIS', 'NORTHERN CARDINAL', 'NORTHERN FLICKER', 'NORTHERN GANNET', 'NORTHERN GOSHAWK', 'NORTHERN JACANA', 'NORTHERN MOCKINGBIRD', 'NORTHERN PARULA', 'NORTHERN RED BISHOP', 'OCELLATED TURKEY', 'OKINAWA RAIL', 'OSPREY', 'OSTRICH', 'OYSTER CATCHER', 'PAINTED BUNTIG', 'PALILA', 'PARADISE TANAGER', 'PARUS MAJOR', 'PEACOCK', 'PELICAN', 'PEREGRINE FALCON', 'PHILIPPINE EAGLE', 'PINK ROBIN', 'PUFFIN', 'PURPLE FINCH', 'PURPLE GALLINULE', 'PURPLE MARTIN', 'PURPLE SWAMPHEN', 'QUETZAL', 'RAINBOW LORIKEET', 'RAZORBILL', 'RED BEARDED BEE EATER', 'RED BELLIED PITTA', 'RED FACED CORMORANT', 'RED FACED WARBLER', 'RED HEADED DUCK', 'RED HEADED WOODPECKER', 'RED HONEY CREEPER', 'RED WINGED BLACKBIRD', 'RED WISKERED BULBUL', 'RING-NECKED PHEASANT', 'ROADRUNNER', 'ROBIN', 'ROCK DOVE', 'ROSY FACED LOVEBIRD', 'ROUGH LEG BUZZARD', 'RUBY THROATED HUMMINGBIRD', 'RUFOUS KINGFISHER', 'RUFUOS MOTMOT', 'SAMATRAN THRUSH', 'SAND MARTIN', 'SCARLET IBIS', 'SCARLET MACAW', 'SHOEBILL', 'SHORT BILLED DOWITCHER', 'SMITHS LONGSPUR', 'SNOWY EGRET', 'SNOWY OWL', 'SORA', 'SPANGLED COTINGA', 'SPLENDID WREN', 'SPOON BILED SANDPIPER', 'SPOONBILL', 'SRI LANKA BLUE MAGPIE', 'STEAMER DUCK', 'STORK BILLED KINGFISHER', 'STRAWBERRY FINCH', 'STRIPPED SWALLOW', 'SUPERB STARLING', 'SWINHOES PHEASANT', 'TAIWAN MAGPIE', 'TAKAHE', 'TASMANIAN HEN', 'TEAL DUCK', 'TIT MOUSE', 'TOUCHAN', 'TOWNSENDS WARBLER', 'TREE SWALLOW', 'TRUMPTER SWAN', 'TURKEY VULTURE', 'TURQUOISE MOTMOT', 'UMBRELLA BIRD', 'VARIED THRUSH', 'VENEZUELIAN TROUPIAL', 'VERMILION FLYCATHER', 'VIOLET GREEN SWALLOW', 'VULTURINE GUINEAFOWL', 'WATTLED CURASSOW', 'WHIMBREL', 'WHITE CHEEKED TURACO', 'WHITE NECKED RAVEN', 'WHITE TAILED TROPIC', 'WILD TURKEY', 'WILSONS BIRD OF PARADISE', 'WOOD DUCK', 'YELLOW BELLIED FLOWERPECKER', 'YELLOW CACIQUE', 'YELLOW HEADED BLACKBIRD']
In [ ]:
HORNBILL_files = os.listdir(data_dir + "/train/HORNBILL")
print('No. of training examples for HORNBILL:', len(HORNBILL_files))
print(HORNBILL_files[:10])
No. of training examples for HORNBILL: 122 ['025.jpg', '075.jpg', '079.jpg', '028.jpg', '097.jpg', '053.jpg', '076.jpg', '047.jpg', '103.jpg', '096.jpg']
In [ ]:
cls_test = os.listdir(data_dir + "/test")
print(len(cls_test))
print(cls_test)
250 ['AFRICAN CROWNED CRANE', 'AFRICAN FIREFINCH', 'ALBATROSS', 'ALEXANDRINE PARAKEET', 'AMERICAN AVOCET', 'AMERICAN BITTERN', 'AMERICAN COOT', 'AMERICAN GOLDFINCH', 'AMERICAN KESTREL', 'AMERICAN PIPIT', 'AMERICAN REDSTART', 'ANHINGA', 'ANNAS HUMMINGBIRD', 'ANTBIRD', 'ARARIPE MANAKIN', 'ASIAN CRESTED IBIS', 'BALD EAGLE', 'BALI STARLING', 'BALTIMORE ORIOLE', 'BANANAQUIT', 'BANDED BROADBILL', 'BAR-TAILED GODWIT', 'BARN OWL', 'BARN SWALLOW', 'BARRED PUFFBIRD', 'BAY-BREASTED WARBLER', 'BEARDED BARBET', 'BELTED KINGFISHER', 'BIRD OF PARADISE', 'BLACK FRANCOLIN', 'BLACK SKIMMER', 'BLACK SWAN', 'BLACK THROATED WARBLER', 'BLACK VULTURE', 'BLACK-CAPPED CHICKADEE', 'BLACK-NECKED GREBE', 'BLACK-THROATED SPARROW', 'BLACKBURNIAM WARBLER', 'BLUE GROUSE', 'BLUE HERON', 'BOBOLINK', 'BROWN NOODY', 'BROWN THRASHER', 'CACTUS WREN', 'CALIFORNIA CONDOR', 'CALIFORNIA GULL', 'CALIFORNIA QUAIL', 'CANARY', 'CAPE MAY WARBLER', 'CAPUCHINBIRD', 'CARMINE BEE-EATER', 'CASPIAN TERN', 'CASSOWARY', 'CHARA DE COLLAR', 'CHIPPING SPARROW', 'CHUKAR PARTRIDGE', 'CINNAMON TEAL', 'COCK OF THE ROCK', 'COCKATOO', 'COMMON FIRECREST', 'COMMON GRACKLE', 'COMMON HOUSE MARTIN', 'COMMON LOON', 'COMMON POORWILL', 'COMMON STARLING', 'COUCHS KINGBIRD', 'CRESTED AUKLET', 'CRESTED CARACARA', 'CRESTED NUTHATCH', 'CROW', 'CROWNED PIGEON', 'CUBAN TODY', 'CURL CRESTED ARACURI', 'D-ARNAUDS BARBET', 'DARK EYED JUNCO', 'DOWNY WOODPECKER', 'EASTERN BLUEBIRD', 'EASTERN MEADOWLARK', 'EASTERN ROSELLA', 'EASTERN TOWEE', 'ELEGANT TROGON', 'ELLIOTS PHEASANT', 'EMPEROR PENGUIN', 'EMU', 'EURASIAN GOLDEN ORIOLE', 'EURASIAN MAGPIE', 'EVENING GROSBEAK', 'FIRE TAILLED MYZORNIS', 'FLAME TANAGER', 'FLAMINGO', 'FRIGATE', 'GAMBELS QUAIL', 'GILA WOODPECKER', 'GILDED FLICKER', 'GLOSSY IBIS', 'GO AWAY BIRD', 'GOLD WING WARBLER', 'GOLDEN CHEEKED WARBLER', 'GOLDEN CHLOROPHONIA', 'GOLDEN EAGLE', 'GOLDEN PHEASANT', 'GOLDEN PIPIT', 'GOULDIAN FINCH', 'GRAY CATBIRD', 'GRAY PARTRIDGE', 'GREAT POTOO', 'GREATOR SAGE GROUSE', 'GREEN JAY', 'GREY PLOVER', 'GUINEA TURACO', 'GUINEAFOWL', 'GYRFALCON', 'HARPY EAGLE', 'HAWAIIAN GOOSE', 'HELMET VANGA', 'HOATZIN', 'HOODED MERGANSER', 'HOOPOES', 'HORNBILL', 'HORNED GUAN', 'HORNED SUNGEM', 'HOUSE FINCH', 'HOUSE SPARROW', 'IMPERIAL SHAQ', 'INCA TERN', 'INDIAN BUSTARD', 'INDIAN PITTA', 'INDIGO BUNTING', 'JABIRU', 'JAVA SPARROW', 'JAVAN MAGPIE', 'KAKAPO', 'KILLDEAR', 'KING VULTURE', 'KIWI', 'KOOKABURRA', 'LARK BUNTING', 'LEARS MACAW', 'LILAC ROLLER', 'LONG-EARED OWL', 'MALABAR HORNBILL', 'MALACHITE KINGFISHER', 'MALEO', 'MALLARD DUCK', 'MANDRIN DUCK', 'MARABOU STORK', 'MASKED BOOBY', 'MASKED LAPWING', 'MIKADO PHEASANT', 'MOURNING DOVE', 'MYNA', 'NICOBAR PIGEON', 'NORTHERN BALD IBIS', 'NORTHERN CARDINAL', 'NORTHERN FLICKER', 'NORTHERN GANNET', 'NORTHERN GOSHAWK', 'NORTHERN JACANA', 'NORTHERN MOCKINGBIRD', 'NORTHERN PARULA', 'NORTHERN RED BISHOP', 'OCELLATED TURKEY', 'OKINAWA RAIL', 'OSPREY', 'OSTRICH', 'OYSTER CATCHER', 'PAINTED BUNTIG', 'PALILA', 'PARADISE TANAGER', 'PARUS MAJOR', 'PEACOCK', 'PELICAN', 'PEREGRINE FALCON', 'PHILIPPINE EAGLE', 'PINK ROBIN', 'PUFFIN', 'PURPLE FINCH', 'PURPLE GALLINULE', 'PURPLE MARTIN', 'PURPLE SWAMPHEN', 'QUETZAL', 'RAINBOW LORIKEET', 'RAZORBILL', 'RED BEARDED BEE EATER', 'RED BELLIED PITTA', 'RED FACED CORMORANT', 'RED FACED WARBLER', 'RED HEADED DUCK', 'RED HEADED WOODPECKER', 'RED HONEY CREEPER', 'RED WINGED BLACKBIRD', 'RED WISKERED BULBUL', 'RING-NECKED PHEASANT', 'ROADRUNNER', 'ROBIN', 'ROCK DOVE', 'ROSY FACED LOVEBIRD', 'ROUGH LEG BUZZARD', 'RUBY THROATED HUMMINGBIRD', 'RUFOUS KINGFISHER', 'RUFUOS MOTMOT', 'SAMATRAN THRUSH', 'SAND MARTIN', 'SCARLET IBIS', 'SCARLET MACAW', 'SHOEBILL', 'SHORT BILLED DOWITCHER', 'SMITHS LONGSPUR', 'SNOWY EGRET', 'SNOWY OWL', 'SORA', 'SPANGLED COTINGA', 'SPLENDID WREN', 'SPOON BILED SANDPIPER', 'SPOONBILL', 'SRI LANKA BLUE MAGPIE', 'STEAMER DUCK', 'STORK BILLED KINGFISHER', 'STRAWBERRY FINCH', 'STRIPPED SWALLOW', 'SUPERB STARLING', 'SWINHOES PHEASANT', 'TAIWAN MAGPIE', 'TAKAHE', 'TASMANIAN HEN', 'TEAL DUCK', 'TIT MOUSE', 'TOUCHAN', 'TOWNSENDS WARBLER', 'TREE SWALLOW', 'TRUMPTER SWAN', 'TURKEY VULTURE', 'TURQUOISE MOTMOT', 'UMBRELLA BIRD', 'VARIED THRUSH', 'VENEZUELIAN TROUPIAL', 'VERMILION FLYCATHER', 'VIOLET GREEN SWALLOW', 'VULTURINE GUINEAFOWL', 'WATTLED CURASSOW', 'WHIMBREL', 'WHITE CHEEKED TURACO', 'WHITE NECKED RAVEN', 'WHITE TAILED TROPIC', 'WILD TURKEY', 'WILSONS BIRD OF PARADISE', 'WOOD DUCK', 'YELLOW BELLIED FLOWERPECKER', 'YELLOW CACIQUE', 'YELLOW HEADED BLACKBIRD']
In [ ]:
CROW_test_files = os.listdir(data_dir + "/test/CROW")
print("No. of test examples for CROW:", len(CROW_test_files))
print(CROW_test_files[:5])
No. of test examples for CROW: 5 ['2.jpg', '1.jpg', '4.jpg', '3.jpg', '5.jpg']
In [ ]:
# os.walk('/Kaggle/input'):
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

Data Exploration

Let Find the number of Images for each class

In [ ]:
for i in classes:
  train_images = os.listdir(data_dir + '/train/' + i)
  img_count = len(train_images)
  print("Number of images: {} belongs to the class: {}".format(img_count, i))
Number of images: 137 belongs to the class AFRICAN CROWNED CRANE Number of images: 140 belongs to the class AFRICAN FIREFINCH Number of images: 133 belongs to the class ALBATROSS Number of images: 165 belongs to the class ALEXANDRINE PARAKEET Number of images: 179 belongs to the class AMERICAN AVOCET Number of images: 170 belongs to the class AMERICAN BITTERN Number of images: 158 belongs to the class AMERICAN COOT Number of images: 133 belongs to the class AMERICAN GOLDFINCH Number of images: 130 belongs to the class AMERICAN KESTREL Number of images: 179 belongs to the class AMERICAN PIPIT Number of images: 139 belongs to the class AMERICAN REDSTART Number of images: 147 belongs to the class ANHINGA Number of images: 139 belongs to the class ANNAS HUMMINGBIRD Number of images: 150 belongs to the class ANTBIRD Number of images: 105 belongs to the class ARARIPE MANAKIN Number of images: 105 belongs to the class ASIAN CRESTED IBIS Number of images: 160 belongs to the class BALD EAGLE Number of images: 132 belongs to the class BALI STARLING Number of images: 137 belongs to the class BALTIMORE ORIOLE Number of images: 106 belongs to the class BANANAQUIT Number of images: 194 belongs to the class BANDED BROADBILL Number of images: 114 belongs to the class BAR-TAILED GODWIT Number of images: 119 belongs to the class BARN OWL Number of images: 132 belongs to the class BARN SWALLOW Number of images: 136 belongs to the class BARRED PUFFBIRD Number of images: 143 belongs to the class BAY-BREASTED WARBLER Number of images: 160 belongs to the class BEARDED BARBET Number of images: 125 belongs to the class BELTED KINGFISHER Number of images: 104 belongs to the class BIRD OF PARADISE Number of images: 131 belongs to the class BLACK FRANCOLIN Number of images: 111 belongs to the class BLACK SKIMMER Number of images: 112 belongs to the class BLACK SWAN Number of images: 135 belongs to the class BLACK THROATED WARBLER Number of images: 126 belongs to the class BLACK VULTURE Number of images: 133 belongs to the class BLACK-CAPPED CHICKADEE Number of images: 105 belongs to the class BLACK-NECKED GREBE Number of images: 168 belongs to the class BLACK-THROATED SPARROW Number of images: 134 belongs to the class BLACKBURNIAM WARBLER Number of images: 185 belongs to the class BLUE GROUSE Number of images: 104 belongs to the class BLUE HERON Number of images: 157 belongs to the class BOBOLINK Number of images: 129 belongs to the class BROWN NOODY Number of images: 99 belongs to the class BROWN THRASHER Number of images: 122 belongs to the class CACTUS WREN Number of images: 153 belongs to the class CALIFORNIA CONDOR Number of images: 109 belongs to the class CALIFORNIA GULL Number of images: 115 belongs to the class CALIFORNIA QUAIL Number of images: 160 belongs to the class CANARY Number of images: 145 belongs to the class CAPE MAY WARBLER Number of images: 133 belongs to the class CAPUCHINBIRD Number of images: 121 belongs to the class CARMINE BEE-EATER Number of images: 213 belongs to the class CASPIAN TERN Number of images: 114 belongs to the class CASSOWARY Number of images: 104 belongs to the class CHARA DE COLLAR Number of images: 115 belongs to the class CHIPPING SPARROW Number of images: 168 belongs to the class CHUKAR PARTRIDGE Number of images: 117 belongs to the class CINNAMON TEAL Number of images: 124 belongs to the class COCK OF THE ROCK Number of images: 166 belongs to the class COCKATOO Number of images: 139 belongs to the class COMMON FIRECREST Number of images: 177 belongs to the class COMMON GRACKLE Number of images: 127 belongs to the class COMMON HOUSE MARTIN Number of images: 109 belongs to the class COMMON LOON Number of images: 161 belongs to the class COMMON POORWILL Number of images: 141 belongs to the class COMMON STARLING Number of images: 140 belongs to the class COUCHS KINGBIRD Number of images: 106 belongs to the class CRESTED AUKLET Number of images: 146 belongs to the class CRESTED CARACARA Number of images: 163 belongs to the class CRESTED NUTHATCH Number of images: 107 belongs to the class CROW Number of images: 118 belongs to the class CROWNED PIGEON Number of images: 122 belongs to the class CUBAN TODY Number of images: 137 belongs to the class CURL CRESTED ARACURI Number of images: 233 belongs to the class D-ARNAUDS BARBET Number of images: 203 belongs to the class DARK EYED JUNCO Number of images: 127 belongs to the class DOWNY WOODPECKER Number of images: 128 belongs to the class EASTERN BLUEBIRD Number of images: 190 belongs to the class EASTERN MEADOWLARK Number of images: 118 belongs to the class EASTERN ROSELLA Number of images: 127 belongs to the class EASTERN TOWEE Number of images: 144 belongs to the class ELEGANT TROGON Number of images: 148 belongs to the class ELLIOTS PHEASANT Number of images: 129 belongs to the class EMPEROR PENGUIN Number of images: 106 belongs to the class EMU Number of images: 135 belongs to the class EURASIAN GOLDEN ORIOLE Number of images: 155 belongs to the class EURASIAN MAGPIE Number of images: 144 belongs to the class EVENING GROSBEAK Number of images: 150 belongs to the class FIRE TAILLED MYZORNIS Number of images: 177 belongs to the class FLAME TANAGER Number of images: 122 belongs to the class FLAMINGO Number of images: 105 belongs to the class FRIGATE Number of images: 147 belongs to the class GAMBELS QUAIL Number of images: 146 belongs to the class GILA WOODPECKER Number of images: 138 belongs to the class GILDED FLICKER Number of images: 175 belongs to the class GLOSSY IBIS Number of images: 131 belongs to the class GO AWAY BIRD Number of images: 128 belongs to the class GOLD WING WARBLER Number of images: 176 belongs to the class GOLDEN CHEEKED WARBLER Number of images: 135 belongs to the class GOLDEN CHLOROPHONIA Number of images: 123 belongs to the class GOLDEN EAGLE Number of images: 107 belongs to the class GOLDEN PHEASANT Number of images: 113 belongs to the class GOLDEN PIPIT Number of images: 130 belongs to the class GOULDIAN FINCH Number of images: 155 belongs to the class GRAY CATBIRD Number of images: 103 belongs to the class GRAY PARTRIDGE Number of images: 138 belongs to the class GREAT POTOO Number of images: 184 belongs to the class GREATOR SAGE GROUSE Number of images: 156 belongs to the class GREEN JAY Number of images: 120 belongs to the class GREY PLOVER Number of images: 162 belongs to the class GUINEA TURACO Number of images: 104 belongs to the class GUINEAFOWL Number of images: 124 belongs to the class GYRFALCON Number of images: 175 belongs to the class HARPY EAGLE Number of images: 113 belongs to the class HAWAIIAN GOOSE Number of images: 107 belongs to the class HELMET VANGA Number of images: 155 belongs to the class HOATZIN Number of images: 135 belongs to the class HOODED MERGANSER Number of images: 125 belongs to the class HOOPOES Number of images: 122 belongs to the class HORNBILL Number of images: 113 belongs to the class HORNED GUAN Number of images: 126 belongs to the class HORNED SUNGEM Number of images: 249 belongs to the class HOUSE FINCH Number of images: 125 belongs to the class HOUSE SPARROW Number of images: 144 belongs to the class IMPERIAL SHAQ Number of images: 118 belongs to the class INCA TERN Number of images: 131 belongs to the class INDIAN BUSTARD Number of images: 186 belongs to the class INDIAN PITTA Number of images: 147 belongs to the class INDIGO BUNTING Number of images: 143 belongs to the class JABIRU Number of images: 122 belongs to the class JAVA SPARROW Number of images: 109 belongs to the class JAVAN MAGPIE Number of images: 130 belongs to the class KAKAPO Number of images: 175 belongs to the class KILLDEAR Number of images: 136 belongs to the class KING VULTURE Number of images: 138 belongs to the class KIWI Number of images: 143 belongs to the class KOOKABURRA Number of images: 117 belongs to the class LARK BUNTING Number of images: 131 belongs to the class LEARS MACAW Number of images: 138 belongs to the class LILAC ROLLER Number of images: 106 belongs to the class LONG-EARED OWL Number of images: 130 belongs to the class MALABAR HORNBILL Number of images: 163 belongs to the class MALACHITE KINGFISHER Number of images: 120 belongs to the class MALEO Number of images: 135 belongs to the class MALLARD DUCK Number of images: 130 belongs to the class MANDRIN DUCK Number of images: 197 belongs to the class MARABOU STORK Number of images: 132 belongs to the class MASKED BOOBY Number of images: 131 belongs to the class MASKED LAPWING Number of images: 146 belongs to the class MIKADO PHEASANT Number of images: 126 belongs to the class MOURNING DOVE Number of images: 141 belongs to the class MYNA Number of images: 129 belongs to the class NICOBAR PIGEON Number of images: 128 belongs to the class NORTHERN BALD IBIS Number of images: 130 belongs to the class NORTHERN CARDINAL Number of images: 139 belongs to the class NORTHERN FLICKER Number of images: 145 belongs to the class NORTHERN GANNET Number of images: 112 belongs to the class NORTHERN GOSHAWK Number of images: 156 belongs to the class NORTHERN JACANA Number of images: 140 belongs to the class NORTHERN MOCKINGBIRD Number of images: 196 belongs to the class NORTHERN PARULA Number of images: 135 belongs to the class NORTHERN RED BISHOP Number of images: 118 belongs to the class OCELLATED TURKEY Number of images: 107 belongs to the class OKINAWA RAIL Number of images: 127 belongs to the class OSPREY Number of images: 123 belongs to the class OSTRICH Number of images: 207 belongs to the class OYSTER CATCHER Number of images: 163 belongs to the class PAINTED BUNTIG Number of images: 119 belongs to the class PALILA Number of images: 176 belongs to the class PARADISE TANAGER Number of images: 122 belongs to the class PARUS MAJOR Number of images: 156 belongs to the class PEACOCK Number of images: 118 belongs to the class PELICAN Number of images: 126 belongs to the class PEREGRINE FALCON Number of images: 154 belongs to the class PHILIPPINE EAGLE Number of images: 128 belongs to the class PINK ROBIN Number of images: 124 belongs to the class PUFFIN Number of images: 128 belongs to the class PURPLE FINCH Number of images: 128 belongs to the class PURPLE GALLINULE Number of images: 109 belongs to the class PURPLE MARTIN Number of images: 154 belongs to the class PURPLE SWAMPHEN Number of images: 152 belongs to the class QUETZAL Number of images: 141 belongs to the class RAINBOW LORIKEET Number of images: 194 belongs to the class RAZORBILL Number of images: 197 belongs to the class RED BEARDED BEE EATER Number of images: 151 belongs to the class RED BELLIED PITTA Number of images: 127 belongs to the class RED FACED CORMORANT Number of images: 167 belongs to the class RED FACED WARBLER Number of images: 103 belongs to the class RED HEADED DUCK Number of images: 133 belongs to the class RED HEADED WOODPECKER Number of images: 132 belongs to the class RED HONEY CREEPER Number of images: 127 belongs to the class RED WINGED BLACKBIRD Number of images: 123 belongs to the class RED WISKERED BULBUL Number of images: 97 belongs to the class RING-NECKED PHEASANT Number of images: 107 belongs to the class ROADRUNNER Number of images: 95 belongs to the class ROBIN Number of images: 132 belongs to the class ROCK DOVE Number of images: 139 belongs to the class ROSY FACED LOVEBIRD Number of images: 127 belongs to the class ROUGH LEG BUZZARD Number of images: 135 belongs to the class RUBY THROATED HUMMINGBIRD Number of images: 156 belongs to the class RUFOUS KINGFISHER Number of images: 189 belongs to the class RUFUOS MOTMOT Number of images: 128 belongs to the class SAMATRAN THRUSH Number of images: 95 belongs to the class SAND MARTIN Number of images: 138 belongs to the class SCARLET IBIS Number of images: 105 belongs to the class SCARLET MACAW Number of images: 175 belongs to the class SHOEBILL Number of images: 164 belongs to the class SHORT BILLED DOWITCHER Number of images: 116 belongs to the class SMITHS LONGSPUR Number of images: 132 belongs to the class SNOWY EGRET Number of images: 161 belongs to the class SNOWY OWL Number of images: 300 belongs to the class SORA Number of images: 112 belongs to the class SPANGLED COTINGA Number of images: 121 belongs to the class SPLENDID WREN Number of images: 144 belongs to the class SPOON BILED SANDPIPER Number of images: 192 belongs to the class SPOONBILL Number of images: 161 belongs to the class SRI LANKA BLUE MAGPIE Number of images: 109 belongs to the class STEAMER DUCK Number of images: 135 belongs to the class STORK BILLED KINGFISHER Number of images: 167 belongs to the class STRAWBERRY FINCH Number of images: 120 belongs to the class STRIPPED SWALLOW Number of images: 144 belongs to the class SUPERB STARLING Number of images: 217 belongs to the class SWINHOES PHEASANT Number of images: 136 belongs to the class TAIWAN MAGPIE Number of images: 108 belongs to the class TAKAHE Number of images: 135 belongs to the class TASMANIAN HEN Number of images: 159 belongs to the class TEAL DUCK Number of images: 146 belongs to the class TIT MOUSE Number of images: 136 belongs to the class TOUCHAN Number of images: 165 belongs to the class TOWNSENDS WARBLER Number of images: 181 belongs to the class TREE SWALLOW Number of images: 137 belongs to the class TRUMPTER SWAN Number of images: 149 belongs to the class TURKEY VULTURE Number of images: 156 belongs to the class TURQUOISE MOTMOT Number of images: 144 belongs to the class UMBRELLA BIRD Number of images: 193 belongs to the class VARIED THRUSH Number of images: 127 belongs to the class VENEZUELIAN TROUPIAL Number of images: 155 belongs to the class VERMILION FLYCATHER Number of images: 201 belongs to the class VIOLET GREEN SWALLOW Number of images: 169 belongs to the class VULTURINE GUINEAFOWL Number of images: 138 belongs to the class WATTLED CURASSOW Number of images: 138 belongs to the class WHIMBREL Number of images: 153 belongs to the class WHITE CHEEKED TURACO Number of images: 112 belongs to the class WHITE NECKED RAVEN Number of images: 175 belongs to the class WHITE TAILED TROPIC Number of images: 144 belongs to the class WILD TURKEY Number of images: 126 belongs to the class WILSONS BIRD OF PARADISE Number of images: 214 belongs to the class WOOD DUCK Number of images: 129 belongs to the class YELLOW BELLIED FLOWERPECKER Number of images: 155 belongs to the class YELLOW CACIQUE Number of images: 159 belongs to the class YELLOW HEADED BLACKBIRD
In [ ]:
data_dir  = './'
train_dir = data_dir + '/train'
test_dir  = data_dir + '/test'
val_dir   = data_dir + '/valid'

Image Rescaling

In [ ]:
train_datagen=ImageDataGenerator(rescale=1/255)
val_datagen=ImageDataGenerator(rescale=1/255)
test_datagen=ImageDataGenerator(rescale=1/255)

Image Reading Using the flow from directory function

In [ ]:
train_generator=train_datagen.flow_from_directory(train_dir,target_size=(224,224),color_mode='rgb',batch_size=256,class_mode='sparse')
val_generator=val_datagen.flow_from_directory(val_dir,target_size=(224,224),batch_size=256,color_mode='rgb',class_mode='sparse')
test_generator=test_datagen.flow_from_directory(test_dir,batch_size=256,target_size=(224,224),color_mode='rgb',class_mode='sparse')
Found 35215 images belonging to 250 classes. Found 1250 images belonging to 250 classes. Found 1250 images belonging to 250 classes.

Creating Dictionary for Bird Species Classes

In [ ]:
num_classes=(len(train_generator.class_indices))
print(num_classes)
250
In [ ]:
train_generator.class_indices
Out[]:
{'AFRICAN CROWNED CRANE': 0,
 'AFRICAN FIREFINCH': 1,
 'ALBATROSS': 2,
 'ALEXANDRINE PARAKEET': 3,
 'AMERICAN AVOCET': 4,
 'AMERICAN BITTERN': 5,
 'AMERICAN COOT': 6,
 'AMERICAN GOLDFINCH': 7,
 'AMERICAN KESTREL': 8,
 'AMERICAN PIPIT': 9,
 'AMERICAN REDSTART': 10,
 'ANHINGA': 11,
 'ANNAS HUMMINGBIRD': 12,
 'ANTBIRD': 13,
 'ARARIPE MANAKIN': 14,
 'ASIAN CRESTED IBIS': 15,
 'BALD EAGLE': 16,
 'BALI STARLING': 17,
 'BALTIMORE ORIOLE': 18,
 'BANANAQUIT': 19,
 'BANDED BROADBILL': 20,
 'BAR-TAILED GODWIT': 21,
 'BARN OWL': 22,
 'BARN SWALLOW': 23,
 'BARRED PUFFBIRD': 24,
 'BAY-BREASTED WARBLER': 25,
 'BEARDED BARBET': 26,
 'BELTED KINGFISHER': 27,
 'BIRD OF PARADISE': 28,
 'BLACK FRANCOLIN': 29,
 'BLACK SKIMMER': 30,
 'BLACK SWAN': 31,
 'BLACK THROATED WARBLER': 32,
 'BLACK VULTURE': 33,
 'BLACK-CAPPED CHICKADEE': 34,
 'BLACK-NECKED GREBE': 35,
 'BLACK-THROATED SPARROW': 36,
 'BLACKBURNIAM WARBLER': 37,
 'BLUE GROUSE': 38,
 'BLUE HERON': 39,
 'BOBOLINK': 40,
 'BROWN NOODY': 41,
 'BROWN THRASHER': 42,
 'CACTUS WREN': 43,
 'CALIFORNIA CONDOR': 44,
 'CALIFORNIA GULL': 45,
 'CALIFORNIA QUAIL': 46,
 'CANARY': 47,
 'CAPE MAY WARBLER': 48,
 'CAPUCHINBIRD': 49,
 'CARMINE BEE-EATER': 50,
 'CASPIAN TERN': 51,
 'CASSOWARY': 52,
 'CHARA DE COLLAR': 53,
 'CHIPPING SPARROW': 54,
 'CHUKAR PARTRIDGE': 55,
 'CINNAMON TEAL': 56,
 'COCK OF THE  ROCK': 57,
 'COCKATOO': 58,
 'COMMON FIRECREST': 59,
 'COMMON GRACKLE': 60,
 'COMMON HOUSE MARTIN': 61,
 'COMMON LOON': 62,
 'COMMON POORWILL': 63,
 'COMMON STARLING': 64,
 'COUCHS KINGBIRD': 65,
 'CRESTED AUKLET': 66,
 'CRESTED CARACARA': 67,
 'CRESTED NUTHATCH': 68,
 'CROW': 69,
 'CROWNED PIGEON': 70,
 'CUBAN TODY': 71,
 'CURL CRESTED ARACURI': 72,
 'D-ARNAUDS BARBET': 73,
 'DARK EYED JUNCO': 74,
 'DOWNY WOODPECKER': 75,
 'EASTERN BLUEBIRD': 76,
 'EASTERN MEADOWLARK': 77,
 'EASTERN ROSELLA': 78,
 'EASTERN TOWEE': 79,
 'ELEGANT TROGON': 80,
 'ELLIOTS  PHEASANT': 81,
 'EMPEROR PENGUIN': 82,
 'EMU': 83,
 'EURASIAN GOLDEN ORIOLE': 84,
 'EURASIAN MAGPIE': 85,
 'EVENING GROSBEAK': 86,
 'FIRE TAILLED MYZORNIS': 87,
 'FLAME TANAGER': 88,
 'FLAMINGO': 89,
 'FRIGATE': 90,
 'GAMBELS QUAIL': 91,
 'GILA WOODPECKER': 92,
 'GILDED FLICKER': 93,
 'GLOSSY IBIS': 94,
 'GO AWAY BIRD': 95,
 'GOLD WING WARBLER': 96,
 'GOLDEN CHEEKED WARBLER': 97,
 'GOLDEN CHLOROPHONIA': 98,
 'GOLDEN EAGLE': 99,
 'GOLDEN PHEASANT': 100,
 'GOLDEN PIPIT': 101,
 'GOULDIAN FINCH': 102,
 'GRAY CATBIRD': 103,
 'GRAY PARTRIDGE': 104,
 'GREAT POTOO': 105,
 'GREATOR SAGE GROUSE': 106,
 'GREEN JAY': 107,
 'GREY PLOVER': 108,
 'GUINEA TURACO': 109,
 'GUINEAFOWL': 110,
 'GYRFALCON': 111,
 'HARPY EAGLE': 112,
 'HAWAIIAN GOOSE': 113,
 'HELMET VANGA': 114,
 'HOATZIN': 115,
 'HOODED MERGANSER': 116,
 'HOOPOES': 117,
 'HORNBILL': 118,
 'HORNED GUAN': 119,
 'HORNED SUNGEM': 120,
 'HOUSE FINCH': 121,
 'HOUSE SPARROW': 122,
 'IMPERIAL SHAQ': 123,
 'INCA TERN': 124,
 'INDIAN BUSTARD': 125,
 'INDIAN PITTA': 126,
 'INDIGO BUNTING': 127,
 'JABIRU': 128,
 'JAVA SPARROW': 129,
 'JAVAN MAGPIE': 130,
 'KAKAPO': 131,
 'KILLDEAR': 132,
 'KING VULTURE': 133,
 'KIWI': 134,
 'KOOKABURRA': 135,
 'LARK BUNTING': 136,
 'LEARS MACAW': 137,
 'LILAC ROLLER': 138,
 'LONG-EARED OWL': 139,
 'MALABAR HORNBILL': 140,
 'MALACHITE KINGFISHER': 141,
 'MALEO': 142,
 'MALLARD DUCK': 143,
 'MANDRIN DUCK': 144,
 'MARABOU STORK': 145,
 'MASKED BOOBY': 146,
 'MASKED LAPWING': 147,
 'MIKADO  PHEASANT': 148,
 'MOURNING DOVE': 149,
 'MYNA': 150,
 'NICOBAR PIGEON': 151,
 'NORTHERN BALD IBIS': 152,
 'NORTHERN CARDINAL': 153,
 'NORTHERN FLICKER': 154,
 'NORTHERN GANNET': 155,
 'NORTHERN GOSHAWK': 156,
 'NORTHERN JACANA': 157,
 'NORTHERN MOCKINGBIRD': 158,
 'NORTHERN PARULA': 159,
 'NORTHERN RED BISHOP': 160,
 'OCELLATED TURKEY': 161,
 'OKINAWA RAIL': 162,
 'OSPREY': 163,
 'OSTRICH': 164,
 'OYSTER CATCHER': 165,
 'PAINTED BUNTIG': 166,
 'PALILA': 167,
 'PARADISE TANAGER': 168,
 'PARUS MAJOR': 169,
 'PEACOCK': 170,
 'PELICAN': 171,
 'PEREGRINE FALCON': 172,
 'PHILIPPINE EAGLE': 173,
 'PINK ROBIN': 174,
 'PUFFIN': 175,
 'PURPLE FINCH': 176,
 'PURPLE GALLINULE': 177,
 'PURPLE MARTIN': 178,
 'PURPLE SWAMPHEN': 179,
 'QUETZAL': 180,
 'RAINBOW LORIKEET': 181,
 'RAZORBILL': 182,
 'RED BEARDED BEE EATER': 183,
 'RED BELLIED PITTA': 184,
 'RED FACED CORMORANT': 185,
 'RED FACED WARBLER': 186,
 'RED HEADED DUCK': 187,
 'RED HEADED WOODPECKER': 188,
 'RED HONEY CREEPER': 189,
 'RED WINGED BLACKBIRD': 190,
 'RED WISKERED BULBUL': 191,
 'RING-NECKED PHEASANT': 192,
 'ROADRUNNER': 193,
 'ROBIN': 194,
 'ROCK DOVE': 195,
 'ROSY FACED LOVEBIRD': 196,
 'ROUGH LEG BUZZARD': 197,
 'RUBY THROATED HUMMINGBIRD': 198,
 'RUFOUS KINGFISHER': 199,
 'RUFUOS MOTMOT': 200,
 'SAMATRAN THRUSH': 201,
 'SAND MARTIN': 202,
 'SCARLET IBIS': 203,
 'SCARLET MACAW': 204,
 'SHOEBILL': 205,
 'SHORT BILLED DOWITCHER': 206,
 'SMITHS LONGSPUR': 207,
 'SNOWY EGRET': 208,
 'SNOWY OWL': 209,
 'SORA': 210,
 'SPANGLED COTINGA': 211,
 'SPLENDID WREN': 212,
 'SPOON BILED SANDPIPER': 213,
 'SPOONBILL': 214,
 'SRI LANKA BLUE MAGPIE': 215,
 'STEAMER DUCK': 216,
 'STORK BILLED KINGFISHER': 217,
 'STRAWBERRY FINCH': 218,
 'STRIPPED SWALLOW': 219,
 'SUPERB STARLING': 220,
 'SWINHOES PHEASANT': 221,
 'TAIWAN MAGPIE': 222,
 'TAKAHE': 223,
 'TASMANIAN HEN': 224,
 'TEAL DUCK': 225,
 'TIT MOUSE': 226,
 'TOUCHAN': 227,
 'TOWNSENDS WARBLER': 228,
 'TREE SWALLOW': 229,
 'TRUMPTER SWAN': 230,
 'TURKEY VULTURE': 231,
 'TURQUOISE MOTMOT': 232,
 'UMBRELLA BIRD': 233,
 'VARIED THRUSH': 234,
 'VENEZUELIAN TROUPIAL': 235,
 'VERMILION FLYCATHER': 236,
 'VIOLET GREEN SWALLOW': 237,
 'VULTURINE GUINEAFOWL': 238,
 'WATTLED CURASSOW': 239,
 'WHIMBREL': 240,
 'WHITE CHEEKED TURACO': 241,
 'WHITE NECKED RAVEN': 242,
 'WHITE TAILED TROPIC': 243,
 'WILD TURKEY': 244,
 'WILSONS BIRD OF PARADISE': 245,
 'WOOD DUCK': 246,
 'YELLOW BELLIED FLOWERPECKER': 247,
 'YELLOW CACIQUE': 248,
 'YELLOW HEADED BLACKBIRD': 249}
In [ ]:
breeds=list(train_generator.class_indices.keys())
In [ ]:
x=list(train_generator.classes)
In [ ]:
label=[]
for i in range (0, num_classes):
    label.append(x.count(i))

Data Insight Through Plotting Barplot Graphs

In [ ]:
fig_dims = (20, 6)
fig, ax = plt.subplots(figsize=fig_dims)
sns.barplot(x=breeds[:49],y=label[:49])
plt.xticks(rotation=90)
plt.show()
In [ ]:
fig, ax = plt.subplots(figsize=fig_dims)
sns.barplot(x=breeds[50:99],y=label[50:99])
plt.xticks(rotation=90)
plt.show()
In [ ]:
fig, ax = plt.subplots(figsize=fig_dims)
sns.barplot(x=breeds[100:149],y=label[100:149])
plt.xticks(rotation=90)
plt.show()
In [ ]:
fig, ax = plt.subplots(figsize=fig_dims)
sns.barplot(x=breeds[150:199],y=label[150:199])
plt.xticks(rotation=90)
plt.show()
In [ ]:
fig, ax = plt.subplots(figsize=fig_dims)
sns.barplot(x=breeds[200:],y=label[200:])
plt.xticks(rotation=90)
plt.show()
In [ ]:
image1 = np.array(Image.open('./train/BALD EAGLE/100.jpg'))
plt.imshow(image1, cmap='gray');
In [ ]:
image2 = np.array(Image.open('./train/AFRICAN FIREFINCH/023.jpg'))
plt.imshow(image2, cmap='gray');
In [ ]:
image3 = np.array(Image.open('./train/FLAMINGO/040.jpg'))
plt.imshow(image3, cmap='gray');
In [ ]:
np.array(image1).shape
Out[]:
(224, 224, 3)
In [ ]:
np.array(image2).shape
Out[]:
(224, 224, 3)
In [ ]:
image2_1 = np.array(Image.open('./train/AFRICAN FIREFINCH/023.jpg'))  
image2_1 = cv2.Canny(image2_1,224,224)
plt.imshow(image2_1);

Now create the training and validation datasets using the ImageFolder class of torchvision. For the ToTensor transform, I apply some other transforms to the images. The following changes are few changes which create PyTorch datasets for training and validation:

  • Channel-wise data normalization
  • Randomized data augmentations
In [ ]:
image_size = 32 # 32
In [ ]:
import PIL.Image
In [ ]:
# Data transforms (normalization & data augmentation) # tt.RandomCrop(64, padding=4, padding_mode='reflect')
stats = ((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
randomrotate = 10 # 0.175 rad
train_tfms = tt.Compose([tt.Resize((image_size, image_size)),
                         tt.RandomCrop(image_size, padding=4, padding_mode='reflect'),
                         tt.RandomHorizontalFlip(), 
                         tt.RandomRotation(randomrotate, resample=PIL.Image.NEAREST, expand=False, center=None, fill=None), ##
                         tt.ToTensor(), 
                         tt.Normalize(*stats,inplace=True)])
valid_tfms = tt.Compose([tt.Resize((image_size, image_size)), tt.ToTensor(), tt.Normalize(*stats)])

test_tfms = tt.Compose([tt.Resize((image_size, image_size)), tt.ToTensor(), tt.Normalize(*stats)])
In [ ]:
# PyTorch Training & Validation & Test Datasets
train_ds = ImageFolder('./train', train_tfms)
valid_ds = ImageFolder('./valid', valid_tfms)
test_ds = ImageFolder('./test', test_tfms)
In [ ]:
batch_size = 256 # 256
In [ ]:
# PyTorch Training & Validation & Test Data Loaders
train_dl = DataLoader(train_ds, batch_size, shuffle=True, num_workers=3, pin_memory=True)
valid_dl = DataLoader(valid_ds, batch_size*2, num_workers=3, pin_memory=True)
test_dl = DataLoader(test_ds, batch_size*2, num_workers=3, pin_memory=True)

Let's check some random sample images from the training dataloader. To display the images, I have to denormalize the pixels values to bring them back into the range (0,1).

In [ ]:
def denormalize(images, means, stds):
    means = torch.tensor(means).reshape(1, 3, 1, 1)
    stds = torch.tensor(stds).reshape(1, 3, 1, 1)
    return images * stds + means

def show_batch(dl):
    for images, labels in dl:
        fig, ax = plt.subplots(figsize=(12, 12))
        ax.set_xticks([]); ax.set_yticks([])
        denorm_images = denormalize(images, *stats)
        ax.imshow(make_grid(denorm_images[:64], nrow=8).permute(1, 2, 0).clamp(0,1))
        break
In [ ]:
show_batch(train_dl)