Jovian
⭐️
Sign In

Generative Adverserial Networks

Deep neural networks are used mainly for supervised learning: classification or regression. Generative Adverserial Netoworks or GANs, however, use neural networks for a very different purpose: Generative modeling

Generative modeling is an unsupervised learning task in machine learning that involves automatically discovering and learning the regularities or patterns in input data in such a way that the model can be used to generate or output new examples that plausibly could have been drawn from the original dataset. - Source

While there are many approaches used for generative modeling, a Generative Adverserial Network takes the following approach:

GAN Flowchart

There are two neural networks: a Generator and a Discriminator. The generator generates a "fake" sample given a random vector/matrix, and the discriminator attempts to detect whether a given sample is "real" (picked from the training data) or "fake" (generated by the generator). Training happens in tandem: we train the discriminator for a few epochs, then train the generator for a few epochs, and repeat. This way both the generator and the discriminator get better at doing their jobs. This rather simple approach can lead to some astounding results. The following images (source), for instances, were all generated using GANs:

gans_results

GANs however, can be notoriously difficult to train, and are extremely senstive to hyperparameters, activation functions and regularlization. In this tutorial, we'll train a GAN to generate images of handwritten digits similar to those from the MNIST database.

Most of the code for this tutorial has been borrowed for this excellent repository of PyTorch tutorials: https://github.com/yunjey/pytorch-tutorial . Here's what we're going to do:

  • Define the problem statement
  • Load the data (with transforms and normalization)
    • Denormalize for visual inspection of samples
  • Define the Discriminator network
    • Study the activation function: Leaky ReLU
  • Define the Generator network
    • Explain the output activation function: TanH
    • Look at some sample outputs
  • Define losses, optimizers and helper functions for training
    • For discriminator
    • For generator
  • Train the model
    • Save intermediate generated images to file
  • Look at some outputs
  • Save the models
  • Commit to Jovian.ml

Load the Data

We begin by downloading and importing the data as a PyTorch dataset using the MNIST helper class from torchvision.datasets.

In [14]:
import torch
import torchvision
from torchvision.transforms import ToTensor, Normalize, Compose
from torchvision.datasets import MNIST

mnist = MNIST(root='data', 
              train=True, 
              download=True,
              transform=Compose([ToTensor(), Normalize(mean=(0.5,), std=(0.5,))]))

Note that we are are transforming the pixel values from the range [0, 1] to the range [-1, 1]. The reason for doing this will become clear when define the generator network. Let's look at a sample tensor from the data.

In [15]:

img, label = mnist[0]

In [16]:
img, label = mnist[0]
print('Label: ', label)
print(img[:,10:15,10:15])
torch.min(img), torch.max(img)
Label: 5 tensor([[[-0.9922, 0.2078, 0.9843, -0.2941, -1.0000], [-1.0000, 0.0902, 0.9843, 0.4902, -0.9843], [-1.0000, -0.9137, 0.4902, 0.9843, -0.4510], [-1.0000, -1.0000, -0.7255, 0.8902, 0.7647], [-1.0000, -1.0000, -1.0000, -0.3647, 0.8824]]])
Out[16]:
(tensor(-1.), tensor(1.))

As expected, the pixel values range from -1 to 1. Let's define a helper to denormalize and view the images. This function will also be useful for viewing the generated images.

In [17]:
def denorm(x):
    out = (x + 1) / 2
    return out.clamp(0, 1)
In [18]:
import matplotlib.pyplot as plt
%matplotlib inline

img_norm = denorm(img)
plt.imshow(img_norm[0], cmap='gray')
print('Label:', label)
Label: 5
Notebook Image

Finally, let's create a dataloader to load the images in batches.

In [21]:
from torch.utils.data import DataLoader

batch_size = 100
data_loader = DataLoader(mnist, batch_size, shuffle=True)

We'll also create a device which can be used to move the data and models to a GPU, if one is available.

In [22]:
# Device configuration
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
In [23]:
device
Out[23]:
device(type='cuda')

Discriminator Network

The discriminator takes an image as input, and tries to classify it as "real" or "generated". In this sense, it's like any other neural network. While we can use a CNN for the discriminator, we'll use a simple feedforward network with 3 linear layers to keep things since. We'll treat each 28x28 image as a vector of size 784.

In [27]:
image_size = 784
hidden_size = 256
In [28]:
import torch.nn as nn

D = nn.Sequential(
        nn.Linear(image_size, hidden_size),
        nn.LeakyReLU(0.2),
        nn.Linear(hidden_size, hidden_size),
        nn.LeakyReLU(0.2),
        nn.Linear(hidden_size, 1),
        nn.Sigmoid())

We use the Leaky ReLU activation for the discriminator.

Different from the regular ReLU function, Leaky ReLU allows the pass of a small gradient signal for negative values. As a result, it makes the gradients from the discriminator flows stronger into the generator. Instead of passing a gradient (slope) of 0 in the back-prop pass, it passes a small negative gradient. - Source

Just like any other binary classification model, the output of the discriminator is a single number between 0 and 1, which can be interpreted as the probability of the input image being fake i.e. generated.

Let's move the discriminator model to the chosen device.

In [29]:
D.to(device);

Generator Network

The input to the generator is typically a vector or a matrix which is used as a seed for generating an image. Once again, to keep things simple, we'll use a feedfoward neural network with 3 layers, and the output will be a vector of size 784, which can be transformed to a 28x28 px image.

In [30]:
latent_size = 64
In [31]:
G = nn.Sequential(
    nn.Linear(latent_size, hidden_size),
    nn.ReLU(),
    nn.Linear(hidden_size, hidden_size),
    nn.ReLU(),
    nn.Linear(hidden_size, image_size),
    nn.Tanh())

We use the TanH activation function for the output layer of the generator.

"The ReLU activation (Nair & Hinton, 2010) is used in the generator with the exception of the output layer which uses the Tanh function. We observed that using a bounded activation allowed the model to learn more quickly to saturate and cover the color space of the training distribution. Within the discriminator we found the leaky rectified activation (Maas et al., 2013) (Xu et al., 2015) to work well, especially for higher resolution modeling." - Source

Note that since the outputs of the TanH activation lie in the range [-1,1], we have applied the same transformation to the images in the training dataset. Let's generate an output vector using the generator and view it as an image by transforming and denormalizing the output.

In [32]:
y = G(torch.randn(2, latent_size))
gen_imgs = denorm(y.reshape((-1, 28,28)).detach())
In [33]:
plt.imshow(gen_imgs[0], cmap='gray');
Notebook Image
In [34]:
plt.imshow(gen_imgs[1], cmap='gray');
Notebook Image

As one might expect, the output from the generator is basically random noise. Let's define a helper function which can save a batch of outputs from the generator to a file.

Let's move the generator to the chosen device.

In [35]:
G.to(device);

Discriminator Training

Since the discriminator is a binary classification model, we can use the binary cross entropy loss function to quantify how well it is able to differentiate between real and generated images.

In [36]:
criterion = nn.BCELoss()
d_optimizer = torch.optim.Adam(D.parameters(), lr=0.0002)
g_optimizer = torch.optim.Adam(G.parameters(), lr=0.0002)

Let's define helper functions to reset gradients and train the discriminator.

In [39]:
def reset_grad():
    d_optimizer.zero_grad()
    g_optimizer.zero_grad()

def train_discriminator(images):
    # Create the labels which are later used as input for the BCE loss
    real_labels = torch.ones(batch_size, 1).to(device)
    fake_labels = torch.zeros(batch_size, 1).to(device)
        
    # Loss for real images
    outputs = D(images)
    d_loss_real = criterion(outputs, real_labels)
    real_score = outputs

    # Loss for fake images
    z = torch.randn(batch_size, latent_size).to(device)
    fake_images = G(z)
    outputs = D(fake_images)
    d_loss_fake = criterion(outputs, fake_labels)
    fake_score = outputs

    # Combine losses
    d_loss = d_loss_real + d_loss_fake
    # Reset gradients
    reset_grad()
    # Compute gradients
    d_loss.backward()
    # Adjust the parameters using backprop
    d_optimizer.step()
    
    return d_loss, real_score, fake_score

Generator Training

Since the outputs of the generator are vectors (which can be transformed to images), it's not obvious how we can "train" the generator. This is where we employ a rather elegant "trick". Since we know that the output images are "generated" or "fake", we can pass them into the discriminator, and compare the output of the discriminator with the ground truth (i.e. all fake), and use that to calculate the loss for the generator.

In other words, we the disciminiator as a part of the loss function. Here's what this looks like in code.

In [41]:
def train_generator():
    # Generate fake images and calculate loss
    z = torch.randn(batch_size, latent_size).to(device)
    fake_images = G(z)
    labels = torch.ones(batch_size, 1).to(device)
    g_loss = criterion(D(fake_images), labels)

    # Backprop and optimize
    reset_grad()
    g_loss.backward()
    g_optimizer.step()
    return g_loss, fake_images

Training the Model

Let's create a directory where we can save intermediate outputs from the generator to visually inspect the progress of the model

In [42]:
import os

sample_dir = 'samples'
if not os.path.exists(sample_dir):
    os.makedirs(sample_dir)

We are now ready to train the model. We train the discriminator first, and then the generator. The training might take a while if you're not using a GPU.

In [43]:
num_epochs = 200
In [44]:
from torchvision.utils import save_image

total_step = len(data_loader)
for epoch in range(num_epochs):
    for i, (images, _) in enumerate(data_loader):
        # Load a batch & transform to vectors
        images = images.reshape(batch_size, -1).to(device)
        
        # Train the discriminator and generator
        d_loss, real_score, fake_score = train_discriminator(images)
        g_loss, fake_images = train_generator()
        
        # Inspect the losses
        if (i+1) % 200 == 0:
            print('Epoch [{}/{}], Step [{}/{}], d_loss: {:.4f}, g_loss: {:.4f}, D(x): {:.2f}, D(G(z)): {:.2f}' 
                  .format(epoch, num_epochs, i+1, total_step, d_loss.item(), g_loss.item(), 
                          real_score.mean().item(), fake_score.mean().item()))
    
    # Save real images (just one batch)
    if (epoch+1) == 1:
        images = images.reshape(images.size(0), 1, 28, 28)
        save_image(denorm(images), os.path.join(sample_dir, 'real_images.png'))
    
    # Save sampled images
    fake_images = fake_images.reshape(fake_images.size(0), 1, 28, 28)
    fake_fname = 'fake_images-{}.png'.format(epoch+1)
    print('Saving', fake_fname)
    save_image(denorm(fake_images), os.path.join(sample_dir, fake_fname))
Epoch [0/200], Step [200/600], d_loss: 0.0368, g_loss: 4.1486, D(x): 1.00, D(G(z)): 0.03 Epoch [0/200], Step [400/600], d_loss: 0.0477, g_loss: 5.5623, D(x): 0.98, D(G(z)): 0.02 Epoch [0/200], Step [600/600], d_loss: 0.0619, g_loss: 4.3400, D(x): 0.98, D(G(z)): 0.04 Saving fake_images-1.png Epoch [1/200], Step [200/600], d_loss: 0.1484, g_loss: 4.1908, D(x): 0.96, D(G(z)): 0.10 Epoch [1/200], Step [400/600], d_loss: 0.1914, g_loss: 3.5414, D(x): 0.94, D(G(z)): 0.08 Epoch [1/200], Step [600/600], d_loss: 0.9804, g_loss: 2.4519, D(x): 0.80, D(G(z)): 0.36 Saving fake_images-2.png Epoch [2/200], Step [200/600], d_loss: 0.0842, g_loss: 4.8010, D(x): 0.96, D(G(z)): 0.03 Epoch [2/200], Step [400/600], d_loss: 0.7265, g_loss: 1.9127, D(x): 0.80, D(G(z)): 0.33 Epoch [2/200], Step [600/600], d_loss: 1.5218, g_loss: 2.3879, D(x): 0.68, D(G(z)): 0.46 Saving fake_images-3.png Epoch [3/200], Step [200/600], d_loss: 0.4083, g_loss: 2.3923, D(x): 0.84, D(G(z)): 0.11 Epoch [3/200], Step [400/600], d_loss: 0.5173, g_loss: 3.9420, D(x): 0.82, D(G(z)): 0.10 Epoch [3/200], Step [600/600], d_loss: 0.2957, g_loss: 3.8771, D(x): 0.89, D(G(z)): 0.10 Saving fake_images-4.png Epoch [4/200], Step [200/600], d_loss: 0.1649, g_loss: 3.1161, D(x): 0.95, D(G(z)): 0.09 Epoch [4/200], Step [400/600], d_loss: 0.1935, g_loss: 3.7495, D(x): 0.92, D(G(z)): 0.04 Epoch [4/200], Step [600/600], d_loss: 0.8499, g_loss: 3.0338, D(x): 0.74, D(G(z)): 0.07 Saving fake_images-5.png Epoch [5/200], Step [200/600], d_loss: 0.3927, g_loss: 3.5246, D(x): 0.89, D(G(z)): 0.06 Epoch [5/200], Step [400/600], d_loss: 0.3491, g_loss: 2.7642, D(x): 0.93, D(G(z)): 0.19 Epoch [5/200], Step [600/600], d_loss: 0.4572, g_loss: 4.4920, D(x): 0.83, D(G(z)): 0.04 Saving fake_images-6.png Epoch [6/200], Step [200/600], d_loss: 0.2497, g_loss: 4.6277, D(x): 0.90, D(G(z)): 0.05 Epoch [6/200], Step [400/600], d_loss: 0.1145, g_loss: 3.9734, D(x): 0.95, D(G(z)): 0.04 Epoch [6/200], Step [600/600], d_loss: 0.3478, g_loss: 3.0536, D(x): 0.90, D(G(z)): 0.05 Saving fake_images-7.png Epoch [7/200], Step [200/600], d_loss: 0.3338, g_loss: 3.2273, D(x): 0.90, D(G(z)): 0.04 Epoch [7/200], Step [400/600], d_loss: 0.1056, g_loss: 6.6240, D(x): 0.97, D(G(z)): 0.05 Epoch [7/200], Step [600/600], d_loss: 0.1072, g_loss: 4.9987, D(x): 0.96, D(G(z)): 0.02 Saving fake_images-8.png Epoch [8/200], Step [200/600], d_loss: 0.0887, g_loss: 3.8202, D(x): 0.96, D(G(z)): 0.04 Epoch [8/200], Step [400/600], d_loss: 0.1645, g_loss: 6.1845, D(x): 0.97, D(G(z)): 0.08 Epoch [8/200], Step [600/600], d_loss: 0.0791, g_loss: 5.3206, D(x): 0.98, D(G(z)): 0.05 Saving fake_images-9.png Epoch [9/200], Step [200/600], d_loss: 0.0813, g_loss: 6.0105, D(x): 0.97, D(G(z)): 0.04 Epoch [9/200], Step [400/600], d_loss: 0.1601, g_loss: 4.2983, D(x): 0.94, D(G(z)): 0.03 Epoch [9/200], Step [600/600], d_loss: 0.1463, g_loss: 5.7765, D(x): 0.97, D(G(z)): 0.08 Saving fake_images-10.png Epoch [10/200], Step [200/600], d_loss: 0.2244, g_loss: 4.0149, D(x): 0.92, D(G(z)): 0.02 Epoch [10/200], Step [400/600], d_loss: 0.2485, g_loss: 3.8447, D(x): 0.94, D(G(z)): 0.11 Epoch [10/200], Step [600/600], d_loss: 0.1885, g_loss: 3.8256, D(x): 0.93, D(G(z)): 0.04 Saving fake_images-11.png Epoch [11/200], Step [200/600], d_loss: 0.1955, g_loss: 4.1098, D(x): 0.97, D(G(z)): 0.12 Epoch [11/200], Step [400/600], d_loss: 0.0983, g_loss: 5.1657, D(x): 0.97, D(G(z)): 0.04 Epoch [11/200], Step [600/600], d_loss: 0.1850, g_loss: 4.5331, D(x): 0.95, D(G(z)): 0.05 Saving fake_images-12.png Epoch [12/200], Step [200/600], d_loss: 0.1473, g_loss: 6.8483, D(x): 0.97, D(G(z)): 0.06 Epoch [12/200], Step [400/600], d_loss: 0.2700, g_loss: 5.6541, D(x): 0.98, D(G(z)): 0.15 Epoch [12/200], Step [600/600], d_loss: 0.3607, g_loss: 3.6330, D(x): 0.90, D(G(z)): 0.04 Saving fake_images-13.png Epoch [13/200], Step [200/600], d_loss: 0.1130, g_loss: 3.6693, D(x): 1.00, D(G(z)): 0.10 Epoch [13/200], Step [400/600], d_loss: 0.2824, g_loss: 4.7340, D(x): 0.92, D(G(z)): 0.03 Epoch [13/200], Step [600/600], d_loss: 0.2836, g_loss: 5.1638, D(x): 0.94, D(G(z)): 0.13 Saving fake_images-14.png Epoch [14/200], Step [200/600], d_loss: 0.2925, g_loss: 4.4015, D(x): 0.92, D(G(z)): 0.05 Epoch [14/200], Step [400/600], d_loss: 0.5847, g_loss: 4.2443, D(x): 0.95, D(G(z)): 0.31 Epoch [14/200], Step [600/600], d_loss: 0.1057, g_loss: 3.8513, D(x): 0.98, D(G(z)): 0.06 Saving fake_images-15.png Epoch [15/200], Step [200/600], d_loss: 0.1139, g_loss: 2.8332, D(x): 0.97, D(G(z)): 0.06 Epoch [15/200], Step [400/600], d_loss: 0.1304, g_loss: 4.6939, D(x): 0.97, D(G(z)): 0.05 Epoch [15/200], Step [600/600], d_loss: 0.3364, g_loss: 4.8741, D(x): 0.97, D(G(z)): 0.16 Saving fake_images-16.png Epoch [16/200], Step [200/600], d_loss: 0.1945, g_loss: 6.2433, D(x): 0.95, D(G(z)): 0.08 Epoch [16/200], Step [400/600], d_loss: 0.1614, g_loss: 3.7477, D(x): 0.94, D(G(z)): 0.05 Epoch [16/200], Step [600/600], d_loss: 0.2618, g_loss: 4.8928, D(x): 0.95, D(G(z)): 0.11 Saving fake_images-17.png Epoch [17/200], Step [200/600], d_loss: 0.1477, g_loss: 4.9693, D(x): 0.97, D(G(z)): 0.07 Epoch [17/200], Step [400/600], d_loss: 0.1689, g_loss: 4.8419, D(x): 0.95, D(G(z)): 0.04 Epoch [17/200], Step [600/600], d_loss: 0.3764, g_loss: 5.3964, D(x): 0.91, D(G(z)): 0.06 Saving fake_images-18.png Epoch [18/200], Step [200/600], d_loss: 0.3661, g_loss: 9.2950, D(x): 0.89, D(G(z)): 0.00 Epoch [18/200], Step [400/600], d_loss: 0.2679, g_loss: 6.6296, D(x): 0.96, D(G(z)): 0.12 Epoch [18/200], Step [600/600], d_loss: 0.2333, g_loss: 4.9586, D(x): 0.98, D(G(z)): 0.14 Saving fake_images-19.png Epoch [19/200], Step [200/600], d_loss: 0.3190, g_loss: 6.0207, D(x): 0.88, D(G(z)): 0.01 Epoch [19/200], Step [400/600], d_loss: 0.3333, g_loss: 5.5758, D(x): 0.91, D(G(z)): 0.08 Epoch [19/200], Step [600/600], d_loss: 0.2681, g_loss: 5.1981, D(x): 0.95, D(G(z)): 0.06 Saving fake_images-20.png Epoch [20/200], Step [200/600], d_loss: 0.2246, g_loss: 4.8635, D(x): 0.98, D(G(z)): 0.09 Epoch [20/200], Step [400/600], d_loss: 0.2803, g_loss: 4.8187, D(x): 0.93, D(G(z)): 0.07 Epoch [20/200], Step [600/600], d_loss: 0.2897, g_loss: 6.0313, D(x): 0.89, D(G(z)): 0.01 Saving fake_images-21.png Epoch [21/200], Step [200/600], d_loss: 0.2880, g_loss: 4.4973, D(x): 0.91, D(G(z)): 0.08 Epoch [21/200], Step [400/600], d_loss: 0.2778, g_loss: 4.5586, D(x): 0.93, D(G(z)): 0.10 Epoch [21/200], Step [600/600], d_loss: 0.1451, g_loss: 4.3355, D(x): 0.96, D(G(z)): 0.08 Saving fake_images-22.png Epoch [22/200], Step [200/600], d_loss: 0.1500, g_loss: 4.6824, D(x): 0.96, D(G(z)): 0.06 Epoch [22/200], Step [400/600], d_loss: 0.2202, g_loss: 4.3258, D(x): 0.94, D(G(z)): 0.05 Epoch [22/200], Step [600/600], d_loss: 0.2495, g_loss: 3.3011, D(x): 0.95, D(G(z)): 0.09 Saving fake_images-23.png Epoch [23/200], Step [200/600], d_loss: 0.1420, g_loss: 4.2327, D(x): 0.96, D(G(z)): 0.06 Epoch [23/200], Step [400/600], d_loss: 0.5867, g_loss: 5.2520, D(x): 0.91, D(G(z)): 0.22 Epoch [23/200], Step [600/600], d_loss: 0.4146, g_loss: 4.9831, D(x): 0.85, D(G(z)): 0.02 Saving fake_images-24.png Epoch [24/200], Step [200/600], d_loss: 0.3057, g_loss: 5.4440, D(x): 0.89, D(G(z)): 0.03 Epoch [24/200], Step [400/600], d_loss: 0.3892, g_loss: 3.5115, D(x): 0.91, D(G(z)): 0.12 Epoch [24/200], Step [600/600], d_loss: 0.4054, g_loss: 3.8188, D(x): 0.90, D(G(z)): 0.10 Saving fake_images-25.png Epoch [25/200], Step [200/600], d_loss: 0.2783, g_loss: 7.2347, D(x): 0.98, D(G(z)): 0.13 Epoch [25/200], Step [400/600], d_loss: 0.2135, g_loss: 4.3919, D(x): 0.92, D(G(z)): 0.05 Epoch [25/200], Step [600/600], d_loss: 0.2869, g_loss: 6.1736, D(x): 0.90, D(G(z)): 0.02 Saving fake_images-26.png Epoch [26/200], Step [200/600], d_loss: 0.2039, g_loss: 5.0607, D(x): 0.91, D(G(z)): 0.05 Epoch [26/200], Step [400/600], d_loss: 0.2751, g_loss: 5.5857, D(x): 0.92, D(G(z)): 0.10 Epoch [26/200], Step [600/600], d_loss: 0.4442, g_loss: 4.5887, D(x): 0.86, D(G(z)): 0.08 Saving fake_images-27.png Epoch [27/200], Step [200/600], d_loss: 0.2367, g_loss: 4.0819, D(x): 0.92, D(G(z)): 0.04 Epoch [27/200], Step [400/600], d_loss: 0.1663, g_loss: 4.3976, D(x): 0.94, D(G(z)): 0.06 Epoch [27/200], Step [600/600], d_loss: 0.2705, g_loss: 4.5919, D(x): 0.91, D(G(z)): 0.05 Saving fake_images-28.png
Epoch [28/200], Step [200/600], d_loss: 0.3381, g_loss: 4.9855, D(x): 0.90, D(G(z)): 0.09 Epoch [28/200], Step [400/600], d_loss: 0.4052, g_loss: 4.2258, D(x): 0.90, D(G(z)): 0.09 Epoch [28/200], Step [600/600], d_loss: 0.3389, g_loss: 3.8460, D(x): 0.88, D(G(z)): 0.09 Saving fake_images-29.png Epoch [29/200], Step [200/600], d_loss: 0.2387, g_loss: 5.0074, D(x): 0.96, D(G(z)): 0.13 Epoch [29/200], Step [400/600], d_loss: 0.2280, g_loss: 5.7922, D(x): 0.93, D(G(z)): 0.07 Epoch [29/200], Step [600/600], d_loss: 0.2370, g_loss: 5.5324, D(x): 0.95, D(G(z)): 0.10 Saving fake_images-30.png Epoch [30/200], Step [200/600], d_loss: 0.4260, g_loss: 4.0324, D(x): 0.86, D(G(z)): 0.07 Epoch [30/200], Step [400/600], d_loss: 0.3166, g_loss: 4.8742, D(x): 0.90, D(G(z)): 0.07 Epoch [30/200], Step [600/600], d_loss: 0.4982, g_loss: 4.1840, D(x): 0.90, D(G(z)): 0.18 Saving fake_images-31.png Epoch [31/200], Step [200/600], d_loss: 0.3271, g_loss: 4.0697, D(x): 0.88, D(G(z)): 0.06 Epoch [31/200], Step [400/600], d_loss: 0.4122, g_loss: 4.6621, D(x): 0.92, D(G(z)): 0.15 Epoch [31/200], Step [600/600], d_loss: 0.4538, g_loss: 3.8567, D(x): 0.88, D(G(z)): 0.13 Saving fake_images-32.png Epoch [32/200], Step [200/600], d_loss: 0.4459, g_loss: 3.4706, D(x): 0.84, D(G(z)): 0.08 Epoch [32/200], Step [400/600], d_loss: 0.2866, g_loss: 4.4012, D(x): 0.92, D(G(z)): 0.12 Epoch [32/200], Step [600/600], d_loss: 0.6572, g_loss: 2.6002, D(x): 0.76, D(G(z)): 0.10 Saving fake_images-33.png Epoch [33/200], Step [200/600], d_loss: 0.5328, g_loss: 4.0871, D(x): 0.80, D(G(z)): 0.06 Epoch [33/200], Step [400/600], d_loss: 0.4037, g_loss: 2.5258, D(x): 0.85, D(G(z)): 0.06 Epoch [33/200], Step [600/600], d_loss: 0.3623, g_loss: 3.1938, D(x): 0.94, D(G(z)): 0.17 Saving fake_images-34.png Epoch [34/200], Step [200/600], d_loss: 0.2894, g_loss: 4.0787, D(x): 0.92, D(G(z)): 0.12 Epoch [34/200], Step [400/600], d_loss: 0.2601, g_loss: 3.8992, D(x): 0.92, D(G(z)): 0.10 Epoch [34/200], Step [600/600], d_loss: 0.3017, g_loss: 3.8858, D(x): 0.91, D(G(z)): 0.11 Saving fake_images-35.png Epoch [35/200], Step [200/600], d_loss: 0.3898, g_loss: 2.7057, D(x): 0.88, D(G(z)): 0.12 Epoch [35/200], Step [400/600], d_loss: 0.3050, g_loss: 3.6706, D(x): 0.91, D(G(z)): 0.13 Epoch [35/200], Step [600/600], d_loss: 0.4615, g_loss: 3.9409, D(x): 0.81, D(G(z)): 0.03 Saving fake_images-36.png Epoch [36/200], Step [200/600], d_loss: 0.2534, g_loss: 3.3188, D(x): 0.91, D(G(z)): 0.08 Epoch [36/200], Step [400/600], d_loss: 0.5179, g_loss: 4.1806, D(x): 0.92, D(G(z)): 0.22 Epoch [36/200], Step [600/600], d_loss: 0.4107, g_loss: 2.9260, D(x): 0.86, D(G(z)): 0.11 Saving fake_images-37.png Epoch [37/200], Step [200/600], d_loss: 0.3674, g_loss: 3.3533, D(x): 0.92, D(G(z)): 0.15 Epoch [37/200], Step [400/600], d_loss: 0.5021, g_loss: 2.9050, D(x): 0.85, D(G(z)): 0.15 Epoch [37/200], Step [600/600], d_loss: 0.4176, g_loss: 2.8705, D(x): 0.90, D(G(z)): 0.18 Saving fake_images-38.png Epoch [38/200], Step [200/600], d_loss: 0.4067, g_loss: 4.0045, D(x): 0.85, D(G(z)): 0.09 Epoch [38/200], Step [400/600], d_loss: 0.3564, g_loss: 4.8107, D(x): 0.87, D(G(z)): 0.08 Epoch [38/200], Step [600/600], d_loss: 0.5933, g_loss: 4.8943, D(x): 0.85, D(G(z)): 0.14 Saving fake_images-39.png Epoch [39/200], Step [200/600], d_loss: 0.2764, g_loss: 3.6858, D(x): 0.94, D(G(z)): 0.15 Epoch [39/200], Step [400/600], d_loss: 0.2070, g_loss: 4.2059, D(x): 0.98, D(G(z)): 0.13 Epoch [39/200], Step [600/600], d_loss: 0.4209, g_loss: 3.6347, D(x): 0.96, D(G(z)): 0.21 Saving fake_images-40.png Epoch [40/200], Step [200/600], d_loss: 0.1308, g_loss: 4.7458, D(x): 0.96, D(G(z)): 0.06 Epoch [40/200], Step [400/600], d_loss: 0.3882, g_loss: 2.7140, D(x): 0.92, D(G(z)): 0.17 Epoch [40/200], Step [600/600], d_loss: 0.5257, g_loss: 3.4519, D(x): 0.84, D(G(z)): 0.14 Saving fake_images-41.png Epoch [41/200], Step [200/600], d_loss: 0.3522, g_loss: 3.8787, D(x): 0.88, D(G(z)): 0.09 Epoch [41/200], Step [400/600], d_loss: 0.5247, g_loss: 3.0050, D(x): 0.95, D(G(z)): 0.25 Epoch [41/200], Step [600/600], d_loss: 0.6201, g_loss: 2.6881, D(x): 0.83, D(G(z)): 0.16 Saving fake_images-42.png Epoch [42/200], Step [200/600], d_loss: 0.5625, g_loss: 2.7222, D(x): 0.85, D(G(z)): 0.17 Epoch [42/200], Step [400/600], d_loss: 0.3164, g_loss: 3.6033, D(x): 0.89, D(G(z)): 0.13 Epoch [42/200], Step [600/600], d_loss: 0.7049, g_loss: 2.3558, D(x): 0.85, D(G(z)): 0.25 Saving fake_images-43.png Epoch [43/200], Step [200/600], d_loss: 0.4462, g_loss: 2.5196, D(x): 0.84, D(G(z)): 0.13 Epoch [43/200], Step [400/600], d_loss: 0.3404, g_loss: 4.0889, D(x): 0.88, D(G(z)): 0.12 Epoch [43/200], Step [600/600], d_loss: 0.4430, g_loss: 3.7213, D(x): 0.86, D(G(z)): 0.14 Saving fake_images-44.png Epoch [44/200], Step [200/600], d_loss: 0.6170, g_loss: 2.9866, D(x): 0.83, D(G(z)): 0.17 Epoch [44/200], Step [400/600], d_loss: 0.5682, g_loss: 2.8797, D(x): 0.82, D(G(z)): 0.16 Epoch [44/200], Step [600/600], d_loss: 0.9155, g_loss: 2.0664, D(x): 0.69, D(G(z)): 0.16 Saving fake_images-45.png Epoch [45/200], Step [200/600], d_loss: 0.4976, g_loss: 2.0836, D(x): 0.84, D(G(z)): 0.16 Epoch [45/200], Step [400/600], d_loss: 0.6303, g_loss: 2.5046, D(x): 0.81, D(G(z)): 0.19 Epoch [45/200], Step [600/600], d_loss: 0.4203, g_loss: 2.1281, D(x): 0.85, D(G(z)): 0.14 Saving fake_images-46.png Epoch [46/200], Step [200/600], d_loss: 0.6186, g_loss: 2.4944, D(x): 0.85, D(G(z)): 0.25 Epoch [46/200], Step [400/600], d_loss: 0.5977, g_loss: 2.6280, D(x): 0.78, D(G(z)): 0.17 Epoch [46/200], Step [600/600], d_loss: 0.4290, g_loss: 2.1607, D(x): 0.84, D(G(z)): 0.11 Saving fake_images-47.png Epoch [47/200], Step [200/600], d_loss: 0.6107, g_loss: 3.0314, D(x): 0.84, D(G(z)): 0.22 Epoch [47/200], Step [400/600], d_loss: 0.4796, g_loss: 2.7853, D(x): 0.83, D(G(z)): 0.14 Epoch [47/200], Step [600/600], d_loss: 0.6196, g_loss: 2.3343, D(x): 0.82, D(G(z)): 0.21 Saving fake_images-48.png Epoch [48/200], Step [200/600], d_loss: 0.3461, g_loss: 3.1870, D(x): 0.85, D(G(z)): 0.11 Epoch [48/200], Step [400/600], d_loss: 0.6490, g_loss: 4.1644, D(x): 0.73, D(G(z)): 0.08 Epoch [48/200], Step [600/600], d_loss: 0.4753, g_loss: 3.3790, D(x): 0.81, D(G(z)): 0.09 Saving fake_images-49.png Epoch [49/200], Step [200/600], d_loss: 0.6222, g_loss: 2.4931, D(x): 0.83, D(G(z)): 0.20 Epoch [49/200], Step [400/600], d_loss: 0.5714, g_loss: 2.5149, D(x): 0.85, D(G(z)): 0.20 Epoch [49/200], Step [600/600], d_loss: 0.4748, g_loss: 3.4715, D(x): 0.88, D(G(z)): 0.19 Saving fake_images-50.png Epoch [50/200], Step [200/600], d_loss: 0.3824, g_loss: 3.0530, D(x): 0.89, D(G(z)): 0.14 Epoch [50/200], Step [400/600], d_loss: 0.3882, g_loss: 3.3273, D(x): 0.88, D(G(z)): 0.15 Epoch [50/200], Step [600/600], d_loss: 0.5707, g_loss: 3.2075, D(x): 0.78, D(G(z)): 0.12 Saving fake_images-51.png Epoch [51/200], Step [200/600], d_loss: 0.5137, g_loss: 2.6292, D(x): 0.85, D(G(z)): 0.18 Epoch [51/200], Step [400/600], d_loss: 0.3591, g_loss: 3.2114, D(x): 0.88, D(G(z)): 0.12 Epoch [51/200], Step [600/600], d_loss: 0.4616, g_loss: 3.3745, D(x): 0.89, D(G(z)): 0.20 Saving fake_images-52.png Epoch [52/200], Step [200/600], d_loss: 0.3227, g_loss: 3.3519, D(x): 0.93, D(G(z)): 0.17 Epoch [52/200], Step [400/600], d_loss: 0.4952, g_loss: 3.1056, D(x): 0.78, D(G(z)): 0.07 Epoch [52/200], Step [600/600], d_loss: 0.5004, g_loss: 2.5508, D(x): 0.80, D(G(z)): 0.13 Saving fake_images-53.png Epoch [53/200], Step [200/600], d_loss: 0.5512, g_loss: 2.7263, D(x): 0.86, D(G(z)): 0.22 Epoch [53/200], Step [400/600], d_loss: 0.5572, g_loss: 2.4056, D(x): 0.77, D(G(z)): 0.11 Epoch [53/200], Step [600/600], d_loss: 0.7323, g_loss: 2.3611, D(x): 0.76, D(G(z)): 0.22 Saving fake_images-54.png Epoch [54/200], Step [200/600], d_loss: 0.5656, g_loss: 2.8647, D(x): 0.82, D(G(z)): 0.17 Epoch [54/200], Step [400/600], d_loss: 0.5531, g_loss: 1.9734, D(x): 0.87, D(G(z)): 0.24 Epoch [54/200], Step [600/600], d_loss: 0.5611, g_loss: 2.3445, D(x): 0.82, D(G(z)): 0.16 Saving fake_images-55.png Epoch [55/200], Step [200/600], d_loss: 0.5199, g_loss: 3.0128, D(x): 0.84, D(G(z)): 0.18 Epoch [55/200], Step [400/600], d_loss: 0.7905, g_loss: 1.8519, D(x): 0.75, D(G(z)): 0.19 Epoch [55/200], Step [600/600], d_loss: 0.6144, g_loss: 1.7771, D(x): 0.86, D(G(z)): 0.25 Saving fake_images-56.png
Epoch [56/200], Step [200/600], d_loss: 0.5307, g_loss: 2.3492, D(x): 0.86, D(G(z)): 0.23 Epoch [56/200], Step [400/600], d_loss: 0.7315, g_loss: 2.1073, D(x): 0.76, D(G(z)): 0.19 Epoch [56/200], Step [600/600], d_loss: 0.7797, g_loss: 2.4241, D(x): 0.72, D(G(z)): 0.15 Saving fake_images-57.png Epoch [57/200], Step [200/600], d_loss: 0.6381, g_loss: 1.6368, D(x): 0.80, D(G(z)): 0.23 Epoch [57/200], Step [400/600], d_loss: 0.7536, g_loss: 2.1915, D(x): 0.69, D(G(z)): 0.13 Epoch [57/200], Step [600/600], d_loss: 0.5838, g_loss: 3.3461, D(x): 0.73, D(G(z)): 0.07 Saving fake_images-58.png Epoch [58/200], Step [200/600], d_loss: 0.6520, g_loss: 2.4046, D(x): 0.77, D(G(z)): 0.20 Epoch [58/200], Step [400/600], d_loss: 0.5205, g_loss: 2.5537, D(x): 0.89, D(G(z)): 0.24 Epoch [58/200], Step [600/600], d_loss: 0.5934, g_loss: 3.1467, D(x): 0.79, D(G(z)): 0.15 Saving fake_images-59.png Epoch [59/200], Step [200/600], d_loss: 0.6037, g_loss: 2.1968, D(x): 0.83, D(G(z)): 0.20 Epoch [59/200], Step [400/600], d_loss: 0.3036, g_loss: 3.2215, D(x): 0.88, D(G(z)): 0.10 Epoch [59/200], Step [600/600], d_loss: 0.5576, g_loss: 2.5574, D(x): 0.81, D(G(z)): 0.17 Saving fake_images-60.png Epoch [60/200], Step [200/600], d_loss: 0.5943, g_loss: 3.1721, D(x): 0.85, D(G(z)): 0.23 Epoch [60/200], Step [400/600], d_loss: 0.5263, g_loss: 3.1948, D(x): 0.79, D(G(z)): 0.11 Epoch [60/200], Step [600/600], d_loss: 0.6112, g_loss: 2.8308, D(x): 0.78, D(G(z)): 0.13 Saving fake_images-61.png Epoch [61/200], Step [200/600], d_loss: 0.4659, g_loss: 2.8071, D(x): 0.83, D(G(z)): 0.13 Epoch [61/200], Step [400/600], d_loss: 0.5923, g_loss: 3.6180, D(x): 0.80, D(G(z)): 0.16 Epoch [61/200], Step [600/600], d_loss: 0.6831, g_loss: 2.2917, D(x): 0.82, D(G(z)): 0.23 Saving fake_images-62.png Epoch [62/200], Step [200/600], d_loss: 0.4049, g_loss: 2.9846, D(x): 0.90, D(G(z)): 0.18 Epoch [62/200], Step [400/600], d_loss: 0.5553, g_loss: 2.8905, D(x): 0.84, D(G(z)): 0.22 Epoch [62/200], Step [600/600], d_loss: 0.6991, g_loss: 3.0752, D(x): 0.79, D(G(z)): 0.22 Saving fake_images-63.png Epoch [63/200], Step [200/600], d_loss: 0.6135, g_loss: 3.0198, D(x): 0.80, D(G(z)): 0.15 Epoch [63/200], Step [400/600], d_loss: 0.8987, g_loss: 2.2852, D(x): 0.66, D(G(z)): 0.17 Epoch [63/200], Step [600/600], d_loss: 0.7471, g_loss: 2.0134, D(x): 0.77, D(G(z)): 0.22 Saving fake_images-64.png Epoch [64/200], Step [200/600], d_loss: 0.6890, g_loss: 2.2854, D(x): 0.75, D(G(z)): 0.20 Epoch [64/200], Step [400/600], d_loss: 0.6603, g_loss: 2.8311, D(x): 0.74, D(G(z)): 0.13 Epoch [64/200], Step [600/600], d_loss: 0.6788, g_loss: 2.2177, D(x): 0.79, D(G(z)): 0.25 Saving fake_images-65.png Epoch [65/200], Step [200/600], d_loss: 0.7112, g_loss: 2.2775, D(x): 0.79, D(G(z)): 0.25 Epoch [65/200], Step [400/600], d_loss: 0.8474, g_loss: 2.8152, D(x): 0.64, D(G(z)): 0.13 Epoch [65/200], Step [600/600], d_loss: 0.7285, g_loss: 2.0987, D(x): 0.82, D(G(z)): 0.28 Saving fake_images-66.png Epoch [66/200], Step [200/600], d_loss: 0.5681, g_loss: 2.3763, D(x): 0.81, D(G(z)): 0.17 Epoch [66/200], Step [400/600], d_loss: 0.6359, g_loss: 2.1534, D(x): 0.80, D(G(z)): 0.17 Epoch [66/200], Step [600/600], d_loss: 0.7235, g_loss: 2.5864, D(x): 0.82, D(G(z)): 0.28 Saving fake_images-67.png Epoch [67/200], Step [200/600], d_loss: 0.6004, g_loss: 2.5890, D(x): 0.84, D(G(z)): 0.26 Epoch [67/200], Step [400/600], d_loss: 0.9171, g_loss: 1.6026, D(x): 0.69, D(G(z)): 0.25 Epoch [67/200], Step [600/600], d_loss: 0.6408, g_loss: 1.9177, D(x): 0.86, D(G(z)): 0.30 Saving fake_images-68.png Epoch [68/200], Step [200/600], d_loss: 0.8228, g_loss: 2.9365, D(x): 0.72, D(G(z)): 0.18 Epoch [68/200], Step [400/600], d_loss: 0.7530, g_loss: 2.4185, D(x): 0.77, D(G(z)): 0.28 Epoch [68/200], Step [600/600], d_loss: 0.6875, g_loss: 1.9871, D(x): 0.90, D(G(z)): 0.33 Saving fake_images-69.png Epoch [69/200], Step [200/600], d_loss: 0.8205, g_loss: 2.1659, D(x): 0.72, D(G(z)): 0.23 Epoch [69/200], Step [400/600], d_loss: 0.7377, g_loss: 2.0010, D(x): 0.77, D(G(z)): 0.27 Epoch [69/200], Step [600/600], d_loss: 0.7037, g_loss: 2.2231, D(x): 0.75, D(G(z)): 0.21 Saving fake_images-70.png Epoch [70/200], Step [200/600], d_loss: 0.7086, g_loss: 1.8680, D(x): 0.71, D(G(z)): 0.19 Epoch [70/200], Step [400/600], d_loss: 0.8111, g_loss: 2.1075, D(x): 0.66, D(G(z)): 0.15 Epoch [70/200], Step [600/600], d_loss: 0.7277, g_loss: 1.8202, D(x): 0.80, D(G(z)): 0.27 Saving fake_images-71.png Epoch [71/200], Step [200/600], d_loss: 0.6391, g_loss: 2.0054, D(x): 0.79, D(G(z)): 0.21 Epoch [71/200], Step [400/600], d_loss: 0.7755, g_loss: 2.6408, D(x): 0.82, D(G(z)): 0.27 Epoch [71/200], Step [600/600], d_loss: 0.5087, g_loss: 2.5443, D(x): 0.82, D(G(z)): 0.18 Saving fake_images-72.png Epoch [72/200], Step [200/600], d_loss: 0.7893, g_loss: 1.5719, D(x): 0.80, D(G(z)): 0.29
Exception ignored in: <bound method Image.__del__ of <PIL.Image.Image image mode=L size=28x28 at 0x7F65419A2C18>> Traceback (most recent call last): File "/opt/conda/lib/python3.6/site-packages/PIL/Image.py", line 616, in __del__ if (hasattr(self, 'fp') and hasattr(self, '_exclusive_fp') KeyboardInterrupt
Epoch [72/200], Step [400/600], d_loss: 0.6698, g_loss: 2.0528, D(x): 0.75, D(G(z)): 0.20 Epoch [72/200], Step [600/600], d_loss: 0.4970, g_loss: 1.7228, D(x): 0.84, D(G(z)): 0.19 Saving fake_images-73.png Epoch [73/200], Step [200/600], d_loss: 0.8222, g_loss: 2.0590, D(x): 0.75, D(G(z)): 0.24 Epoch [73/200], Step [400/600], d_loss: 0.7449, g_loss: 1.9482, D(x): 0.75, D(G(z)): 0.25 Epoch [73/200], Step [600/600], d_loss: 0.7072, g_loss: 2.5808, D(x): 0.80, D(G(z)): 0.26 Saving fake_images-74.png Epoch [74/200], Step [200/600], d_loss: 0.9070, g_loss: 2.3779, D(x): 0.66, D(G(z)): 0.16 Epoch [74/200], Step [400/600], d_loss: 0.4813, g_loss: 3.0617, D(x): 0.84, D(G(z)): 0.17 Epoch [74/200], Step [600/600], d_loss: 1.0139, g_loss: 1.9312, D(x): 0.64, D(G(z)): 0.22 Saving fake_images-75.png Epoch [75/200], Step [200/600], d_loss: 0.7343, g_loss: 2.5908, D(x): 0.71, D(G(z)): 0.17 Epoch [75/200], Step [400/600], d_loss: 0.4579, g_loss: 2.7525, D(x): 0.85, D(G(z)): 0.19 Epoch [75/200], Step [600/600], d_loss: 0.6056, g_loss: 1.6635, D(x): 0.85, D(G(z)): 0.27 Saving fake_images-76.png Epoch [76/200], Step [200/600], d_loss: 0.3811, g_loss: 2.7648, D(x): 0.79, D(G(z)): 0.06 Epoch [76/200], Step [400/600], d_loss: 0.7310, g_loss: 2.4639, D(x): 0.76, D(G(z)): 0.22 Epoch [76/200], Step [600/600], d_loss: 0.5597, g_loss: 2.5021, D(x): 0.82, D(G(z)): 0.20 Saving fake_images-77.png Epoch [77/200], Step [200/600], d_loss: 0.6596, g_loss: 2.4048, D(x): 0.76, D(G(z)): 0.16 Epoch [77/200], Step [400/600], d_loss: 0.4955, g_loss: 2.5651, D(x): 0.83, D(G(z)): 0.17 Epoch [77/200], Step [600/600], d_loss: 0.6216, g_loss: 2.1742, D(x): 0.80, D(G(z)): 0.21 Saving fake_images-78.png Epoch [78/200], Step [200/600], d_loss: 0.6920, g_loss: 2.2449, D(x): 0.86, D(G(z)): 0.31 Epoch [78/200], Step [400/600], d_loss: 0.6244, g_loss: 2.2366, D(x): 0.80, D(G(z)): 0.22 Epoch [78/200], Step [600/600], d_loss: 0.6378, g_loss: 2.2754, D(x): 0.80, D(G(z)): 0.23 Saving fake_images-79.png Epoch [79/200], Step [200/600], d_loss: 0.7031, g_loss: 2.5150, D(x): 0.74, D(G(z)): 0.20 Epoch [79/200], Step [400/600], d_loss: 0.8812, g_loss: 2.5055, D(x): 0.73, D(G(z)): 0.19 Epoch [79/200], Step [600/600], d_loss: 0.5665, g_loss: 2.6947, D(x): 0.88, D(G(z)): 0.27 Saving fake_images-80.png Epoch [80/200], Step [200/600], d_loss: 0.6143, g_loss: 2.0029, D(x): 0.80, D(G(z)): 0.25 Epoch [80/200], Step [400/600], d_loss: 0.9320, g_loss: 2.3396, D(x): 0.69, D(G(z)): 0.20 Epoch [80/200], Step [600/600], d_loss: 0.9089, g_loss: 2.0766, D(x): 0.75, D(G(z)): 0.28 Saving fake_images-81.png Epoch [81/200], Step [200/600], d_loss: 1.0123, g_loss: 1.4511, D(x): 0.72, D(G(z)): 0.30 Epoch [81/200], Step [400/600], d_loss: 0.5886, g_loss: 2.2336, D(x): 0.80, D(G(z)): 0.23 Epoch [81/200], Step [600/600], d_loss: 0.8012, g_loss: 2.4067, D(x): 0.76, D(G(z)): 0.25 Saving fake_images-82.png Epoch [82/200], Step [200/600], d_loss: 0.6223, g_loss: 2.4748, D(x): 0.83, D(G(z)): 0.25 Epoch [82/200], Step [400/600], d_loss: 0.6179, g_loss: 2.7216, D(x): 0.79, D(G(z)): 0.22 Epoch [82/200], Step [600/600], d_loss: 0.5481, g_loss: 2.7939, D(x): 0.89, D(G(z)): 0.24 Saving fake_images-83.png Epoch [83/200], Step [200/600], d_loss: 0.7883, g_loss: 2.5383, D(x): 0.74, D(G(z)): 0.23 Epoch [83/200], Step [400/600], d_loss: 0.7012, g_loss: 2.1059, D(x): 0.76, D(G(z)): 0.22 Epoch [83/200], Step [600/600], d_loss: 0.8294, g_loss: 1.4241, D(x): 0.79, D(G(z)): 0.27 Saving fake_images-84.png Epoch [84/200], Step [200/600], d_loss: 0.7192, g_loss: 2.5942, D(x): 0.73, D(G(z)): 0.20 Epoch [84/200], Step [400/600], d_loss: 0.5759, g_loss: 2.2972, D(x): 0.82, D(G(z)): 0.22 Epoch [84/200], Step [600/600], d_loss: 0.8353, g_loss: 2.1791, D(x): 0.74, D(G(z)): 0.26 Saving fake_images-85.png Epoch [85/200], Step [200/600], d_loss: 0.8873, g_loss: 2.0683, D(x): 0.75, D(G(z)): 0.28 Epoch [85/200], Step [400/600], d_loss: 0.6268, g_loss: 2.7777, D(x): 0.76, D(G(z)): 0.18 Epoch [85/200], Step [600/600], d_loss: 0.7168, g_loss: 2.1689, D(x): 0.90, D(G(z)): 0.35 Saving fake_images-86.png Epoch [86/200], Step [200/600], d_loss: 0.7876, g_loss: 2.1433, D(x): 0.75, D(G(z)): 0.28 Epoch [86/200], Step [400/600], d_loss: 0.5393, g_loss: 1.6272, D(x): 0.84, D(G(z)): 0.20 Epoch [86/200], Step [600/600], d_loss: 0.6478, g_loss: 2.1545, D(x): 0.82, D(G(z)): 0.24 Saving fake_images-87.png Epoch [87/200], Step [200/600], d_loss: 0.8858, g_loss: 1.6927, D(x): 0.79, D(G(z)): 0.35 Epoch [87/200], Step [400/600], d_loss: 0.6959, g_loss: 2.2276, D(x): 0.78, D(G(z)): 0.22 Epoch [87/200], Step [600/600], d_loss: 0.5480, g_loss: 2.5375, D(x): 0.84, D(G(z)): 0.21 Saving fake_images-88.png Epoch [88/200], Step [200/600], d_loss: 0.6067, g_loss: 2.3443, D(x): 0.81, D(G(z)): 0.21 Epoch [88/200], Step [400/600], d_loss: 1.0743, g_loss: 1.4923, D(x): 0.67, D(G(z)): 0.29 Epoch [88/200], Step [600/600], d_loss: 0.8223, g_loss: 1.6032, D(x): 0.78, D(G(z)): 0.30 Saving fake_images-89.png Epoch [89/200], Step [200/600], d_loss: 0.7564, g_loss: 2.2622, D(x): 0.73, D(G(z)): 0.24 Epoch [89/200], Step [400/600], d_loss: 0.6608, g_loss: 2.9469, D(x): 0.72, D(G(z)): 0.10 Epoch [89/200], Step [600/600], d_loss: 0.7502, g_loss: 2.1785, D(x): 0.78, D(G(z)): 0.25 Saving fake_images-90.png Epoch [90/200], Step [200/600], d_loss: 0.7515, g_loss: 1.9592, D(x): 0.78, D(G(z)): 0.25 Epoch [90/200], Step [400/600], d_loss: 0.7773, g_loss: 1.8207, D(x): 0.82, D(G(z)): 0.32 Epoch [90/200], Step [600/600], d_loss: 0.6195, g_loss: 2.1637, D(x): 0.84, D(G(z)): 0.26 Saving fake_images-91.png Epoch [91/200], Step [200/600], d_loss: 0.5954, g_loss: 1.9436, D(x): 0.82, D(G(z)): 0.24 Epoch [91/200], Step [400/600], d_loss: 0.8885, g_loss: 2.0942, D(x): 0.72, D(G(z)): 0.26 Epoch [91/200], Step [600/600], d_loss: 0.7492, g_loss: 2.4276, D(x): 0.78, D(G(z)): 0.26 Saving fake_images-92.png Epoch [92/200], Step [200/600], d_loss: 0.6546, g_loss: 2.0184, D(x): 0.77, D(G(z)): 0.23 Epoch [92/200], Step [400/600], d_loss: 0.6591, g_loss: 2.0140, D(x): 0.83, D(G(z)): 0.27 Epoch [92/200], Step [600/600], d_loss: 0.6838, g_loss: 1.8910, D(x): 0.81, D(G(z)): 0.27 Saving fake_images-93.png Epoch [93/200], Step [200/600], d_loss: 0.4914, g_loss: 2.0064, D(x): 0.82, D(G(z)): 0.19 Epoch [93/200], Step [400/600], d_loss: 0.9269, g_loss: 2.2415, D(x): 0.66, D(G(z)): 0.21 Epoch [93/200], Step [600/600], d_loss: 0.6982, g_loss: 2.0645, D(x): 0.73, D(G(z)): 0.21 Saving fake_images-94.png Epoch [94/200], Step [200/600], d_loss: 0.8215, g_loss: 1.9076, D(x): 0.77, D(G(z)): 0.31 Epoch [94/200], Step [400/600], d_loss: 0.7767, g_loss: 1.9038, D(x): 0.78, D(G(z)): 0.29 Epoch [94/200], Step [600/600], d_loss: 0.8882, g_loss: 1.2789, D(x): 0.72, D(G(z)): 0.30 Saving fake_images-95.png Epoch [95/200], Step [200/600], d_loss: 0.9337, g_loss: 2.2290, D(x): 0.67, D(G(z)): 0.21 Epoch [95/200], Step [400/600], d_loss: 0.7894, g_loss: 2.2941, D(x): 0.70, D(G(z)): 0.21 Epoch [95/200], Step [600/600], d_loss: 0.8956, g_loss: 2.2964, D(x): 0.67, D(G(z)): 0.20 Saving fake_images-96.png Epoch [96/200], Step [200/600], d_loss: 0.7505, g_loss: 2.1374, D(x): 0.83, D(G(z)): 0.33 Epoch [96/200], Step [400/600], d_loss: 0.8142, g_loss: 1.6617, D(x): 0.85, D(G(z)): 0.37 Epoch [96/200], Step [600/600], d_loss: 0.8163, g_loss: 1.4099, D(x): 0.76, D(G(z)): 0.30 Saving fake_images-97.png Epoch [97/200], Step [200/600], d_loss: 0.6815, g_loss: 2.2155, D(x): 0.74, D(G(z)): 0.19 Epoch [97/200], Step [400/600], d_loss: 0.8907, g_loss: 1.6872, D(x): 0.73, D(G(z)): 0.31 Epoch [97/200], Step [600/600], d_loss: 0.6261, g_loss: 2.6885, D(x): 0.79, D(G(z)): 0.22 Saving fake_images-98.png
--------------------------------------------------------------------------- KeyboardInterrupt Traceback (most recent call last) <ipython-input-44-2261b1e09a0c> in <module> 9 # Train the discriminator and generator 10 d_loss, real_score, fake_score = train_discriminator(images) ---> 11 g_loss, fake_images = train_generator() 12 13 # Inspect the losses <ipython-input-41-cead41276cd7> in train_generator() 7 8 # Backprop and optimize ----> 9 reset_grad() 10 g_loss.backward() 11 g_optimizer.step() <ipython-input-39-13b325afbe63> in reset_grad() 1 def reset_grad(): 2 d_optimizer.zero_grad() ----> 3 g_optimizer.zero_grad() 4 5 def train_discriminator(images): /opt/conda/lib/python3.6/site-packages/torch/optim/optimizer.py in zero_grad(self) 161 for group in self.param_groups: 162 for p in group['params']: --> 163 if p.grad is not None: 164 p.grad.detach_() 165 p.grad.zero_() KeyboardInterrupt:

Now that we have trained the models, we can save checkpoints.

In [ ]:
# Save the model checkpoints 
torch.save(G.state_dict(), 'G.ckpt')
torch.save(D.state_dict(), 'D.ckpt')
In [ ]:
import jovian
In [ ]:
jovian.log_metrics({
    'd_loss': 0.5275, 
    'g_loss': 3.2325,
    'D(x)': 0.82, 
    'D(G(z))': 0.11
})

Save and Commit

In [45]:
pip install jovian --upgrade
Collecting jovian Downloading https://files.pythonhosted.org/packages/b8/a1/dd7b5bcf0a3f043c894151f5e523c1f88ab72ed5672694f325e2f39e85f1/jovian-0.1.97-py2.py3-none-any.whl (50kB) |████████████████████████████████| 51kB 2.0MB/s eta 0:00:011 Requirement already satisfied, skipping upgrade: requests in /opt/conda/lib/python3.6/site-packages (from jovian) (2.22.0) Requirement already satisfied, skipping upgrade: click in /opt/conda/lib/python3.6/site-packages (from jovian) (7.0) Collecting uuid Downloading https://files.pythonhosted.org/packages/ce/63/f42f5aa951ebf2c8dac81f77a8edcc1c218640a2a35a03b9ff2d4aa64c3d/uuid-1.30.tar.gz Requirement already satisfied, skipping upgrade: pyyaml in /opt/conda/lib/python3.6/site-packages (from jovian) (5.1.2) Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /opt/conda/lib/python3.6/site-packages (from requests->jovian) (2019.9.11) Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.6/site-packages (from requests->jovian) (1.24.2) Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /opt/conda/lib/python3.6/site-packages (from requests->jovian) (2.8) Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.6/site-packages (from requests->jovian) (3.0.4) Building wheels for collected packages: uuid Building wheel for uuid (setup.py) ... done Created wheel for uuid: filename=uuid-1.30-cp36-none-any.whl size=6501 sha256=4b0c18bb968430919b84bfc09a5a58ed346a993d82e3a3b4c1fdbfd825e40dfd Stored in directory: /tmp/.cache/pip/wheels/2a/80/9b/015026567c29fdffe31d91edbe7ba1b17728db79194fca1f21 Successfully built uuid Installing collected packages: uuid, jovian Successfully installed jovian-0.1.97 uuid-1.30 Note: you may need to restart the kernel to use updated packages.
In [46]:
import jovian
In [ ]:
jovian.commit(artifacts=[
    'G.ckpt', 
    'D.ckpt', 
    'samples/fake_images-1.png',
    'samples/fake_images-10.png',
    'samples/fake_images-25.png',
    'samples/fake_images-50.png',
    'samples/fake_images-100.png',
    'samples/fake_images-150.png',
    'samples/fake_images-199.png',
])
[jovian] Saving notebook..
In [ ]: