GANs — Deep Convolutional GANs with CIFAR10 (Part 8)

Brief theoretical introduction to Deep Convolutional Generative Adversarial Networks or DCGANs and practical implementation using Python and Keras/TensorFlow in Jupyter Notebook.

Deep Convolutional GANs or DCGANs with CIFAR10 bt fernanda rodríguez

In this article, you will find:

• Research paper,
• Definition, network design, and cost function, and
• Training DCGANs with CIFAR10 dataset using Python and Keras/TensorFlow in Jupyter Notebook.

Research Paper

Radford, A., Metz, L., & Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434.

Deep Convolutional Generative Adversarial Networks — DCGANs

The difference between the simple GAN and the DCGAN, is the generator of the DCGAN uses the transposed convolution (Fractionally-strided convolution or Deconvolution) technique to perform up-sampling of 2D image size.

DCGAN are mainly composes of:

• Convolution layers without max pooling or fully connected layers.
• It uses convolutional stride and transposed convolution for the downsampling and the upsampling.

Read more about GANs:

GANs — A brief overview (Part 1)

Read more about Convolutional neural networks — CNN:

ML & DL — Convolutional neural networks (Part 6)

Network design

Deep Convolutional Generative Adversarial Networks Architecture or DCGAN by fernanda rodríguez.

x is the real data and z is the latent space.

Cost function

Cost function DCGANs by fernanda rodríguez.

Training DCGANs

1. Data: CIFAR10 dataset

(X_train, y_train), (X_test, y_test) = cifar10.load_data()

2. Model:

• Generator

generator = Sequential()
# FC: 2x2x512
generator.add(Dense(2*2*512, input_shape=(latent_dim,), kernel_initializer=init))
generator.add(Reshape((7, 7, 128)))
# Conv 1: 4x4x256
generator.add(Conv2DTranspose(256, kernel_size=3, strides=2, padding='same'))
generator.add(BatchNormalization(momentum=0.8))
generator.add(ReLU(0.2))
# Conv 2 and Conv 3
...
# Conv 4: 32x32x3
generator.add(Conv2DTranspose(3, kernel_size=5, strides=2, padding='same', activation='tanh'))

• Discriminator

# Discriminator network
discriminator = Sequential()
# Conv 1: 16x16x64
discriminator.add(Conv2D(64, kernel_size=5, strides=2, padding='same',
                         input_shape=(img_shape), kernel_initializer=init))
discriminator.add(LeakyReLU(0.2))

# Conv 2, 3 and 4
...
# FC
discriminator.add(Flatten())

# Output
discriminator.add(Dense(1, activation='sigmoid'))

3. Compile

discriminator.compile(Adam(lr=0.0003, beta_1=0.5), loss='binary_crossentropy', metrics=['binary_accuracy'])
discriminator.trainable = False

z = Input(shape=(latent_dim,))
img = generator(z)
decision = discriminator(img)
d_g = Model(inputs=z, outputs=decision)

d_g.compile(Adam(lr=0.0004, beta_1=0.5), loss='binary_crossentropy',
            metrics=['binary_accuracy'])

4. Fit

# Train Discriminator weights
discriminator.trainable = True
        
# Real samples
X_batch = X_train[i*batch_size:(i+1)*batch_size]
d_loss_real = discriminator.train_on_batch(x=X_batch, y=real * (1 - smooth))
        
# Fake Samples
z = np.random.normal(loc=0, scale=1, size=(batch_size, latent_dim))
X_fake = generator.predict_on_batch(z)
d_loss_fake = discriminator.train_on_batch(x=X_fake, y=fake)
         
# Discriminator loss
d_loss_batch = 0.5 * (d_loss_real[0] + d_loss_fake[0])
        
# Train Generator weights
discriminator.trainable = False
d_g_loss_batch = d_g.train_on_batch(x=z, y=real)

5. Evaluate

# plotting the metrics 
plt.plot(d_loss) 
plt.plot(d_g_loss) 
plt.show()

DCGANs — CIFAR10 results

epoch = 1/100, d_loss=0.199, g_loss=0.032 in DCGAN_CIFAR10

epoch = 100/100, d_loss=0.184, g_loss=5.819 in DCGAN_CIFAR10

Train summary

Train summary DCGAN by fernanda rodríguez

Github repository

Look the complete training DCGAN with CIFAR10 dataset, using Python and Keras/TensorFlow in Jupyter Notebook.

mafda/generative_adversarial_networks_101

For those looking for all the articles in our GANs series. Here is the link.

GANs — Generative Adversarial Networks 101

References

• Convolution
• Unsupervised representation learning with deep convolutional generative adversarial networks.
• Generative Adversarial Networks
• CIFAR10 dataset
• How to Train a GAN? Tips and tricks to make GANs work
• The GAN Zoo
• Adversarial Nets Papers
• Keras-GAN