#100daysoftensorflow

in #100DaysOfCode, #100DaysOfData, #100DaysOfTensorflow

Structuring Tensorflow code

For this third day of challenge, let’s re-explore the Fashion MNIST dataset. We will restructure the code so it’s more readable and introduce CNNs.

During the next days, I will explore Tensorflow for at least 1 hour per day and post the notebooks, data and models to this repository.

The notebook for the third day is available here.

Step 1: a brief introduction to the problem

Python is a great scripting language, which makes it great for fast prototyping. Notebooks are also great: you can write pieces of code and visualize the results of your work right after typing.

The downside with Python and notebooks is that we tend to write code that is hardly reusable and that looks like a mess.

Also, some kinds of structures are more adapted to certain types of networks. Tensorflow puts at your disposal a sequential and a functional APIs. More on these here, here and here.

Today, we will refactor our MNIST code so we use the subclass approach. We will also create a very basic Convolutional Neural Network (CNN).

Before moving on, here are some things to retain (source: Tensorflow guides):

  • A Sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor.
  • A Sequential model is not appropriate when the model (or one of the layers) has multiple inputs or multiple outputs; you need to do layer sharing or you want non-linear topology.
  • The functional API can handle models with non-linear topology, models with shared layers, and models with multiple inputs or outputs.
  • The main idea that a deep learning model is usually a directed acyclic graph (DAG) of layers. So the functional API is a way to build graphs of layers.

Usually, in introductory tutorials, you will be introduced to the Sequential API, which is simpler to understand. It looks like to something like this:

# source: https://www.tensorflow.org/guide/keras/sequential_model
# define Sequential model with 3 layers
model = keras.Sequential(
    [
        layers.Dense(2, activation="relu", name="layer1"),
        layers.Dense(3, activation="relu", name="layer2"),
        layers.Dense(4, name="layer3"),
    ]
)

A functional approach looks like to this:

# source: https://machinelearningmastery.com/keras-functional-api-deep-learning/
visible = Input(shape=(10,))
hidden1 = Dense(10, activation='relu')(visible)
hidden2 = Dense(20, activation='relu')(hidden1)
hidden3 = Dense(10, activation='relu')(hidden2)
output = Dense(1, activation='sigmoid')(hidden3)
model = Model(inputs=visible, outputs=output)

Step 2: repeat the same steps of Day 2 until we reach the model

If you haven’t, please read the Day 2 post to understand what’s happening here.

# import libraries
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

# import the fashion MNIST dataset
mnist = tf.keras.datasets.fashion_mnist

# get  the train and test sets
(train_imgs, train_labels), (test_imgs, test_labels) = mnist.load_data()

# set the labels
labels = ['Top', 'Pants', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

# Check shapes
print('There are {} images on this set.'.format(train_imgs.shape[0]))
print('The images are in the {} x {} pxs format.'.format(train_imgs.shape[1], train_imgs.shape[2]))

Step 3: preprocess the images

Before feeding the network with the images, they have to be reshaped in order to add 1 single color channel. Since reshaping is a preprocessing step – such like normalisation – I have created a function that will normalize and reshape the arrays of images.

def preprocess(trainset, testset):
  # normalize
  trainset = trainset / 255.0
  testset  = testset / 255.0
  
  # reshape dataset to single channel
  trainset = trainset.reshape((trainset.shape[0], 28, 28, 1))
  testset  = testset.reshape((testset.shape[0], 28, 28, 1))

  return trainset, testset

# normalize and reshape images
train_imgs, test_imgs = preprocess(train_imgs, test_imgs)

Step 4: build the model

Today, we will use the model class to define the outer model. It uses the Model API provided by Keras. This class:

  • Exposes the fit, evaluate and predict methods.
  • Exposes the list of its inner layers.
  • Exposes saving and serialization APIs.

But before, a few explanations about the model.

I have decided to add a Convolutional and a MaxPool layer. To understand what they are for, please read this article. Then, I will add a Dropout layer to try avoiding overfitting.

class MNISTModel(tf.keras.Model):
  def __init__(self):
    super(MNISTModel, self).__init__()
    self.conv1   = tf.keras.layers.Conv2D(32, 
                                          kernel_size=(3,3), 
                                          input_shape=(28,28,1), 
                                          activation='relu')
    
    self.maxpool = tf.keras.layers.MaxPooling2D(pool_size=(2,2))
    self.dropout = tf.keras.layers.Dropout(0.2)

    self.flatten = tf.keras.layers.Flatten()
    self.d1      = tf.keras.layers.Dense(128, activation='relu')
    self.d2      = tf.keras.layers.Dense(10, activation='softmax')

  def call(self, x):
    x = self.conv1(x)
    x = self.maxpool(x)
    x = self.dropout(x)

    x = self.flatten(x)
    x = self.d1(x)
    return self.d2(x)

model = MNISTModel()

Now, we just compile and train our model as usual.

# compile
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# fit
model.fit(train_imgs, 
          train_labels, 
          epochs=10)

And then, we evaluate it:

test_loss, test_acc = model.evaluate(test_imgs, test_labels, verbose=2)
print('\nModel accuracy: {:.0f}%'.format(test_acc*100))

This model has an accuracy of 92% – 4% more than we achieved in day 2.

Conclusion: what we learned today

Today, we saw that’s possible to better organize code so it’s more readable, but also, to take advantage of the tools that the Tensorflow API provides.

We also built a Convolutional Neural Network to improve our mode. But the gain in performance here was minimal – only 4%.

This brings the question: such an improvement justifies this change? It depends on the application of your model.

Here, IT and deontology merge. If your model has a high impact on people’s lives and a decision made using it has a decisive effect on people’s health, their economies, their civil rights and so on, then even the slightest improvement matters.

As you saw, we also refactored the preprocessing step, transforming it into a function.


Do you want to connect? It will be a pleasure to discuss Machine Learning with you. Drop me a message on LinkedIn.

Leave a Reply