"Hello MNIST" of Deep Neural Network

Predicting Hand Written Digits using Deep Neural Network

Images of digits were taken from a variety of scanned documents, normalized in size and centered. We will be using this dataset to create our Neural model and predict the digit from its image.

Each image is a 28 by 28 pixel square (784 pixels total). A standard spit of the dataset is used to evaluate and compare models, where 60,000 images are used to train a model and a separate set of 10,000 images are used to test it

Prerequisites

1. Python 3.5
2. TensorFlow
3. Keras Python Module

Note: Recommended to download Anaconda Package which comes with almost every essential module

It is a digit recognition task. As such there are 10 digits (0 to 9) or 10 classes to predict. Results are reported using prediction error, which is nothing more than the inverted classification accuracy.

Lets Import the python libraries

Python Code
from keras.datasets import mnist import numpy from keras.datasets import mnist from keras.models import Sequential from keras.layers import Dense from keras.layers import Dropout from keras.layers import Flatten from keras.layers.convolutional import Conv2D from keras.layers.convolutional import MaxPooling2D from keras.utils import np_utils from keras import backend as Keras

Python Code

Keras.set_image_dim_ordering('th') # fix random seed for reproducibility seed = 7 numpy.random.seed(seed) # load data (X_train, y_train), (X_test, y_test) = mnist.load_data() # reshape to be [samples][pixels][width][height] X_train = X_train.reshape(X_train.shape[0], 1, 28, 28).astype('float32') X_test = X_test.reshape(X_test.shape[0], 1, 28, 28).astype('float32') # normalize inputs from 0-255 to 0-1 X_train = X_train / 255 X_test = X_test / 255 # one hot encode outputs y_train = np_utils.to_categorical(y_train) y_test = np_utils.to_categorical(y_test) num_classes = y_test.shape[1]

The network topology can be summarized as follows.

• Convolutional layer with 30 feature maps of size 5×5.
• Pooling layer taking the max over 2*2 patches.
• Convolutional layer with 15 feature maps of size 3×3.
• Pooling layer taking the max over 2*2 patches.
• Dropout layer with a probability of 20%.
• Flatten layer.
• Fully connected layer with 128 neurons and rectifier activation.
• Fully connected layer with 50 neurons and rectifier activation.
• Output layer.

Python Code

def larger_model():   # create model   model = Sequential()   model.add(Conv2D(30, (5, 5), input_shape=(1, 28, 28), activation='relu'))   model.add(MaxPooling2D(pool_size=(2, 2)))   model.add(Conv2D(15, (3, 3), activation='relu'))   model.add(MaxPooling2D(pool_size=(2, 2)))   model.add(Dropout(0.2))   model.add(Flatten())   model.add(Dense(128, activation='relu'))   model.add(Dense(50, activation='relu'))   model.add(Dense(num_classes, activation='softmax'))   # Compile model   model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])   return model # build the model model = larger_model() # Fit the model model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, batch_size=200) # Final evaluation of the model scores = model.evaluate(X_test, y_test, verbose=0) print("Large CNN Error: %.2f%%" % (100-scores[1]*100))

🙂 Wolla You are done!

Running the example prints accuracy on the training and validation datasets each epoch and a final classification error rate.

The model takes about 100 seconds to run per epoch. This slightly larger model achieves the respectable classification error rate of about 0.89% (on my machine, yours can be different)