Traffic Sign Classification with Keras

Keras exists to make coding deep neural networks simpler. To demonstrate just how easy it is, you’re going to use Keras to build a convolutional neural network in a few dozen lines of code.

You’ll be connecting the concepts from the previous lessons to the methods that Keras provides.

Dataset

The network you'll build with Keras is similar to the example that you can find in Keras’s GitHub repository that builds out a convolutional neural network for MNIST.

However, instead of using the MNIST dataset, you're going to use the German Traffic Sign Recognition Benchmark dataset that you've used previously.

You can download pickle files with sanitized traffic sign data here.

Overview

Here are the steps you'll take to build the network:

  1. First load the training data and do a train/validation split.
  2. Preprocess data.
  3. Build a feedforward neural network to classify traffic signs.
  4. Build a convolutional neural network to classify traffic signs.
  5. Evaluate performance of final neural network on testing data.

Keep an eye on the network’s accuracy over time. Once the accuracy reaches the 98% range, you can be confident that you’ve built and trained an effective model.

In [12]:
import pickle
import numpy as np
from sklearn.model_selection import train_test_split
import math

Load the Data

Start by importing the data from the pickle file.

In [13]:
# TODO: Implement load the data here.
with open('./data/train.p', 'rb') as f:
    data = pickle.load(f)

Validate the Network

Split the training data into a training and validation set.

Measure the validation accuracy of the network after two training epochs.

Hint: Use the train_test_split() method from scikit-learn.

In [14]:
# TODO: Use `train_test_split` here.
X_train, X_val, y_train, y_val = train_test_split(data['features'], data['labels'], random_state=0, test_size=0.33)
In [15]:
print(X_train.shape)
(26270, 32, 32, 3)
In [16]:
# STOP: Do not change the tests below. Your implementation should pass these tests. 
assert(X_train.shape[0] == y_train.shape[0]), "The number of images is not equal to the number of labels."
assert(X_train.shape[1:] == (32,32,3)), "The dimensions of the images are not 32 x 32 x 3."
assert(X_val.shape[0] == y_val.shape[0]), "The number of images is not equal to the number of labels."
assert(X_val.shape[1:] == (32,32,3)), "The dimensions of the images are not 32 x 32 x 3."

Preprocess the Data

Now that you've loaded the training data, preprocess the data such that it's in the range between -0.5 and 0.5.

In [17]:
# TODO: Implement data normalization here.
X_train = X_train.astype('float32')
X_val = X_val.astype('float32')
X_train = X_train / 255 - 0.5
X_val = X_val / 255 - 0.5
In [18]:
# STOP: Do not change the tests below. Your implementation should pass these tests. 
assert(math.isclose(np.min(X_train), -0.5, abs_tol=1e-5) and math.isclose(np.max(X_train), 0.5, abs_tol=1e-5)), "The range of the training data is: %.1f to %.1f" % (np.min(X_train), np.max(X_train))
assert(math.isclose(np.min(X_val), -0.5, abs_tol=1e-5) and math.isclose(np.max(X_val), 0.5, abs_tol=1e-5)), "The range of the validation data is: %.1f to %.1f" % (np.min(X_val), np.max(X_val))

Build a Two-Layer Feedfoward Network

The code you've written so far is for data processing, not specific to Keras. Here you're going to build Keras-specific code.

Build a two-layer feedforward neural network, with 128 neurons in the fully-connected hidden layer.

To get started, review the Keras documentation about models and layers.

The Keras example of a Multi-Layer Perceptron network is similar to what you need to do here. Use that as a guide, but keep in mind that there are a number of differences.

In [19]:
# TODO: Build a two-layer feedforward neural network with Keras here.
from keras.models import Sequential
from keras.layers import Dense, Input, Activation

model = Sequential()
model.add(Dense(128, activation='relu', input_shape=(32*32*3,)))
model.add(Dense(43, activation='softmax'))
In [20]:
# STOP: Do not change the tests below. Your implementation should pass these tests.
dense_layers = []
for l in model.layers:
    if type(l) == Dense:
        dense_layers.append(l)
assert(len(dense_layers) == 2), "There should be 2 Dense layers."
d1 = dense_layers[0]
d2 = dense_layers[1]
assert(d1.input_shape == (None, 3072))
assert(d1.output_shape == (None, 128))
assert(d2.input_shape == (None, 128))
assert(d2.output_shape == (None, 43))

last_layer = model.layers[-1]
assert(last_layer.activation.__name__ == 'softmax'), "Last layer should be softmax activation, is {}.".format(last_layer.activation.__name__)
In [21]:
# Debugging
for l in model.layers:
    print(l.name, l.input_shape, l.output_shape, l.activation)
dense_1 (None, 3072) (None, 128) <function relu at 0x125dcee18>
dense_2 (None, 128) (None, 43) <function softmax at 0x125dcebf8>

Train the Network

Compile and train the network for 2 epochs. Use the adam optimizer, with categorical_crossentropy loss.

Hint 1: In order to use categorical cross entropy, you will need to one-hot encode the labels.

Hint 2: In order to pass the input images to the fully-connected hidden layer, you will need to reshape the input.

Hint 3: Keras's .fit() method returns a History.history object, which the tests below use. Save that to a variable named history.

In [24]:
# TODO: Compile and train the model here.
from keras.utils import np_utils

Y_train = np_utils.to_categorical(y_train, 43)
Y_val = np_utils.to_categorical(y_val, 43)

X_train_flat = X_train.reshape(-1, 32*32*3)
X_val_flat = X_val.reshape(-1, 32*32*3)

model.summary()
# TODO: Compile and train the model here.
model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])
history = model.fit(X_train_flat, Y_train,
                    batch_size=128, nb_epoch=20,
                    verbose=1, validation_data=(X_val_flat, Y_val))
____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
dense_1 (Dense)                  (None, 128)           393344      dense_input_1[0][0]              
____________________________________________________________________________________________________
dense_2 (Dense)                  (None, 43)            5547        dense_1[0][0]                    
====================================================================================================
Total params: 398,891
Trainable params: 398,891
Non-trainable params: 0
____________________________________________________________________________________________________
Train on 26270 samples, validate on 12939 samples
Epoch 1/20
26270/26270 [==============================] - 2s - loss: 1.9058 - acc: 0.5064 - val_loss: 1.1807 - val_acc: 0.6739
Epoch 2/20
26270/26270 [==============================] - 1s - loss: 0.9279 - acc: 0.7625 - val_loss: 0.8255 - val_acc: 0.7953
Epoch 3/20
26270/26270 [==============================] - 1s - loss: 0.6643 - acc: 0.8303 - val_loss: 0.6450 - val_acc: 0.8341
Epoch 4/20
26270/26270 [==============================] - 1s - loss: 0.5315 - acc: 0.8649 - val_loss: 0.5970 - val_acc: 0.8382
Epoch 5/20
26270/26270 [==============================] - 1s - loss: 0.4483 - acc: 0.8838 - val_loss: 0.4866 - val_acc: 0.8623
Epoch 6/20
26270/26270 [==============================] - 1s - loss: 0.3888 - acc: 0.9000 - val_loss: 0.3849 - val_acc: 0.9022
Epoch 7/20
26270/26270 [==============================] - 1s - loss: 0.3466 - acc: 0.9093 - val_loss: 0.3878 - val_acc: 0.8868
Epoch 8/20
26270/26270 [==============================] - 1s - loss: 0.3109 - acc: 0.9209 - val_loss: 0.3497 - val_acc: 0.9090
Epoch 9/20
26270/26270 [==============================] - 1s - loss: 0.3135 - acc: 0.9140 - val_loss: 0.3788 - val_acc: 0.8919
Epoch 10/20
26270/26270 [==============================] - 1s - loss: 0.2650 - acc: 0.9315 - val_loss: 0.2906 - val_acc: 0.9226
Epoch 11/20
26270/26270 [==============================] - 1s - loss: 0.2424 - acc: 0.9374 - val_loss: 0.3340 - val_acc: 0.9077
Epoch 12/20
26270/26270 [==============================] - 1s - loss: 0.2269 - acc: 0.9405 - val_loss: 0.2889 - val_acc: 0.9200
Epoch 13/20
26270/26270 [==============================] - 1s - loss: 0.2065 - acc: 0.9477 - val_loss: 0.2914 - val_acc: 0.9225
Epoch 14/20
26270/26270 [==============================] - 1s - loss: 0.2132 - acc: 0.9428 - val_loss: 0.3072 - val_acc: 0.9118
Epoch 15/20
26270/26270 [==============================] - 1s - loss: 0.1928 - acc: 0.9499 - val_loss: 0.3107 - val_acc: 0.9088
Epoch 16/20
26270/26270 [==============================] - 2s - loss: 0.1879 - acc: 0.9515 - val_loss: 0.2292 - val_acc: 0.9447
Epoch 17/20
26270/26270 [==============================] - 1s - loss: 0.1866 - acc: 0.9495 - val_loss: 0.2469 - val_acc: 0.9335
Epoch 18/20
26270/26270 [==============================] - 1s - loss: 0.1669 - acc: 0.9559 - val_loss: 0.3115 - val_acc: 0.9088
Epoch 19/20
26270/26270 [==============================] - 1s - loss: 0.1820 - acc: 0.9491 - val_loss: 0.3168 - val_acc: 0.9036
Epoch 20/20
26270/26270 [==============================] - 1s - loss: 0.1627 - acc: 0.9556 - val_loss: 0.2454 - val_acc: 0.9339
In [25]:
# STOP: Do not change the tests below. Your implementation should pass these tests.
assert(history.history['acc'][-1] > 0.92), "The training accuracy was: %.3f" % history.history['acc'][-1]
assert(history.history['val_acc'][-1] > 0.9), "The validation accuracy is: %.3f" % history.history['val_acc'][-1]

Validation Accuracy: (fill in here)

Congratulations

You've built a feedforward neural network in Keras!

Don't stop here! Next, you'll add a convolutional layer to drive.py.

Convolutions

Build a new network, similar to your existing network. Before the hidden layer, add a 3x3 convolutional layer with 32 filters and valid padding.

Then compile and train the network.

Hint 1: The Keras example of a convolutional neural network for MNIST would be a good example to review.

Hint 2: Now that the first layer of the network is a convolutional layer, you no longer need to reshape the input images before passing them to the network. You might need to reload your training data to recover the original shape.

Hint 3: Add a Flatten() layer between the convolutional layer and the fully-connected hidden layer.

In [26]:
# TODO: Re-construct the network and add a convolutional layer before the first fully-connected layer.
# NOTE: RELOAD DATA & NORMALIZE BEFORE RUNNING 
from keras.layers import Conv2D, Flatten

Y_train = np_utils.to_categorical(y_train, 43)
Y_val = np_utils.to_categorical(y_val, 43)


model = Sequential()
model.add(Conv2D(32, 3, 3, input_shape=(32, 32, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(43, activation='softmax'))

model.summary()
# TODO: Compile and train the model here.
model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])
history = model.fit(X_train, Y_train,
                    batch_size=128, nb_epoch=20,
                    verbose=1, validation_data=(X_val, Y_val))
____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
convolution2d_1 (Convolution2D)  (None, 30, 30, 32)    896         convolution2d_input_1[0][0]      
____________________________________________________________________________________________________
flatten_1 (Flatten)              (None, 28800)         0           convolution2d_1[0][0]            
____________________________________________________________________________________________________
dense_3 (Dense)                  (None, 128)           3686528     flatten_1[0][0]                  
____________________________________________________________________________________________________
dense_4 (Dense)                  (None, 43)            5547        dense_3[0][0]                    
====================================================================================================
Total params: 3,692,971
Trainable params: 3,692,971
Non-trainable params: 0
____________________________________________________________________________________________________
Train on 26270 samples, validate on 12939 samples
Epoch 1/20
 6272/26270 [======>.......................] - ETA: 15s - loss: 2.9047 - acc: 0.2680
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
<ipython-input-26-173262e94825> in <module>()
     20 history = model.fit(X_train, Y_train,
     21                     batch_size=128, nb_epoch=20,
---> 22                     verbose=1, validation_data=(X_val, Y_val))

/Users/tylerfolkman/anaconda/envs/carnd/lib/python3.5/site-packages/keras/models.py in fit(self, x, y, batch_size, nb_epoch, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, **kwargs)
    662                               shuffle=shuffle,
    663                               class_weight=class_weight,
--> 664                               sample_weight=sample_weight)
    665 
    666     def evaluate(self, x, y, batch_size=32, verbose=1,

/Users/tylerfolkman/anaconda/envs/carnd/lib/python3.5/site-packages/keras/engine/training.py in fit(self, x, y, batch_size, nb_epoch, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch)
   1141                               val_f=val_f, val_ins=val_ins, shuffle=shuffle,
   1142                               callback_metrics=callback_metrics,
-> 1143                               initial_epoch=initial_epoch)
   1144 
   1145     def evaluate(self, x, y, batch_size=32, verbose=1, sample_weight=None):

/Users/tylerfolkman/anaconda/envs/carnd/lib/python3.5/site-packages/keras/engine/training.py in _fit_loop(self, f, ins, out_labels, batch_size, nb_epoch, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch)
    841                 batch_logs['size'] = len(batch_ids)
    842                 callbacks.on_batch_begin(batch_index, batch_logs)
--> 843                 outs = f(ins_batch)
    844                 if not isinstance(outs, list):
    845                     outs = [outs]

/Users/tylerfolkman/anaconda/envs/carnd/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py in __call__(self, inputs)
   1601         session = get_session()
   1602         updated = session.run(self.outputs + [self.updates_op],
-> 1603                               feed_dict=feed_dict)
   1604         return updated[:len(self.outputs)]
   1605 

/Users/tylerfolkman/anaconda/envs/carnd/lib/python3.5/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
    715     try:
    716       result = self._run(None, fetches, feed_dict, options_ptr,
--> 717                          run_metadata_ptr)
    718       if run_metadata:
    719         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/Users/tylerfolkman/anaconda/envs/carnd/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
    913     if final_fetches or final_targets:
    914       results = self._do_run(handle, final_targets, final_fetches,
--> 915                              feed_dict_string, options, run_metadata)
    916     else:
    917       results = []

/Users/tylerfolkman/anaconda/envs/carnd/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
    963     if handle is None:
    964       return self._do_call(_run_fn, self._session, feed_dict, fetch_list,
--> 965                            target_list, options, run_metadata)
    966     else:
    967       return self._do_call(_prun_fn, self._session, handle, feed_dict,

/Users/tylerfolkman/anaconda/envs/carnd/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
    970   def _do_call(self, fn, *args):
    971     try:
--> 972       return fn(*args)
    973     except errors.OpError as e:
    974       message = compat.as_text(e.message)

/Users/tylerfolkman/anaconda/envs/carnd/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
    952         return tf_session.TF_Run(session, options,
    953                                  feed_dict, fetch_list, target_list,
--> 954                                  status, run_metadata)
    955 
    956     def _prun_fn(session, handle, feed_dict, fetch_list):

KeyboardInterrupt: 
In [31]:
# STOP: Do not change the tests below. Your implementation should pass these tests.
assert(history.history['val_acc'][-1] > 0.93), "The validation accuracy is: %.3f" % history.history['val_acc'][-1]

Validation Accuracy: (fill in here)

Pooling

Re-construct your network and add a 2x2 pooling layer immediately following your convolutional layer.

Then compile and train the network.

In [32]:
# TODO: Re-construct the network and add a pooling layer after the convolutional layer.
from keras.layers import Conv2D, Flatten, MaxPooling2D, Activation

Y_train = np_utils.to_categorical(y_train, 43)
Y_val = np_utils.to_categorical(y_val, 43)


model = Sequential()
model.add(Conv2D(32, 3, 3, input_shape=(32, 32, 3)))
model.add(MaxPooling2D((2,2)))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(43, activation='softmax'))

model.summary()
# TODO: Compile and train the model here.
model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])
history = model.fit(X_train, Y_train,
                    batch_size=128, nb_epoch=20,
                    verbose=1, validation_data=(X_val, Y_val))
____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
convolution2d_2 (Convolution2D)  (None, 30, 30, 32)    896         convolution2d_input_2[0][0]      
____________________________________________________________________________________________________
maxpooling2d_1 (MaxPooling2D)    (None, 15, 15, 32)    0           convolution2d_2[0][0]            
____________________________________________________________________________________________________
activation_1 (Activation)        (None, 15, 15, 32)    0           maxpooling2d_1[0][0]             
____________________________________________________________________________________________________
flatten_2 (Flatten)              (None, 7200)          0           activation_1[0][0]               
____________________________________________________________________________________________________
dense_5 (Dense)                  (None, 128)           921728      flatten_2[0][0]                  
____________________________________________________________________________________________________
dense_6 (Dense)                  (None, 43)            5547        dense_5[0][0]                    
====================================================================================================
Total params: 928171
____________________________________________________________________________________________________
Train on 26270 samples, validate on 12939 samples
Epoch 1/20
26270/26270 [==============================] - 3s - loss: 1.6611 - acc: 0.5658 - val_loss: 0.7439 - val_acc: 0.8177
Epoch 2/20
26270/26270 [==============================] - 2s - loss: 0.5127 - acc: 0.8741 - val_loss: 0.3953 - val_acc: 0.8910
Epoch 3/20
26270/26270 [==============================] - 2s - loss: 0.2769 - acc: 0.9351 - val_loss: 0.2366 - val_acc: 0.9492
Epoch 4/20
26270/26270 [==============================] - 2s - loss: 0.1826 - acc: 0.9601 - val_loss: 0.2000 - val_acc: 0.9542
Epoch 5/20
26270/26270 [==============================] - 2s - loss: 0.1341 - acc: 0.9698 - val_loss: 0.1652 - val_acc: 0.9621
Epoch 6/20
26270/26270 [==============================] - 2s - loss: 0.0982 - acc: 0.9804 - val_loss: 0.1420 - val_acc: 0.9673
Epoch 7/20
26270/26270 [==============================] - 2s - loss: 0.0734 - acc: 0.9851 - val_loss: 0.1488 - val_acc: 0.9642
Epoch 8/20
26270/26270 [==============================] - 2s - loss: 0.0630 - acc: 0.9871 - val_loss: 0.1221 - val_acc: 0.9733
Epoch 9/20
26270/26270 [==============================] - 2s - loss: 0.0498 - acc: 0.9902 - val_loss: 0.1270 - val_acc: 0.9689
Epoch 10/20
26270/26270 [==============================] - 2s - loss: 0.0470 - acc: 0.9899 - val_loss: 0.1542 - val_acc: 0.9636
Epoch 11/20
26270/26270 [==============================] - 2s - loss: 0.0431 - acc: 0.9904 - val_loss: 0.1066 - val_acc: 0.9756
Epoch 12/20
26270/26270 [==============================] - 2s - loss: 0.0302 - acc: 0.9948 - val_loss: 0.1181 - val_acc: 0.9760
Epoch 13/20
26270/26270 [==============================] - 2s - loss: 0.0418 - acc: 0.9905 - val_loss: 0.1460 - val_acc: 0.9681
Epoch 14/20
26270/26270 [==============================] - 2s - loss: 0.0239 - acc: 0.9958 - val_loss: 0.1036 - val_acc: 0.9780
Epoch 15/20
26270/26270 [==============================] - 2s - loss: 0.0169 - acc: 0.9974 - val_loss: 0.1165 - val_acc: 0.9764
Epoch 16/20
26270/26270 [==============================] - 2s - loss: 0.0149 - acc: 0.9979 - val_loss: 0.1014 - val_acc: 0.9798
Epoch 17/20
26270/26270 [==============================] - 2s - loss: 0.0121 - acc: 0.9984 - val_loss: 0.1110 - val_acc: 0.9772
Epoch 18/20
26270/26270 [==============================] - 2s - loss: 0.0297 - acc: 0.9935 - val_loss: 0.1761 - val_acc: 0.9624
Epoch 19/20
26270/26270 [==============================] - 2s - loss: 0.0398 - acc: 0.9895 - val_loss: 0.1179 - val_acc: 0.9769
Epoch 20/20
26270/26270 [==============================] - 2s - loss: 0.0145 - acc: 0.9967 - val_loss: 0.1156 - val_acc: 0.9774
In [33]:
# STOP: Do not change the tests below. Your implementation should pass these tests.
assert(history.history['val_acc'][-1] > 0.93), "The validation accuracy is: %.3f" % history.history['val_acc'][-1]

Validation Accuracy: (fill in here)

Dropout

Re-construct your network and add dropout after the pooling layer. Set the dropout rate to 50%.

In [34]:
# TODO: Re-construct the network and add dropout after the pooling layer.
from keras.layers import Dropout

model = Sequential()
model.add(Conv2D(32, 3, 3, input_shape=(32, 32, 3)))
model.add(MaxPooling2D((2,2)))
model.add((Dropout(0.5)))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(43, activation='softmax'))

model.summary()
# TODO: Compile and train the model here.
model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])
history = model.fit(X_train, Y_train,
                    batch_size=128, nb_epoch=20,
                    verbose=1, validation_data=(X_val, Y_val))
____________________________________________________________________________________________________
Layer (type)                     Output Shape          Param #     Connected to                     
====================================================================================================
convolution2d_3 (Convolution2D)  (None, 30, 30, 32)    896         convolution2d_input_3[0][0]      
____________________________________________________________________________________________________
maxpooling2d_2 (MaxPooling2D)    (None, 15, 15, 32)    0           convolution2d_3[0][0]            
____________________________________________________________________________________________________
dropout_1 (Dropout)              (None, 15, 15, 32)    0           maxpooling2d_2[0][0]             
____________________________________________________________________________________________________
activation_2 (Activation)        (None, 15, 15, 32)    0           dropout_1[0][0]                  
____________________________________________________________________________________________________
flatten_3 (Flatten)              (None, 7200)          0           activation_2[0][0]               
____________________________________________________________________________________________________
dense_7 (Dense)                  (None, 128)           921728      flatten_3[0][0]                  
____________________________________________________________________________________________________
dense_8 (Dense)                  (None, 43)            5547        dense_7[0][0]                    
====================================================================================================
Total params: 928171
____________________________________________________________________________________________________
Train on 26270 samples, validate on 12939 samples
Epoch 1/20
26270/26270 [==============================] - 3s - loss: 1.7386 - acc: 0.5394 - val_loss: 0.7919 - val_acc: 0.7802
Epoch 2/20
26270/26270 [==============================] - 2s - loss: 0.6021 - acc: 0.8372 - val_loss: 0.3917 - val_acc: 0.9090
Epoch 3/20
26270/26270 [==============================] - 2s - loss: 0.3669 - acc: 0.9036 - val_loss: 0.2811 - val_acc: 0.9377
Epoch 4/20
26270/26270 [==============================] - 2s - loss: 0.2681 - acc: 0.9304 - val_loss: 0.2067 - val_acc: 0.9528
Epoch 5/20
26270/26270 [==============================] - 2s - loss: 0.2136 - acc: 0.9437 - val_loss: 0.1699 - val_acc: 0.9620
Epoch 6/20
26270/26270 [==============================] - 2s - loss: 0.1720 - acc: 0.9552 - val_loss: 0.1658 - val_acc: 0.9615
Epoch 7/20
26270/26270 [==============================] - 2s - loss: 0.1557 - acc: 0.9593 - val_loss: 0.1678 - val_acc: 0.9568
Epoch 8/20
26270/26270 [==============================] - 3s - loss: 0.1397 - acc: 0.9618 - val_loss: 0.1242 - val_acc: 0.9721
Epoch 9/20
26270/26270 [==============================] - 3s - loss: 0.1182 - acc: 0.9694 - val_loss: 0.1220 - val_acc: 0.9733
Epoch 10/20
26270/26270 [==============================] - 3s - loss: 0.1084 - acc: 0.9700 - val_loss: 0.1199 - val_acc: 0.9737
Epoch 11/20
26270/26270 [==============================] - 3s - loss: 0.1028 - acc: 0.9719 - val_loss: 0.1352 - val_acc: 0.9671
Epoch 12/20
26270/26270 [==============================] - 2s - loss: 0.0945 - acc: 0.9749 - val_loss: 0.1069 - val_acc: 0.9755
Epoch 13/20
26270/26270 [==============================] - 2s - loss: 0.0894 - acc: 0.9753 - val_loss: 0.1126 - val_acc: 0.9746
Epoch 14/20
26270/26270 [==============================] - 2s - loss: 0.0821 - acc: 0.9770 - val_loss: 0.1097 - val_acc: 0.9754
Epoch 15/20
26270/26270 [==============================] - 2s - loss: 0.0771 - acc: 0.9783 - val_loss: 0.1128 - val_acc: 0.9752
Epoch 16/20
26270/26270 [==============================] - 2s - loss: 0.0740 - acc: 0.9789 - val_loss: 0.0990 - val_acc: 0.9781
Epoch 17/20
26270/26270 [==============================] - 2s - loss: 0.0679 - acc: 0.9817 - val_loss: 0.1085 - val_acc: 0.9774
Epoch 18/20
26270/26270 [==============================] - 2s - loss: 0.0656 - acc: 0.9820 - val_loss: 0.1107 - val_acc: 0.9761
Epoch 19/20
26270/26270 [==============================] - 2s - loss: 0.0571 - acc: 0.9837 - val_loss: 0.0894 - val_acc: 0.9816
Epoch 20/20
26270/26270 [==============================] - 2s - loss: 0.0626 - acc: 0.9834 - val_loss: 0.1004 - val_acc: 0.9760
In [35]:
# STOP: Do not change the tests below. Your implementation should pass these tests.
assert(history.history['val_acc'][-1] > 0.93), "The validation accuracy is: %.3f" % history.history['val_acc'][-1]

Validation Accuracy: (fill in here)

Optimization

Congratulations! You've built a neural network with convolutions, pooling, dropout, and fully-connected layers, all in just a few lines of code.

Have fun with the model and see how well you can do! Add more layers, or regularization, or different padding, or batches, or more training epochs.

What is the best validation accuracy you can achieve?

In [ ]:
 

Best Validation Accuracy: (fill in here)

Testing

Once you've picked out your best model, it's time to test it.

Load up the test data and use the evaluate() method to see how well it does.

Hint 1: The evaluate() method should return an array of numbers. Use the metrics_names() method to get the labels.

In [36]:
# TODO: Load test data
with open('./test.p', mode='rb') as f:
    test = pickle.load(f)
    
# TODO: Preprocess data & one-hot encode the labels
X_test = test['features']
y_test = test['labels']
X_test = X_test.astype('float32')
X_test /= 255
X_test -= 0.5
Y_test = np_utils.to_categorical(y_test, 43)

# TODO: Evaluate model on test data
model.evaluate(X_test, Y_test)
12630/12630 [==============================] - 1s     
Out[36]:
[0.5543278131253504, 0.89794140943722034]

Test Accuracy: (fill in here)

Summary

Keras is a great tool to use if you want to quickly build a neural network and evaluate performance.