PL03-Topic02, Keras
Back to the previous page｜page management
List of posts to read before reading this article
Contents
- Installation
- Tutorials
- Data Load/Save
- An Introduction to Deep Learning and Keras
- Keras in Action
- Deep Neural Networks for Supervised Learning: Regression
- Deep Neural Networks for Supervised Learning: Classification
- Tuning and Deploying Deep Neural Networks
- The Path Ahead
Installation
For linux
$
For windows
Version Control
Tutorials
Dataset : loader
Neural net : Custom layers
Sequential
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test)
Module
Optimization : Training
Evaluation : Predicting
Data Load/Save
An Introduction to Deep Learning and Keras
- Defining the Model Structure
- Training the Model
- Making Predictions
A Sneak Peek into the Keras Framework without validation dataset
# Import required packages
from keras.models import Sequential
from keras.layers import Dense
import numpy as np
# Getting the data ready
# Generate train dummy data for 1000 Students and dummy test for 500
# Columns :Age, Hours of Study &Avg Previous test scores
np.random.seed(2018) #Setting seed for reproducibility
train_data, test_data = np.random.random((1000, 3)), np.random.random((500, 3))
# Generate dummy results for 1000 students : Whether Passed (1) or Failed (0)
train_labels = np.random.randint(2, size=(1000, 1))
test_labels = np.random.randint(2, size=(500, 1))
# Defining the model structure with the required layers,
# of neurons, activation function and optimizers
model = Sequential()
model.add(Dense(5, input_dim=3, activation='relu'))
model.add(Dense(4, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(train_data, train_labels, batch_size=32, epochs=3)
Training result
Epoch 1/3
1000/1000 [==============================] - 1s 1ms/step - loss: 0.6939 - acc: 0.4870
Epoch 2/3
1000/1000 [==============================] - 0s 41us/step - loss: 0.6934 - acc: 0.4830
Epoch 3/3
1000/1000 [==============================] - 0s 39us/step - loss: 0.6933 - acc: 0.4880
Testing the Model Performance
results = model.evaluate(test_data, test_labels)
for i in range(len(model.metrics_names)):
print(model.metrics_names[i]," : ", results[i])
OUTPUT
500/500 [==============================] - 0s 813us/step
loss : 0.692422465801239
acc : 0.5200000002384185
Keras in Action
- Input Data
- Neuron
- Activation Function
- Sigmoid Activation Function
- Model
- Layers
- The Loss Function
- Optimizers
- Metrics
- Model Configuration
- Model Training
- Model Evaluation
Putting All the Building Blocks Together
# Import required packages
import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from keras.datasets import boston_housing
#Download the data using Keras; this will need an active internet connection
(x_train, y_train), (x_test, y_test) = boston_housing.load_data()
#Extract the last 100 rows from the training data to create the validation datasets.
x_val = x_train[300:,] # last 300 row
y_val = y_train[300:,] # last 300 row
#Define the model architecture
model = Sequential()
model.add(Dense(13, input_dim=13, kernel_initializer='normal', activation='relu'))
model.add(Dense(6, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_absolute_percentage_error'])
#Train the model
model.fit(x_train, y_train, batch_size=32, epochs=3, validation_data=(x_val,y_val))
Training result
Downloading data from https://s3.amazonaws.com/keras-datasets/boston_housing.npz
57344/57026 [==============================] - 0s 2us/step
Train on 404 samples, validate on 104 samples
Epoch 1/3
404/404 [==============================] - 1s 2ms/step - loss: 580.9334 - mean_absolute_percentage_error: 99.4362 - val_loss: 662.7088 - val_mean_absolute_percentage_error: 98.0074
Epoch 2/3
404/404 [==============================] - 0s 51us/step - loss: 554.0448 - mean_absolute_percentage_error: 95.8691 - val_loss: 618.2133 - val_mean_absolute_percentage_error: 93.0662
Epoch 3/3
404/404 [==============================] - 0s 50us/step - loss: 492.5111 - mean_absolute_percentage_error: 87.4437 - val_loss: 520.5322 - val_mean_absolute_percentage_error: 81.4210
Testing the Model Performance
results = model.evaluate(x_test, y_test)
for i in range(len(model.metrics_names)):
print(model.metrics_names[i]," : ", results[i])
OUTPUT
102/102 [==============================] - 0s 62us/step
loss : 463.09825942095586
mean_absolute_percentage_error : 78.82570214365043
Plotting the Loss Metric Across Epochs
from keras.callbacks import History
history = History() model.fit(x_train, y_train, batch_size=32, epochs=3, validation_data=(x_val,y_val), callbacks=[history])
import matplotlib.pyplot as plt
plt.plot(history.history[‘loss’]) plt.plot(history.history[‘val_loss’])
# Import required packages
import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from keras.datasets import boston_housing
#Download the data using Keras; this will need an active internet connection
(x_train, y_train), (x_test, y_test) = boston_housing.load_data()
#Extract the last 100 rows from the training data to create the validation datasets.
x_val = x_train[300:,] # last 300 row
y_val = y_train[300:,] # last 300 row
# Recode History
from keras.callbacks import History
history = History()
#Define the model architecture
model = Sequential()
model.add(Dense(13, input_dim=13, kernel_initializer='normal', activation='relu'))
model.add(Dense(6, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_absolute_percentage_error'])
#Train the model
model.fit(x_train, y_train, batch_size=32, epochs=3, validation_data=(x_val,y_val), callbacks=[history])
# visualization for metric
import matplotlib.pyplot as plt
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title("Model's Training & Validation loss across epochs")
plt.ylabel('Loss')
plt.xlabel('Epochs')
plt.legend(['Train_loss', 'Validation_loss'], loc='upper right')
plt.show()
Re-training(after tuning fitting parameters)
#Train the model
model.fit(x_train, y_train, batch_size=32, epochs=30, validation_data=(x_val,y_val))
OUTPUT
Train on 404 samples, validate on 104 samples
Epoch 1/30
404/404 [==============================] - 0s 58us/step - loss: 379.4086 - mean_absolute_percentage_error: 70.2558 - val_loss: 360.9293 - val_mean_absolute_percentage_error: 59.3759
Epoch 2/30
404/404 [==============================] - 0s 47us/step - loss: 228.6982 - mean_absolute_percentage_error: 48.1161 - val_loss: 198.9693 - val_mean_absolute_percentage_error: 40.2195
Epoch 3/30
404/404 [==============================] - 0s 49us/step - loss: 137.9041 - mean_absolute_percentage_error: 43.9592 - val_loss: 143.6277 - val_mean_absolute_percentage_error: 42.3353
Epoch 4/30
404/404 [==============================] - 0s 50us/step - loss: 128.0068 - mean_absolute_percentage_error: 48.4147 - val_loss: 136.8944 - val_mean_absolute_percentage_error: 40.8082
Epoch 5/30
404/404 [==============================] - 0s 55us/step - loss: 118.3841 - mean_absolute_percentage_error: 44.0519 - val_loss: 133.1419 - val_mean_absolute_percentage_error: 37.5794
Epoch 6/30
404/404 [==============================] - 0s 49us/step - loss: 111.5623 - mean_absolute_percentage_error: 40.6460 - val_loss: 128.8092 - val_mean_absolute_percentage_error: 35.8778
Epoch 7/30
404/404 [==============================] - 0s 50us/step - loss: 105.6107 - mean_absolute_percentage_error: 39.3712 - val_loss: 121.8723 - val_mean_absolute_percentage_error: 35.2191
Epoch 8/30
404/404 [==============================] - 0s 60us/step - loss: 100.5365 - mean_absolute_percentage_error: 38.5076 - val_loss: 116.7530 - val_mean_absolute_percentage_error: 34.0602
Epoch 9/30
404/404 [==============================] - 0s 55us/step - loss: 95.6473 - mean_absolute_percentage_error: 36.4750 - val_loss: 112.9952 - val_mean_absolute_percentage_error: 32.6358
Epoch 10/30
404/404 [==============================] - 0s 56us/step - loss: 91.3528 - mean_absolute_percentage_error: 35.8936 - val_loss: 107.6109 - val_mean_absolute_percentage_error: 32.4268
Epoch 11/30
404/404 [==============================] - 0s 62us/step - loss: 86.8670 - mean_absolute_percentage_error: 34.5687 - val_loss: 105.0734 - val_mean_absolute_percentage_error: 30.8377
Epoch 12/30
404/404 [==============================] - 0s 57us/step - loss: 83.8678 - mean_absolute_percentage_error: 33.1729 - val_loss: 100.9976 - val_mean_absolute_percentage_error: 30.1347
Epoch 13/30
404/404 [==============================] - 0s 79us/step - loss: 80.3338 - mean_absolute_percentage_error: 32.9414 - val_loss: 98.0269 - val_mean_absolute_percentage_error: 29.1839
Epoch 14/30
404/404 [==============================] - 0s 49us/step - loss: 77.0266 - mean_absolute_percentage_error: 31.3633 - val_loss: 96.0304 - val_mean_absolute_percentage_error: 28.0592
Epoch 15/30
404/404 [==============================] - 0s 69us/step - loss: 74.5713 - mean_absolute_percentage_error: 30.6367 - val_loss: 93.4559 - val_mean_absolute_percentage_error: 27.4387
Epoch 16/30
404/404 [==============================] - 0s 76us/step - loss: 72.4605 - mean_absolute_percentage_error: 30.5300 - val_loss: 90.2561 - val_mean_absolute_percentage_error: 27.5172
Epoch 17/30
404/404 [==============================] - 0s 56us/step - loss: 70.2668 - mean_absolute_percentage_error: 30.4674 - val_loss: 89.6360 - val_mean_absolute_percentage_error: 26.2286
Epoch 18/30
404/404 [==============================] - 0s 59us/step - loss: 69.0524 - mean_absolute_percentage_error: 28.2962 - val_loss: 89.3691 - val_mean_absolute_percentage_error: 25.0894
Epoch 19/30
404/404 [==============================] - 0s 59us/step - loss: 67.1439 - mean_absolute_percentage_error: 28.7645 - val_loss: 86.4003 - val_mean_absolute_percentage_error: 25.7927
Epoch 20/30
404/404 [==============================] - 0s 65us/step - loss: 66.1058 - mean_absolute_percentage_error: 29.1919 - val_loss: 85.8313 - val_mean_absolute_percentage_error: 25.0127
Epoch 21/30
404/404 [==============================] - 0s 55us/step - loss: 65.2683 - mean_absolute_percentage_error: 27.9101 - val_loss: 85.7227 - val_mean_absolute_percentage_error: 24.2154
Epoch 22/30
404/404 [==============================] - 0s 57us/step - loss: 64.3430 - mean_absolute_percentage_error: 28.2435 - val_loss: 84.4628 - val_mean_absolute_percentage_error: 24.4009
Epoch 23/30
404/404 [==============================] - 0s 56us/step - loss: 63.5753 - mean_absolute_percentage_error: 27.9445 - val_loss: 83.9498 - val_mean_absolute_percentage_error: 24.2201
Epoch 24/30
404/404 [==============================] - 0s 62us/step - loss: 62.9955 - mean_absolute_percentage_error: 28.0562 - val_loss: 83.1876 - val_mean_absolute_percentage_error: 24.3564
Epoch 25/30
404/404 [==============================] - 0s 52us/step - loss: 62.5542 - mean_absolute_percentage_error: 27.7791 - val_loss: 83.8650 - val_mean_absolute_percentage_error: 23.6271
Epoch 26/30
404/404 [==============================] - 0s 53us/step - loss: 62.1328 - mean_absolute_percentage_error: 27.1743 - val_loss: 82.4939 - val_mean_absolute_percentage_error: 24.0875
Epoch 27/30
404/404 [==============================] - 0s 58us/step - loss: 61.6133 - mean_absolute_percentage_error: 27.6719 - val_loss: 82.2328 - val_mean_absolute_percentage_error: 23.8878
Epoch 28/30
404/404 [==============================] - 0s 61us/step - loss: 60.8626 - mean_absolute_percentage_error: 26.9424 - val_loss: 82.7886 - val_mean_absolute_percentage_error: 23.2897
Epoch 29/30
404/404 [==============================] - 0s 54us/step - loss: 60.6314 - mean_absolute_percentage_error: 26.1994 - val_loss: 81.6793 - val_mean_absolute_percentage_error: 23.6167
Epoch 30/30
404/404 [==============================] - 0s 52us/step - loss: 60.5237 - mean_absolute_percentage_error: 27.4995 - val_loss: 81.5046 - val_mean_absolute_percentage_error: 23.4413
<keras.callbacks.History at 0x7fd199e50c18>
Re-Testing the Model Performance
results = model.evaluate(x_test, y_test)
for i in range(len(model.metrics_names)):
print(model.metrics_names[i]," : ", results[i])
OUTPUT
102/102 [==============================] - 0s 92us/step
loss : 65.75437209185432
mean_absolute_percentage_error : 31.099634806315105
Deep Neural Networks for Supervised Learning: Regression
Exploring the Data
OUTPUT
Data Engineering
OUTPUT
Defining Model Baseline Performance
OUTPUT
Designing the DNN
Data preprocessing
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import OneHotEncoder
df = pd.read_csv("train.csv")
store = pd.read_csv("store.csv")
df_new = df.merge(store, on=["Store"], how="inner")
# We can extract all date properties from a datetime datatype
df_new['Date'] = pd.to_datetime(df_new['Date'], infer_datetime_format=True)
df_new["Month"] = df_new["Date"].dt.month
df_new["Quarter"] = df_new["Date"].dt.quarter
df_new["Year"] = df_new["Date"].dt.year
df_new["Day"] = df_new["Date"].dt.day
df_new["Week"] = df_new["Date"].dt.week
df_new["Season"] = np.where(df_new["Month"].isin([3, 4, 5]), "Spring",
np.where(df_new["Month"].isin([6, 7, 8]), "Summer",
np.where(df_new["Month"].isin([9, 10, 11]), "Fall",
np.where(df_new["Month"].isin([12, 1, 2]), "Winter", "None"))))
# Replace nulls with the mode
df_new["CompetitionDistance"] = df_new["CompetitionDistance"].fillna(df_new["CompetitionDistance"].mode()[0])
# Double check if we still see nulls for the column
df_new["CompetitionDistance"].isnull().sum() / df_new.shape[0] * 100
# Define a variable for each type of feature
target = ["Sales"]
numeric_columns = ["Customers", "Open", "Promo", "Promo2", "StateHoliday", "SchoolHoliday", "CompetitionDistance"]
categorical_columns = ["DayOfWeek", "Quarter", "Month", "Year", "StoreType", "Assortment", "Season"]
# Define a function that will intake the raw dataframe and the column name and return a one hot encoded DF
def create_ohe(df, col):
le = LabelEncoder()
a = le.fit_transform(df_new[col]).reshape(-1, 1)
ohe = OneHotEncoder(sparse=False)
column_names = [col + "_" + str(i) for i in le.classes_]
return (pd.DataFrame(ohe.fit_transform(a), columns=column_names))
# Since the above function converts the column, one at a time
# We create a loop to create the final dataset with all features
temp = df_new[numeric_columns]
for column in categorical_columns:
temp_df = create_ohe(df_new, column)
temp = pd.concat([temp, temp_df], axis=1)
temp["StateHoliday"] = np.where(temp["StateHoliday"] == '0', 0, 1)
# Create train and test dataset with an 80:20 split
x_train, x_test, y_train, y_test = train_test_split(temp, df_new[target], test_size=0.2, random_state=2018)
# Further divide training dataset into train and validation dataset with an 90:10 split
x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size=0.1, random_state=2018)
Training dataset
from keras.models import Sequential
from keras.layers import Dense
#Create Deep Neural Network Architecture
model = Sequential()
model.add(Dense(150,input_dim = 44,activation="relu")) #The input_dim =44, since the width of the training data=44 (refer data engg section)
model.add(Dense(1,activation = "linear"))
#Configure the model
model.compile(optimizer='adam',loss="mean_absolute_error", metrics=["mean_absolute_error"])
#Train the model
model.fit(x_train.values,y_train.values, validation_data= (x_val,y_val),epochs=10,batch_size=64)
Training result
Train on 732390 samples, validate on 81377 samples
Epoch 1/10
732390/732390 [==============================] - 16s 22us/step - loss: 961.6725 - mean_absolute_error: 961.6725 - val_loss: 832.9768 - val_mean_absolute_error: 832.9768
Epoch 2/10
732390/732390 [==============================] - 15s 20us/step - loss: 788.6400 - mean_absolute_error: 788.6400 - val_loss: 750.9315 - val_mean_absolute_error: 750.9315
Epoch 3/10
732390/732390 [==============================] - 14s 19us/step - loss: 745.1940 - mean_absolute_error: 745.1940 - val_loss: 729.6170 - val_mean_absolute_error: 729.6170
Epoch 4/10
732390/732390 [==============================] - 13s 18us/step - loss: 728.3255 - mean_absolute_error: 728.3255 - val_loss: 714.6160 - val_mean_absolute_error: 714.6160
Epoch 5/10
732390/732390 [==============================] - 14s 19us/step - loss: 717.4465 - mean_absolute_error: 717.4465 - val_loss: 708.1066 - val_mean_absolute_error: 708.1066
Epoch 6/10
732390/732390 [==============================] - 14s 19us/step - loss: 708.3802 - mean_absolute_error: 708.3802 - val_loss: 748.1029 - val_mean_absolute_error: 748.1029
Epoch 7/10
732390/732390 [==============================] - 15s 20us/step - loss: 703.5348 - mean_absolute_error: 703.5348 - val_loss: 713.9228 - val_mean_absolute_error: 713.9228
Epoch 8/10
732390/732390 [==============================] - 14s 19us/step - loss: 698.0711 - mean_absolute_error: 698.0711 - val_loss: 694.0802 - val_mean_absolute_error: 694.0802
Epoch 9/10
732390/732390 [==============================] - 14s 19us/step - loss: 693.7195 - mean_absolute_error: 693.7195 - val_loss: 689.4929 - val_mean_absolute_error: 689.4929
Epoch 10/10
732390/732390 [==============================] - 14s 19us/step - loss: 688.7827 - mean_absolute_error: 688.7827 - val_loss: 687.1623 - val_mean_absolute_error: 687.1623
Testing the Model Performance
#Use the model's evaluate method to predict and evaluate the test datasets
result = model.evaluate(x_test.values,y_test.values)
#Print the results
for i in range(len(model.metrics_names)):
print("Metric ",model.metrics_names[i],":",str(round(result[i],2)))
OUTPUT
203442/203442 [==============================] - 3s 14us/step
Metric loss : 684.04
Metric mean_absolute_error : 684.04
Improving the Model 1
Data preprocessing
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import OneHotEncoder
from keras.models import Sequential
from keras.layers import Dense, Dropout
df = pd.read_csv("train.csv")
store = pd.read_csv("store.csv")
df_new = df.merge(store, on=["Store"], how="inner")
# We can extract all date properties from a datetime datatype
df_new['Date'] = pd.to_datetime(df_new['Date'], infer_datetime_format=True)
df_new["Month"] = df_new["Date"].dt.month
df_new["Quarter"] = df_new["Date"].dt.quarter
df_new["Year"] = df_new["Date"].dt.year
df_new["Day"] = df_new["Date"].dt.day
df_new["Week"] = df_new["Date"].dt.week
df_new["Season"] = np.where(df_new["Month"].isin([3, 4, 5]), "Spring",
np.where(df_new["Month"].isin([6, 7, 8]), "Summer",
np.where(df_new["Month"].isin([9, 10, 11]), "Fall",
np.where(df_new["Month"].isin([12, 1, 2]), "Winter", "None"))))
# Replace nulls with the mode
df_new["CompetitionDistance"] = df_new["CompetitionDistance"].fillna(df_new["CompetitionDistance"].mode()[0])
# Double check if we still see nulls for the column
df_new["CompetitionDistance"].isnull().sum() / df_new.shape[0] * 100
# Define a variable for each type of feature
target = ["Sales"]
numeric_columns = ["Customers", "Open", "Promo", "Promo2", "StateHoliday", "SchoolHoliday", "CompetitionDistance"]
categorical_columns = ["DayOfWeek", "Quarter", "Month", "Year", "StoreType", "Assortment", "Season"]
# Define a function that will intake the raw dataframe and the column name and return a one hot encoded DF
def create_ohe(df, col):
le = LabelEncoder()
a = le.fit_transform(df_new[col]).reshape(-1, 1)
ohe = OneHotEncoder(sparse=False)
column_names = [col + "_" + str(i) for i in le.classes_]
return (pd.DataFrame(ohe.fit_transform(a), columns=column_names))
# Since the above function converts the column, one at a time
# We create a loop to create the final dataset with all features
temp = df_new[numeric_columns]
for column in categorical_columns:
temp_df = create_ohe(df_new, column)
temp = pd.concat([temp, temp_df], axis=1)
temp["StateHoliday"] = np.where(temp["StateHoliday"] == '0', 0, 1)
# Create train and test dataset with an 80:20 split
x_train, x_test, y_train, y_test = train_test_split(temp, df_new[target], test_size=0.2, random_state=2018)
# Further divide training dataset into train and validation dataset with an 90:10 split
x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size=0.1, random_state=2018)
Training dataset
#Create Deep Neural Network Architecture
model = Sequential()
model.add(Dense(150,input_dim = 44,activation="relu"))
model.add(Dense(150,activation="relu"))
model.add(Dense(150,activation="relu"))
model.add(Dense(1,activation = "linear"))
model.compile(optimizer='adam',loss="mean_squared_error",metrics=["mean_absolute_error"])
model.fit(x_train,y_train, validation_data=(x_val, y_val),epochs=10,batch_size=64)
result = model.evaluate(x_test,y_test)
for i in range(len(model.metrics_names)):
print("Metric ",model.metrics_names[i],":",str(round(result[i],2)))
OUTPUT
Train on 732390 samples, validate on 81377 samples
Epoch 1/10
732390/732390 [==============================] - 37s 50us/step - loss: 1751824.2928 - mean_absolute_error: 856.5363 - val_loss: 1177755.9705 - val_mean_absolute_error: 740.5539
Epoch 2/10
732390/732390 [==============================] - 29s 40us/step - loss: 1160876.5663 - mean_absolute_error: 723.2560 - val_loss: 1121593.8268 - val_mean_absolute_error: 706.5737
Epoch 3/10
732390/732390 [==============================] - 30s 41us/step - loss: 1087959.8210 - mean_absolute_error: 699.7806 - val_loss: 1019937.0690 - val_mean_absolute_error: 680.0693
Epoch 4/10
732390/732390 [==============================] - 33s 45us/step - loss: 1055263.2415 - mean_absolute_error: 689.1528 - val_loss: 985578.7209 - val_mean_absolute_error: 668.9547
Epoch 5/10
732390/732390 [==============================] - 31s 43us/step - loss: 1025371.6030 - mean_absolute_error: 679.8681 - val_loss: 1125251.2369 - val_mean_absolute_error: 728.9359
Epoch 6/10
732390/732390 [==============================] - 31s 42us/step - loss: 1000331.7273 - mean_absolute_error: 673.0017 - val_loss: 974033.9541 - val_mean_absolute_error: 665.9759
Epoch 7/10
732390/732390 [==============================] - 31s 42us/step - loss: 980059.8089 - mean_absolute_error: 667.9678 - val_loss: 899603.4905 - val_mean_absolute_error: 645.6330
Epoch 8/10
732390/732390 [==============================] - 31s 42us/step - loss: 959681.5355 - mean_absolute_error: 660.6007 - val_loss: 901967.1271 - val_mean_absolute_error: 644.1287
Epoch 9/10
732390/732390 [==============================] - 31s 42us/step - loss: 938922.9098 - mean_absolute_error: 653.8427 - val_loss: 921845.1646 - val_mean_absolute_error: 661.7743
Epoch 10/10
732390/732390 [==============================] - 31s 43us/step - loss: 918371.6692 - mean_absolute_error: 646.9734 - val_loss: 868885.5710 - val_mean_absolute_error: 631.0993
Metric loss : 680.39
Metric mean_absolute_error : 680.39
Improving the Model 2
Data preprocessing
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import OneHotEncoder
from keras.models import Sequential
from keras.layers import Dense, Dropout
df = pd.read_csv("train.csv")
store = pd.read_csv("store.csv")
df_new = df.merge(store, on=["Store"], how="inner")
# We can extract all date properties from a datetime datatype
df_new['Date'] = pd.to_datetime(df_new['Date'], infer_datetime_format=True)
df_new["Month"] = df_new["Date"].dt.month
df_new["Quarter"] = df_new["Date"].dt.quarter
df_new["Year"] = df_new["Date"].dt.year
df_new["Day"] = df_new["Date"].dt.day
df_new["Week"] = df_new["Date"].dt.week
df_new["Season"] = np.where(df_new["Month"].isin([3, 4, 5]), "Spring",
np.where(df_new["Month"].isin([6, 7, 8]), "Summer",
np.where(df_new["Month"].isin([9, 10, 11]), "Fall",
np.where(df_new["Month"].isin([12, 1, 2]), "Winter", "None"))))
# Replace nulls with the mode
df_new["CompetitionDistance"] = df_new["CompetitionDistance"].fillna(df_new["CompetitionDistance"].mode()[0])
# Double check if we still see nulls for the column
df_new["CompetitionDistance"].isnull().sum() / df_new.shape[0] * 100
# Define a variable for each type of feature
target = ["Sales"]
numeric_columns = ["Customers", "Open", "Promo", "Promo2", "StateHoliday", "SchoolHoliday", "CompetitionDistance"]
categorical_columns = ["DayOfWeek", "Quarter", "Month", "Year", "StoreType", "Assortment", "Season"]
# Define a function that will intake the raw dataframe and the column name and return a one hot encoded DF
def create_ohe(df, col):
le = LabelEncoder()
a = le.fit_transform(df_new[col]).reshape(-1, 1)
ohe = OneHotEncoder(sparse=False)
column_names = [col + "_" + str(i) for i in le.classes_]
return (pd.DataFrame(ohe.fit_transform(a), columns=column_names))
# Since the above function converts the column, one at a time
# We create a loop to create the final dataset with all features
temp = df_new[numeric_columns]
for column in categorical_columns:
temp_df = create_ohe(df_new, column)
temp = pd.concat([temp, temp_df], axis=1)
temp["StateHoliday"] = np.where(temp["StateHoliday"] == '0', 0, 1)
# Create train and test dataset with an 80:20 split
x_train, x_test, y_train, y_test = train_test_split(temp, df_new[target], test_size=0.2, random_state=2018)
# Further divide training dataset into train and validation dataset with an 90:10 split
x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size=0.1, random_state=2018)
Training dataset
#Create Deep Neural Network Architecture
model = Sequential()
model.add(Dense(150,input_dim = 44,activation="relu"))
model.add(Dense(150,activation="relu"))
model.add(Dense(150,activation="relu"))
model.add(Dense(150,activation="relu"))
model.add(Dense(150,activation="relu"))
model.add(Dense(1,activation = "linear"))
model.compile(optimizer='adam',loss="mean_squared_error",metrics=["mean_absolute_error"])
model.fit(x_train,y_train, validation_data=(x_val,y_val), epochs=15,batch_size=64)
result = model.evaluate(x_test,y_test)
for i in range(len(model.metrics_names)):
print("Metric ",model.metrics_names[i],":",str(round(result [i],2)))
OUTPUT
Train on 732390 samples, validate on 81377 samples
Epoch 1/15
732390/732390 [==============================] - 47s 64us/step - loss: 1693355.7884 - mean_absolute_error: 851.3013 - val_loss: 1145939.5552 - val_mean_absolute_error: 723.8296
Epoch 2/15
732390/732390 [==============================] - 44s 60us/step - loss: 1161802.7680 - mean_absolute_error: 722.7223 - val_loss: 1047540.8904 - val_mean_absolute_error: 697.5821
Epoch 3/15
732390/732390 [==============================] - 44s 59us/step - loss: 1096405.1840 - mean_absolute_error: 701.0095 - val_loss: 1066299.1614 - val_mean_absolute_error: 686.2207
Epoch 4/15
732390/732390 [==============================] - 44s 60us/step - loss: 1059591.2408 - mean_absolute_error: 689.2741 - val_loss: 1115480.0252 - val_mean_absolute_error: 722.9864
Epoch 5/15
732390/732390 [==============================] - 44s 60us/step - loss: 1037127.7020 - mean_absolute_error: 682.4807 - val_loss: 972973.2906 - val_mean_absolute_error: 678.2566
Epoch 6/15
732390/732390 [==============================] - 44s 60us/step - loss: 1010953.3777 - mean_absolute_error: 674.9382 - val_loss: 954370.4462 - val_mean_absolute_error: 665.2246
Epoch 7/15
732390/732390 [==============================] - 44s 60us/step - loss: 998037.4484 - mean_absolute_error: 670.7878 - val_loss: 1012626.8049 - val_mean_absolute_error: 675.3255
Epoch 8/15
732390/732390 [==============================] - 45s 61us/step - loss: 974287.9476 - mean_absolute_error: 663.3675 - val_loss: 897542.6413 - val_mean_absolute_error: 648.6293
Epoch 9/15
732390/732390 [==============================] - 46s 63us/step - loss: 952037.8441 - mean_absolute_error: 656.2808 - val_loss: 936938.8103 - val_mean_absolute_error: 663.9085
Epoch 10/15
732390/732390 [==============================] - 44s 60us/step - loss: 939312.1822 - mean_absolute_error: 651.3833 - val_loss: 874905.7340 - val_mean_absolute_error: 631.0943
Epoch 11/15
732390/732390 [==============================] - 43s 58us/step - loss: 918155.9017 - mean_absolute_error: 645.2943 - val_loss: 898404.6137 - val_mean_absolute_error: 650.1163
Epoch 12/15
732390/732390 [==============================] - 43s 58us/step - loss: 905877.2954 - mean_absolute_error: 639.9159 - val_loss: 899350.7806 - val_mean_absolute_error: 635.1855
Epoch 13/15
732390/732390 [==============================] - 42s 57us/step - loss: 895967.5860 - mean_absolute_error: 636.0505 - val_loss: 905506.2252 - val_mean_absolute_error: 641.3708
Epoch 14/15
732390/732390 [==============================] - 42s 57us/step - loss: 885962.6494 - mean_absolute_error: 632.3153 - val_loss: 831220.6510 - val_mean_absolute_error: 616.1441
Epoch 15/15
732390/732390 [==============================] - 43s 58us/step - loss: 876729.0114 - mean_absolute_error: 629.1743 - val_loss: 888962.6661 - val_mean_absolute_error: 641.7786
203442/203442 [==============================] - 7s 35us/step
Metric loss : 882949.09
Metric mean_absolute_error : 637.69
Improving the Model 3
Data preprocessing
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import OneHotEncoder
from keras.models import Sequential
from keras.layers import Dense, Dropout
df = pd.read_csv("train.csv")
store = pd.read_csv("store.csv")
df_new = df.merge(store, on=["Store"], how="inner")
# We can extract all date properties from a datetime datatype
df_new['Date'] = pd.to_datetime(df_new['Date'], infer_datetime_format=True)
df_new["Month"] = df_new["Date"].dt.month
df_new["Quarter"] = df_new["Date"].dt.quarter
df_new["Year"] = df_new["Date"].dt.year
df_new["Day"] = df_new["Date"].dt.day
df_new["Week"] = df_new["Date"].dt.week
df_new["Season"] = np.where(df_new["Month"].isin([3, 4, 5]), "Spring",
np.where(df_new["Month"].isin([6, 7, 8]), "Summer",
np.where(df_new["Month"].isin([9, 10, 11]), "Fall",
np.where(df_new["Month"].isin([12, 1, 2]), "Winter", "None"))))
# Replace nulls with the mode
df_new["CompetitionDistance"] = df_new["CompetitionDistance"].fillna(df_new["CompetitionDistance"].mode()[0])
# Double check if we still see nulls for the column
df_new["CompetitionDistance"].isnull().sum() / df_new.shape[0] * 100
# Define a variable for each type of feature
target = ["Sales"]
numeric_columns = ["Customers", "Open", "Promo", "Promo2", "StateHoliday", "SchoolHoliday", "CompetitionDistance"]
categorical_columns = ["DayOfWeek", "Quarter", "Month", "Year", "StoreType", "Assortment", "Season"]
# Define a function that will intake the raw dataframe and the column name and return a one hot encoded DF
def create_ohe(df, col):
le = LabelEncoder()
a = le.fit_transform(df_new[col]).reshape(-1, 1)
ohe = OneHotEncoder(sparse=False)
column_names = [col + "_" + str(i) for i in le.classes_]
return (pd.DataFrame(ohe.fit_transform(a), columns=column_names))
# Since the above function converts the column, one at a time
# We create a loop to create the final dataset with all features
temp = df_new[numeric_columns]
for column in categorical_columns:
temp_df = create_ohe(df_new, column)
temp = pd.concat([temp, temp_df], axis=1)
temp["StateHoliday"] = np.where(temp["StateHoliday"] == '0', 0, 1)
# Create train and test dataset with an 80:20 split
x_train, x_test, y_train, y_test = train_test_split(temp, df_new[target], test_size=0.2, random_state=2018)
# Further divide training dataset into train and validation dataset with an 90:10 split
x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size=0.1, random_state=2018)
Training dataset
#Create Deep Neural Network Architecture
model = Sequential()
model.add(Dense(350,input_dim = 44,activation="relu"))
model.add(Dense(350,activation="relu"))
model.add(Dense(1,activation = "linear"))
model.compile(optimizer='adam',loss="mean_squared_error",metrics=["mean_absolute_error"])
model.fit(x_train,y_train, validation_data=(x_val,y_val), epochs=15,batch_size=64)
result = model.evaluate(x_test,y_test)
for i in range(len(model.metrics_names)):
print("Metric ",model.metrics_names[i],":", str(round(result[i],2)))
OUTPUT
Train on 732390 samples, validate on 81377 samples
Epoch 1/15
732390/732390 [==============================] - 50s 69us/step - loss: 1696801.1742 - mean_absolute_error: 852.3380 - val_loss: 1317084.3249 - val_mean_absolute_error: 789.5379
Epoch 2/15
732390/732390 [==============================] - 52s 71us/step - loss: 1155713.0256 - mean_absolute_error: 727.1582 - val_loss: 1062183.7259 - val_mean_absolute_error: 691.4468
Epoch 3/15
732390/732390 [==============================] - 53s 72us/step - loss: 1087896.8604 - mean_absolute_error: 704.8605 - val_loss: 1023563.1386 - val_mean_absolute_error: 688.5022
Epoch 4/15
732390/732390 [==============================] - 52s 72us/step - loss: 1058152.5636 - mean_absolute_error: 695.0822 - val_loss: 986651.8070 - val_mean_absolute_error: 670.2612
Epoch 5/15
732390/732390 [==============================] - 56s 76us/step - loss: 1032482.6717 - mean_absolute_error: 686.3516 - val_loss: 1067894.5107 - val_mean_absolute_error: 694.0162
Epoch 6/15
732390/732390 [==============================] - 52s 72us/step - loss: 1011641.7031 - mean_absolute_error: 679.8068 - val_loss: 969257.7984 - val_mean_absolute_error: 676.1887
Epoch 7/15
732390/732390 [==============================] - 53s 72us/step - loss: 992370.3198 - mean_absolute_error: 673.6577 - val_loss: 1004949.4962 - val_mean_absolute_error: 700.5331
Epoch 8/15
732390/732390 [==============================] - 53s 73us/step - loss: 974573.7028 - mean_absolute_error: 667.2528 - val_loss: 937751.6322 - val_mean_absolute_error: 651.9460
Epoch 9/15
732390/732390 [==============================] - 54s 74us/step - loss: 958369.8043 - mean_absolute_error: 661.7424 - val_loss: 990676.9595 - val_mean_absolute_error: 684.4022
Epoch 10/15
732390/732390 [==============================] - 54s 74us/step - loss: 943994.1679 - mean_absolute_error: 657.4421 - val_loss: 917064.5527 - val_mean_absolute_error: 658.8234
Epoch 11/15
732390/732390 [==============================] - 54s 74us/step - loss: 931808.3522 - mean_absolute_error: 652.8717 - val_loss: 941671.1141 - val_mean_absolute_error: 667.7861
Epoch 12/15
732390/732390 [==============================] - 54s 74us/step - loss: 918254.2089 - mean_absolute_error: 647.1999 - val_loss: 1010805.0772 - val_mean_absolute_error: 681.9616
Epoch 13/15
732390/732390 [==============================] - 52s 71us/step - loss: 908330.2797 - mean_absolute_error: 643.4899 - val_loss: 898364.6416 - val_mean_absolute_error: 644.6445
Epoch 14/15
732390/732390 [==============================] - 57s 78us/step - loss: 894027.9716 - mean_absolute_error: 638.6149 - val_loss: 860392.6973 - val_mean_absolute_error: 633.0040
Epoch 15/15
732390/732390 [==============================] - 58s 79us/step - loss: 884101.0426 - mean_absolute_error: 634.8977 - val_loss: 850341.6795 - val_mean_absolute_error: 627.0114
203442/203442 [==============================] - 9s 44us/step
Metric loss : 854548.0
Metric mean_absolute_error : 624.18
Plotting the Loss Metric Across Epochs
Data preprocessing
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import OneHotEncoder
df = pd.read_csv("train.csv")
store = pd.read_csv("store.csv")
df_new = df.merge(store, on=["Store"], how="inner")
# We can extract all date properties from a datetime datatype
df_new['Date'] = pd.to_datetime(df_new['Date'], infer_datetime_format=True)
df_new["Month"] = df_new["Date"].dt.month
df_new["Quarter"] = df_new["Date"].dt.quarter
df_new["Year"] = df_new["Date"].dt.year
df_new["Day"] = df_new["Date"].dt.day
df_new["Week"] = df_new["Date"].dt.week
df_new["Season"] = np.where(df_new["Month"].isin([3, 4, 5]), "Spring",
np.where(df_new["Month"].isin([6, 7, 8]), "Summer",
np.where(df_new["Month"].isin([9, 10, 11]), "Fall",
np.where(df_new["Month"].isin([12, 1, 2]), "Winter", "None"))))
# Replace nulls with the mode
df_new["CompetitionDistance"] = df_new["CompetitionDistance"].fillna(df_new["CompetitionDistance"].mode()[0])
# Double check if we still see nulls for the column
df_new["CompetitionDistance"].isnull().sum() / df_new.shape[0] * 100
# Define a variable for each type of feature
target = ["Sales"]
numeric_columns = ["Customers", "Open", "Promo", "Promo2", "StateHoliday", "SchoolHoliday", "CompetitionDistance"]
categorical_columns = ["DayOfWeek", "Quarter", "Month", "Year", "StoreType", "Assortment", "Season"]
# Define a function that will intake the raw dataframe and the column name and return a one hot encoded DF
def create_ohe(df, col):
le = LabelEncoder()
a = le.fit_transform(df_new[col]).reshape(-1, 1)
ohe = OneHotEncoder(sparse=False)
column_names = [col + "_" + str(i) for i in le.classes_]
return (pd.DataFrame(ohe.fit_transform(a), columns=column_names))
# Since the above function converts the column, one at a time
# We create a loop to create the final dataset with all features
temp = df_new[numeric_columns]
for column in categorical_columns:
temp_df = create_ohe(df_new, column)
temp = pd.concat([temp, temp_df], axis=1)
temp["StateHoliday"] = np.where(temp["StateHoliday"] == '0', 0, 1)
# Create train and test dataset with an 80:20 split
x_train, x_test, y_train, y_test = train_test_split(temp, df_new[target], test_size=0.2, random_state=2018)
# Further divide training dataset into train and validation dataset with an 90:10 split
x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size=0.1, random_state=2018)
Training dataset
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.callbacks import History
#Create Deep Neural Network Architecture
history = History()
model = Sequential()
model.add(Dense(350,input_dim = 44,activation="relu"))
model.add(Dense(350,activation="relu"))
model.add(Dense(350,activation="relu"))
model.add(Dense(350,activation="relu"))
model.add(Dense(350,activation="relu"))
model.add(Dense(1,activation = "linear"))
model.compile(optimizer='adam',loss="mean_squared_error",metrics=["mean_absolute_error"])
model.fit(x_train,y_train, validation_data=(x_val,y_val), epochs=15,batch_size=64,callbacks=[history])
result = model.evaluate(x_test,y_test)
for i in range(len(model.metrics_names)):
print("Metric ",model.metrics_names[i],":",str(round(result [i],2)))
Visualization for metric
import matplotlib.pyplot as plt
%matplotlib inline
# visualization for metric
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title("Model's Training & Validation loss across epochs")
plt.ylabel('Loss')
plt.xlabel('Epochs')
plt.legend(['Train', 'Validation'], loc='upper right')
plt.show()
OUTPUT
Train on 732390 samples, validate on 81377 samples
Epoch 1/15
732390/732390 [==============================] - 142s 195us/step - loss: 1633522.8718 - mean_absolute_error: 838.6379 - val_loss: 1217993.0883 - val_mean_absolute_error: 732.2864
Epoch 2/15
732390/732390 [==============================] - 146s 200us/step - loss: 1179818.2841 - mean_absolute_error: 726.9929 - val_loss: 1088124.4735 - val_mean_absolute_error: 710.5632
Epoch 3/15
732390/732390 [==============================] - 145s 198us/step - loss: 1110881.7459 - mean_absolute_error: 705.6384 - val_loss: 1019603.4330 - val_mean_absolute_error: 679.5359
Epoch 4/15
732390/732390 [==============================] - 148s 202us/step - loss: 1073248.9016 - mean_absolute_error: 693.0075 - val_loss: 1034116.4268 - val_mean_absolute_error: 677.2357
Epoch 5/15
732390/732390 [==============================] - 144s 197us/step - loss: 1042131.6708 - mean_absolute_error: 683.3028 - val_loss: 971867.9645 - val_mean_absolute_error: 674.6207
Epoch 6/15
732390/732390 [==============================] - 134s 182us/step - loss: 1022145.1901 - mean_absolute_error: 677.9384 - val_loss: 1123250.9915 - val_mean_absolute_error: 730.9583
Epoch 7/15
732390/732390 [==============================] - 131s 178us/step - loss: 998527.2257 - mean_absolute_error: 671.0589 - val_loss: 950288.8741 - val_mean_absolute_error: 666.2436
Epoch 8/15
732390/732390 [==============================] - 133s 182us/step - loss: 979936.5206 - mean_absolute_error: 665.6808 - val_loss: 913612.6051 - val_mean_absolute_error: 647.3661
Epoch 9/15
732390/732390 [==============================] - 136s 186us/step - loss: 959011.7640 - mean_absolute_error: 658.8191 - val_loss: 877278.1993 - val_mean_absolute_error: 633.0947
Epoch 10/15
732390/732390 [==============================] - 138s 189us/step - loss: 944423.2573 - mean_absolute_error: 653.6127 - val_loss: 940223.5264 - val_mean_absolute_error: 650.9949
Epoch 11/15
732390/732390 [==============================] - 134s 183us/step - loss: 921521.4840 - mean_absolute_error: 646.2910 - val_loss: 922973.2254 - val_mean_absolute_error: 659.6578
Epoch 12/15
732390/732390 [==============================] - 136s 185us/step - loss: 908583.9544 - mean_absolute_error: 641.5706 - val_loss: 846290.7568 - val_mean_absolute_error: 625.4439
Epoch 13/15
732390/732390 [==============================] - 133s 182us/step - loss: 896627.8529 - mean_absolute_error: 637.2651 - val_loss: 830765.3276 - val_mean_absolute_error: 619.4513
Epoch 14/15
732390/732390 [==============================] - 15480s 21ms/step - loss: 881627.1912 - mean_absolute_error: 632.3052 - val_loss: 878429.4993 - val_mean_absolute_error: 630.2251
Epoch 15/15
732390/732390 [==============================] - 178s 243us/step - loss: 868846.1121 - mean_absolute_error: 627.2891 - val_loss: 859102.4735 - val_mean_absolute_error: 631.2212
203442/203442 [==============================] - 14s 69us/step
Metric loss : 855410.89
Metric mean_absolute_error : 626.6
Deep Neural Networks for Supervised Learning: Classification
Exploring the Data
OUTPUT
Data Engineering
OUTPUT
Defining Model Baseline Accuracy
OUTPUT
Designing the DNN for Classification
OUTPUT
Revisiting the Data
OUTPUT
DNNs for Classification with Improved Data
OUTPUT
Tuning and Deploying Deep Neural Networks
What Is Regularization
OUTPUT
Hyperparameter Tuning
Hyperparameters in DL
- Number of Neurons in a Layer
- Number of Layers
- Number of Epochs
- Weight Initialization
- Batch Size
- Learning Rate
- Activation Function
- Optimization
OUTPUT
Approaches for Hyperparameter Tuning
- Manual Search
- Grid Search
- Random Search
- Bayesian Optimization
Model Deployment
Saving Models to Memory
# Import required packages
import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from keras.datasets import boston_housing
#Download the data using Keras; this will need an active internet connection
(x_train, y_train), (x_test, y_test) = boston_housing.load_data()
#Extract the last 100 rows from the training data to create the validation datasets.
x_val = x_train[300:,] # last 300 row
y_val = y_train[300:,] # last 300 row
#Define the model architecture
model = Sequential()
model.add(Dense(13, input_dim=13, kernel_initializer='normal', activation='relu'))
model.add(Dense(6, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_absolute_percentage_error'])
#Train the model
model.fit(x_train, y_train, batch_size=32, epochs=3, validation_data=(x_val,y_val))
#Save the model
model.save('dnn_model.h5')
Loading Models from Memory
# Import required packages
from keras.datasets import boston_housing
from keras.models import load_model
#Download the data using Keras; this will need an active internet connection
(x_train, y_train), (x_test, y_test) = boston_housing.load_data()
#Load the model
model = load_model('dnn_model.h5')
results = model.evaluate(x_test, y_test)
for i in range(len(model.metrics_names)):
print(model.metrics_names[i]," : ", results[i])
OUTPUT
32/102 [========>.....................] - ETA: 0s
102/102 [==============================] - 0s 372us/step
loss : 369.7161434397978
mean_absolute_percentage_error : 65.79050146364698
The Path Ahead
What’s Next for DL Expertise
CNN
#Importing the necessary packages
import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
#Importing the CNN related layers as described in Chapter 2
from keras.layers.convolutional import Conv2D, MaxPooling2D
from keras.utils import np_utils
#Loading data from Keras datasets
(x_train, y_train), (x_test, y_test) = mnist.load_data()
#Defining the height and weight and number of samples
#Each Image is a 28 x 28 with 1 channel matrix
training_samples, height, width = x_train.shape
testing_samples,_,_ = x_test.shape
#We now have to engineer the image data into the right form
#For CNN, we would need the data in Height x Width X Channels form Since the image is in grayscale, we will use channel = 1
channel =1
x_train = x_train.reshape(training_samples, height, width,channel).astype('float32')
x_test = x_test.reshape(testing_samples, height, width, channel).astype('float32')
#To improve the training process, we would need to standardize or normalize the values We can achieve this using a simple divide by 256 for all values
x_train = x_train/255
x_test =x_test/255
#Total number of digits =10
target_classes = 10
# numbers 0-9, so ten classes
n_classes = 10
# convert integer labels into one-hot vectors
y_train = np_utils.to_categorical(y_train, n_classes)
y_test = np_utils.to_categorical(y_test, n_classes)
#Designing the CNN Model
model = Sequential()
model.add(Conv2D(64, (5, 5), input_shape=(height,width ,1), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(n_classes, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=10, batch_size=200)
OUTPUT
Train on 60000 samples, validate on 10000 samples
Epoch 1/10
60000/60000 [==============================] - 59s 991us/step - loss: 0.2365 - acc: 0.9305 - val_loss: 0.0592 - val_acc: 0.9811
Epoch 2/10
60000/60000 [==============================] - 65s 1ms/step - loss: 0.0637 - acc: 0.9805 - val_loss: 0.0372 - val_acc: 0.9878
Epoch 3/10
60000/60000 [==============================] - 63s 1ms/step - loss: 0.0454 - acc: 0.9859 - val_loss: 0.0378 - val_acc: 0.9869
Epoch 4/10
60000/60000 [==============================] - 65s 1ms/step - loss: 0.0375 - acc: 0.9880 - val_loss: 0.0306 - val_acc: 0.9898
Epoch 5/10
60000/60000 [==============================] - 67s 1ms/step - loss: 0.0310 - acc: 0.9903 - val_loss: 0.0280 - val_acc: 0.9910
Epoch 6/10
60000/60000 [==============================] - 66s 1ms/step - loss: 0.0251 - acc: 0.9917 - val_loss: 0.0279 - val_acc: 0.9915
Epoch 7/10
60000/60000 [==============================] - 67s 1ms/step - loss: 0.0228 - acc: 0.9924 - val_loss: 0.0238 - val_acc: 0.9927
Epoch 8/10
60000/60000 [==============================] - 63s 1ms/step - loss: 0.0192 - acc: 0.9934 - val_loss: 0.0253 - val_acc: 0.9916
Epoch 9/10
60000/60000 [==============================] - 64s 1ms/step - loss: 0.0170 - acc: 0.9948 - val_loss: 0.0272 - val_acc: 0.9924
Epoch 10/10
60000/60000 [==============================] - 67s 1ms/step - loss: 0.0157 - acc: 0.9949 - val_loss: 0.0213 - val_acc: 0.9927
Testing the Model Performance
metrics = model.evaluate(x_test, y_test, verbose=0)
for i in range(0,len(model.metrics_names)):
print(str(model.metrics_names[i])+" = "+str(metrics[i]))
OUTPUT
loss = 0.021252375301908977
acc = 0.9927
RNN
OUTPUT
List of posts followed by this article
Reference