If a pixel with a high intensity lands in the red area, the evidence is high that the handwritten digit is zero

Conversely, if a pixel with a high intensity lands in the blue area, the evidence is low that the handwritten digit is zero

https://www.tensorflow.org/versions/r1.1/get_started/mnist/beginners

https://www.oreilly.com/learning/not-another-mnist-tutorial-with-tensorflow

Introduction

We first build a computational (or dataflow) graph. Then, we create a Tensorflow session to run the graph.

Tensorflow computations involve tensors; think of tensors as multidimensional arrays.

Images

Each 28 x 28 image is flattened into a list with 784 elements.

Once flattened, the training images are stored in a tensor x, with shape of [none, 784]. The first index is the number of training images ("none" means that we can use an arbitrary number of training images).

Labels

Each training image is associated with a label.

So for an image that displays the digit 5, the label is [ 0,0,0,0,0,1,0,0,0,0]. This is known as a one-hot encoding.

All the labels are stored in a tensor y_ with a shape of [none, 10].

Training

The neural network is trained via multinomial logistic regression (also known as softmax).

Step 1

Calculate the evidence that each image is in a selected class. Do this by performing a weighted sum of the pixel intensity for the flattened image.

where

Step 2

Normalize the evidence into a vector of probabilities with softmax.

Step 3

For each image, calculate the cross-entropy of the vector of predicted probabilities and the actual probabilities (i.e the labels)

where

The smaller the cross entropy, the better the prediction.

Step 4

The mean cross-entropy across all training images is then minimized to find the optimum values of W and b

Testing

For each test image, we will generate 10 ordered probabilities that sum to 1. The location of the highest probability is the predicted value of the digit.

Miscellaneous

This application consists of

in a single zip file. The images are stored in folders; the folders should be extracted to the location as this worksheet.

restart:
with(DeepLearning):
with(DocumentTools):
with(DocumentTools:-Layout):
with(ImageTools):

LEARNING_RATE := 0.01:
TRAIN_STEPS   := 40:

N := 22:

L := 10:

T := 50:

path := "C:/Users/Wilfried/Documents/Maple/Examples/ML/":
for j from 0 to L - 1 do
    images[j] := [seq(Import(cat(path, j, "/", j, " (", i, ").PNG")), i = 1 .. N)];
end do:

for j from 0 to L - 1 do
   labels[j] := ListTools:-Rotate~([[1,0,0,0,0,0,0,0,0,0]$N],-j)[]:
end do:

Embed([seq(images[i-1], i = 1 .. L)]);

x_train := convert~([seq(images[i - 1][], i = 1 .. L)], list):

y_train := [seq(labels[i - 1], i = 1 .. L)]:

SetEagerExecution(false):
x  := Placeholder(float[4], [none, 784]):
y_ := Placeholder(float[4], [none, L]):

W := Variable(Array(1 .. 784, 1 .. L), datatype = float[4]):
b := Variable(Array(1 .. L), datatype = float[4]):

y := SoftMax(x.W + b):

cross_entropy := ReduceMean(-ReduceSum(y_ * log(y), reduction_indicies = [1])):

sess := GetDefaultSession():

init := VariablesInitializer():
sess:-Run(init):

optimizer := Optimizer(GradientDescent(LEARNING_RATE)):
training  := optimizer:-Minimize(cross_entropy):

for i from 1 to TRAIN_STEPS do

   sess:-Run(training, {x in x_train, y_ in y_train}):

   if i mod 200 = 0 then
      print(cat("loss = ", sess:-Run(cross_entropy, {x in x_train, y_ in y_train})));    
   end if:

end do:

i_rand := combinat:-randperm([seq(i, i = 1 .. 100)]);

path:= "C:/Users/Wilfried/Documents/Maple/Examples/ML/test_images":
x_test_images := [seq(Import(cat(path,"/","test (", i, ").png")), i in i_rand[1 .. T])]:
x_train:= convert~(x_test_images, list):

pred := sess:-Run(y, {x in x_train})

predList := seq( max[index]( pred[i, ..] ) - 1, i = 1 .. T )

:
Val:=Vector(10,0):
Val_mean:=Vector(10,0):
for k from 1 to 10 do:
  L1:=[]:
 for i from 1 to 50 do:
  if predList[i]=k-1 then L1:=[op(L1),L[i]] end if:
 end do:
 Val(k):=evalf(L1,3):
 Val_mean(k):=Statistics:-Mean(Array(L1)):
end do:

Val,Val_mean

Embed(x_test_images[1])

pred[1, ..]

add(i, i in pred[1, ..])

maxProbInd := max[index](pred[1, ..])

maxProbInd - 1

T1 := Table(Row(seq(predList[k],k = 1.. 25)),Row( seq(predList[k],k = 26 .. 50 ))
          ):
InsertContent(Worksheet(T1)):