I have written my own basic convolutional neural network in Java as a learning exercise. I am using it to analyze the MIT CBCL face database image set. They are a set of 19x19 pixel greyscale images.

Network specifications are:

Single Convolution Layer with 1 filter: Filter Size: 4x4. Stride Size: 1

Single Pooling Layer 2x2 Max Pooling

3 layer MLP(input, 1 hidden and output) input = 64 neurons hidden = 15 neurons output = 2 neurons learning rate = 0.1

Now I am getting reasonable accuracy(92.85%), but my issue is that it is being achieved at very different points in the epoch count across network runs:

Epochs  Training Accuracy   Test Accuracy   Validation Accuracy

Run 1 415 93.13 92.44 93.35 Run 2 515 92.44 93.18 92.84 Run 3 327 93.83 92.05 92.38

I am using the Java random class with the same seed for every run to initialize the kernel, the MLP weights and break the input data into 3 sets.(training is being done using the 33-33-33 method)

I am a loss as to what is causing this variation in epoch count to achieve the highest point in validation accuracy. Can anybody explain this?


Fixed. Was an issue with the random generator. In my class for the Neuron layer where I initialize the weights I get new doubles from the generator for each of the initial weight values, but I found a bug where I was re-initializing the random generator, which was of course causing different values.

  • 1
    $\begingroup$ If you feel inspired (and want the rep) it would be helpful if you could elaborate and accept your answer. $\endgroup$ – DukeZhou Oct 15 '18 at 16:51

Your Answer

