Take the 2-minute tour ×

I'm working to proof basic neural network results and so far haven't been able to. I'm doing a feed-forward xor problem in encog and export the final weights and calculated output.

To proof I just have an excel sheet where I input the weights, then I1*W1+I2*W2 | I1*W3+I2*W4 to the hidden layer, then sigmoid activation for each, then H1*W5+H2*W6 then sigmoid again for the output.

So no bias, just a basic 2x2x1, but the output values I get once I plug the weights in are no where close to the expected output values that I receive with encog.

I have 8 output sets from encog to test against, but so far, I'm not comming up with the same conclusions. Any help would be appreciated.

Below is a sample output if that would be of any help. Thanks, Israel

Output Weights

61.11812639080170, -70.09419692460420, 2.58264325902522, 2.59015713019213, 1.16050691499417, 1.16295830927117

Output Values

0.01111771776254, 0.96929877340644, 0.96926035361899, 0.04443376315742

In excel, here's what I'm using for the sigmoid function: =1/(1+EXP(-1*(C3))), don't know if more would help since it's just addition and multiplication outside of sigmoid.

Here's Form1.cs:

using Encog.Engine.Network.Activation;
using Encog.ML.Data.Basic;
using Encog.Neural.Networks;
using Encog.Neural.Networks.Layers;
using Encog.Neural.Networks.Training.Propagation.Resilient;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace Encog_Visual
    public partial class Form1 : Form
        public Form1()

            double[][] XOR_Input = 
                new[] {0.0,0.0},
                new[] {1.0,0.0},
                new[] {0.0,1.0},
                new[] {1.0,1.0}

            double[][] XOR_Ideal =
                new[] {0.0},
                new[] {1.0},
                new[] {1.0},
                new[] {0.0}

            var trainingSet = new BasicMLDataSet(XOR_Input, XOR_Ideal);

            BasicNetwork network = CreateNetwork();

            var train = new ResilientPropagation(network, trainingSet);

            int epoch = 0;
                string result0 = String.Format("Iteration No :{0}, Error: {1}", epoch, train.Error);
                textBox1.AppendText(result0 + Environment.NewLine);
            } while (train.Error > 0.001);

            foreach (var item in trainingSet)
                var output = network.Compute(item.Input);
                string result1 = String.Format("Input : {0}, {1} Ideal : {2} Actual : {3}", item.Input[0], item.Input[1], item.Ideal[0], output[0]);
                textBox1.AppendText(result1 + Environment.NewLine + network.DumpWeights() + Environment.NewLine);


        private void Form1_Load(object sender, EventArgs e)


        private static BasicNetwork CreateNetwork()
            var network = new BasicNetwork();
            network.AddLayer(new BasicLayer(null, false, 2));
            network.AddLayer(new BasicLayer(new ActivationSigmoid(), false, 2));
            network.AddLayer(new BasicLayer(new ActivationSigmoid(), false, 1));
            return network;

        private void textBox2_TextChanged(object sender, EventArgs e)


        private void textBox1_TextChanged(object sender, EventArgs e)



share|improve this question
What code do you have at the moment? It is hard to understand what the problem might be without seeing what you're doing. –  Simon Martin Nov 12 '13 at 15:32
Added my form1.cs to the initial post, and what I'm using for sigmoid in excel, let me know if anymore info would be useful, thanks! –  Israel Nov 12 '13 at 16:04
Here's another example, about the closest thing I've found as a proof: generation5.org/content/2001/xornet.asp This network is slightly different since it has 1 bias per neuron, but in excel I redid my calculations to include the bias. Up to the point of the first output (prior to training) I have the same output: 0.367610, but once I plug in his final weights, I don't receive anywhere close to his output. Pretty much the same wall I'm running into with my initial problem. –  Israel Nov 13 '13 at 4:02

2 Answers 2

I did stumble on this, and got here and I think there's an answer left to be detailed:

Yes, indeed, it takes time to figure out to which layer belongs each weight, given that layers seem to be numbered with different criteria across the Encog Framework.

For example, if you write some code like the following, where networkName is just anything you want to call yours (e.g.: "XOR_one"). Then you may call this function from your main public Form1(), after the network training loop, by adding a single line: saveNetwork("XOR_one"); and then...

public DirectoryInfo dataDirRoot;
public FileInfo dataFileRoot;
public StreamWriter fileWriteSW;

public bool saveNetwork(string networkName)
        // File data initialized
        if (dataDirRoot == null) dataDirRoot = new DirectoryInfo(Application.StartupPath + "\\Data");
        if (!dataDirRoot.Exists) dataDirRoot.Create();

        dataFileRoot = new FileInfo(dataDirRoot + "\\" + networkName + ".weights.txt");
        fileWriteSW = new StreamWriter(dataFileRoot.FullName, false, Encoding.Default);

        // (A) Write down weights from left to right layers, meaning input first and output last.
        // ...within each layer, weights are ordered up-down, always, in all three methods.
        for (int j = 0; j < network.LayerCount-1; j++)
            for (int l = 0; l < network.GetLayerNeuronCount(j + 1); l++)
                for (int k = 0; k < network.GetLayerTotalNeuronCount(j); k++)
                    fileWriteSW.Write(network.GetWeight(j, k, l).ToString("r") + ", ");

        // (B) Write down weights from left to right layers, output first, input last
        double[] auxDouble = new double[network.EncodedArrayLength()];  

        for (int j = 0; j < network.EncodedArrayLength(); j++)
            fileWriteSW.Write(auxDouble[j] + "\r\n");

        // (C) Write down network structure
        // ...you will find that "weights" in the same order as with "DumpWeights()"
        dataFileRoot = new FileInfo(dataDirRoot + networkName + ".encog.txt");
        Encog.Persist.EncogDirectoryPersistence.SaveObject(dataFileRoot, network);
    catch (Exception e)
        MessageBox.Show("Error: " + e.Message);
        return false;
    return true;

Important: It is really difficult to train a XOR network without biasing the hidden layer, so the results I am showing have two more weights than your example. This can be achieved by changing one line in your code:

... network.AddLayer(new BasicLayer(null, false, 2));

to network.AddLayer(new BasicLayer(null, true, 2));

...in order to give a weight input to the hidden layer. Neurons in the hidden layer will have three weights each. One coming from neuron input 1, another from neuron input 2 and a third one coming from the Bias Neuron (which is listed as a "third neuron" in the Input layer, having its value fixed to 1.0).

So: the tricky thing here is which layer is to be named "Layer 0".

In case (A), Layer 0 is the Input layer, the first from the left, and weights are dumped from the first hidden layer (since Input has no weights), neuron 0 to 1, and then output layer, neuron 0.

But in case (B) and (C) and "DumpWeights()", Layer 0 is the first from the right, meaning the output layer, and weights are dumped right to left layers and up-down within each layer.

Always, within each layer, weights are dumped in order, Neuron 0 to n, and within each neuron, weight coming from the upper neuron on the layer from the left down to the last neuron or bias if it exists on the left layer.

The output weights results are something like this:

Case (A)
-3.61545321823196, -2.7522256580709645, 3.509680820551957, 
-7.2744584719809806, -6.05682131778526, 7.6850785784618676, 
-35.025902985103983, 31.763309640942925, 

Case (B) 

Lets see then:

**Output Layer** (being it called 0 or N... you decide, I prefer N)
**Neuron 0** (the only one there)
weight 2,0,0 = -35.025902985104 (where 2 is layer, 0 is first neuron in hidden layer and 0 is output neuron)
weight 2,1,0 = 31.7633096409429

**Hidden Layer** (I choose 1)
**Neuron 0** (first one)
weight 1,0,0 = -3.61545321823196 (where 1 is layer, 0 is first neuron in input layer and 0 is this neuron)
weight 1,1,0 = -2.75222565807096
weight 1,2,0 = 3.50968082055196
**Neuron 1** (last one)
weight 1,0,1 = -7.27445847198098 
weight 1,1,1 = -6.05682131778526
weight 1,2,1 = 7.68507857846187 (where 1 is layer, 2 is bias in input layer and 1 is this neuron)

Note that: your example in the question was the result of DumpWeights(): 61.11812639080170, -70.09419692460420, 2.58264325902522, 2.59015713019213, 1.16050691499417, 1.16295830927117

It corresponds to Case (B), only comma separated. The fist two numbers belong to the output neuron and the latter belong, third and fourth to the 1st neuron, hidden layer and fifth and sixth to the 2nd neuron, hidden layer.

I am including here the CSV for an Excel example using your data:

DumpWeights() = ,,,,,,,,,,
"61.11812639080170, -70.09419692460420, 2.58264325902522, 2.59015713019213, 1.16050691499417, 1.16295830927117",,,,,,,,,,

That should do it :)

(for the record, I have used Encog v3.2.0)

share|improve this answer

In case anyone stumbles on this in the future.

The weight output by encog is layerN Neuron:0,1,n,bias, down to layer0 neuron:0,1,n,bias

Figuring out the proper output to feed back into the function, I was able to proof it properly according to the given output.

share|improve this answer

Your Answer


