title: what can you say about two exactly same neural networks after training on same dataset?
body: <p>Supposed you have two convolutional neural networks <a href="https://it.mathworks.com/help/nnet/examples/create-simple-deep-learning-network-for-classification.html" rel="nofollow noreferrer">implemented in matlab</a> and composed by these layers:</p>
  
  <ul>
  <li>imageInputLayer</li>
  <li>ConvolutionalLayer</li>
  <li>maxPoolinglayer</li>
  <li>relulayer</li>
  <li>softmaxlayer</li>
  <li>fullyconnectedlayer</li>
  <li>classification layer</li>
  </ul>
  
  <p>Both of these networks have exactly same architecture. 
  I apply the same method of training for 2 networks with same hyperparameters. 
  Both of these networks have exactly same weights in their corresponding layers.  </p>
  
  <p>That is, both of these networks are a replica of each other.  </p>
  
  <p>Both of these networks are trained using exactly same training set and validation set without shuffle.</p>
  
  <p>I am wondering:</p>
  
  <ul>
  <li>Will the scores (training error and validation error) and trained weights be different for both?</li>
  <li>Does it depend upon the method for training?</li>
  </ul>