title: Understanding the compare command in Matlab
body: <p>I am very puzzled by the following Matlab observation.</p>
  
  <p>In my problem I am trying to estimate an ARX/ARMAX model out of  discrete sampled input-output data. I am following Matlab's <a href="http://ch.mathworks.com/help/ident/gs/identify-linear-models-using-the-command-line.html" rel="nofollow">guide to identify linear models in the command line</a> (using the System Identification toolbox)</p>
  
  <ul>
  <li>My dataset are 980 input-output pairs (experiments), with a sampling time of <code>Ts = 10</code> seconds (with 2 inputs, 1 output). I split the data <code>my_data</code> into two subsets of 490 experiments: <code>experimental</code> and <code>validation</code>.</li>
  <li>Running <code>delayest(my_data)</code> reveals that there is no delay in any of the inputs. Therefore <code>nk1=0</code> and <code>nk2=0</code>.</li>
  <li>Identifying an order with <code>selstruc</code> and <code>arxstruc</code> gives me a choice for the orders <code>na</code>,<code>nb1</code> and <code>nb2</code>.</li>
  <li>I set <code>opt=armaxOptions('InitialCondition','z')</code> to have zero initial condition for the system (I don't Matlab to treat it as an estimation variable).</li>
  </ul>
  
  <p>With all of the above into consideration I run</p>
  
  <pre><code>my_model = arx(experimental,[6,[2,5],[0,0]],opt)
  </code></pre>
  
  <p>which gives the following model in the command line (<a href="http://pastebin.com/1SJgaSt8" rel="nofollow">link on pastebin</a>).</p>
  
  <p>From the link above, if one scrolls down, we can see that the fit to <code>experimental</code> data is fairly well.</p>
  
  <hr>
  
  <p><strong>And here comes my question</strong>:</p>
  
  <p>If I now want to validate the data with the <code>validation</code> subset, running a comparison with the <a href="http://ch.mathworks.com/help/ident/ref/compare.html" rel="nofollow"><code>compare</code></a> command:</p>
  
  <pre><code>% the model had zero initial condition as well
  opt=compareOptions('InitialCondition','z'); 
  [~,fit_vector,~]=compare(validation,my_model,opt);
  disp(num2str(cell2mat(fit_vector)))
  </code></pre>
  
  <p>and this is what is intriguing, now the displayed number is <code>-109.626 %</code> fit.</p>
  
  <ul>
  <li><p>Does anyone know the exact behaviour of this command, and why is there such a discrepancy between the fit when generating the model with <code>arx</code> (in the pastebin link above) and that of using the <code>compare</code> command?</p></li>
  <li><p>I find it very strange because even with different datasets <code>experimental</code> and <code>validation</code>, they both come from the same family of inputs (pulses of different duration and amplitude for input 1; input 2 is an exogenous input).</p></li>
  </ul>