title: Performance hit when combining class member arrays to a single array
body: <p>I'm trying to make my code a bit faster and I'm trying to find out if I can gain some performance by better managing arrays stored in objects and stuff.</p>
  
  <p>So the basic idea behind that is that I tend to keep separate arrays for temporary and permanent states. This means that they have to be indexed separately all the time having to explicitly write the proper member name every time I want to use them. </p>
  
  <p>This is how a particular class with such arrays looks like: </p>
  
  <pre><code>class solution
  {
  public:
  
      //Costs
      float *cost_array;
      float *temp_cost_array;
      //Cost trend
      float *d_cost_array;
      float *temp_d_cost_array;
  
      ...
  
  }
  </code></pre>
  
  <p>Now, because of the fact that I have functions/methods that work on the temp or the permanent status depending on input arguments, these look like this:</p>
  
  <pre><code>void do_stuff(bool temp){
  if (temp)
      work_on(this-&gt;temp_cost_array);
  else
      work_on(this-&gt;cost_array);
  }
  </code></pre>
  
  <p>These are very simplistic examples of such branches. These arrays may be indexed separately here and there within the code. So exactly because of the fact that such stuff is all over the place I thought that this is yet another reason to combine everything so that I could get rid of that code branches as well.</p>
  
  <p>So I converted my class to:</p>
  
  <pre><code>class solution
  {
  public:
  
      //Costs
      float **cost_array;
      //Cost trend
      float **d_cost_array;
      ...
  
  }
  </code></pre>
  
  <p>Those double arrays are of size 2, with each element being a float* array. Those are dynamically allocated just once during object creation at the start of the program and deleted at the end of the program.</p>
  
  <p>So after that I also converted all the temp branches of my code like this:</p>
  
  <pre><code>void do_stuff(bool temp){
      work_on(this-&gt;cost_array[temp]);
  }
  </code></pre>
  
  <p>It looks WAY more elegant than before but for some reason performance got way worse than before (almost 2 times worse), and I seriously can't understand why that happened.</p>
  
  <p>So, as a first insight, I'd really love to hear from more experienced people, if my rationale behind that code optimization was valid or not. </p>
  
  <p>Could that extra indexing required to access each array introduce such major performance hit to overcome all the if branching and stuff? For sure it depends on how the whole thing works but the code is a beast and I've no idea how to properly profile that thing all-together. </p>
  
  <p>Thanks</p>
  
  <p>EDIT: </p>
  
  <p>Environment settings:
  Running on Windows 10, VS 2017, Full Optimization enabled (/Ox)</p>