title: Fastest way to process Millions of Rows in SQL Server for a Chart
body: <p>We are logging realtime data every second to a SQL Server database and we want to generate charts from 10 Million rows or more. At the moment we use something like the code below. The goal is to get at least 1000-2000 values to pass into the chart. </p>
  
  <p>In the query below, we take an avg of every next n'th rows depending on the count of data we pick out from the <code>LargeTable</code>. This works fine up to 200.000 selected rows, but then it is way too slow.</p>
  
  <pre><code>SELECT 
      AVG(X),
      AVG(Y)
  FROM 
      (SELECT 
           X, Y,
           (Id / @AvgCount) AS [Group]
       FROM 
           [LargeTable]
       WHERE 
           Timestmp &gt; @From
           AND Timestmp &lt; @Till) j
  GROUP BY
      [Group]
  ORDER BY 
      X;
  </code></pre>
  
  <p>Now we tried to select out only every n'th row from <code>LargeTable</code> and then make an average of this data to get more performance, but it takes nearly the same time.</p>
  
  <pre><code>SELECT 
      X, Y
  FROM 
      (SELECT 
           X, Y,
           ROW_NUMBER() OVER (ORDER BY Id) AS rownr
       FROM 
           LargeTable
       WHERE 
           Timestmp &gt;= @From
           AND Timestmp &lt;= @Till) a
  WHERE 
      a.rownr % (@count / 10000) = 0;
  </code></pre>
  
  <p>It is only pseudo code! We have indexes on all relevant columns.</p>
  
  <p>Are there better and faster ways to get chart data?</p>