title: web application receiving millions of requests and leads to generating millions of row inserts per 30 seconds in SQL Server 2008
body: <p>I am currently addressing a situation where our web application receives at least a Million requests per 30 seconds. So these requests will lead to generating 3-5 Million row inserts between 5 tables. This is pretty heavy load to handle. Currently we are using multi threading to handle this situation (which is a bit faster but unable to get a better CPU throughput). However the load will definitely increase in future and we will have to account for that too. After 6 months from now we are looking at double the load size we are currently receiving and I am currently looking at a possible new solution that is scalable and should be easy enough to accommodate any further increase to this load.
          Currently with multi threading we are making the whole debugging scenario quite complicated and sometimes we are having problem with tracing issues.</p>
  
  <p>FYI we are already utilizing the SQL Builk Insert/Copy that is mentioned in this previous post</p>
  
  <p><a href="https://stackoverflow.com/questions/3974275/sql-server-2008-performance-tuning-features-for-insert-large-amount-of-data/3974606">Sql server 2008 - performance tuning features for insert large amount of data</a>  </p>
  
  <p>However I am looking for a more capable solution (which I think there should be one) that will address this situation. </p>
  
  <p><strong>Note:</strong> I am not looking for any code snippets or code examples. I am just looking for a big picture of a concept that I could possibly use and I am sure that I can take that further to an elegant solution :)</p>
  
  <p>Also the solution should have a better utilization of the threads and processes. And I do not want my threads/processes to even wait to execute something because of some other resource.</p>
  
  <p>Any suggestions will be deeply appreciated.</p>
  
  <p><strong>Update</strong>: Not every request will lead to an insert...however most of them will lead to some sql operation. The appliciation performs different types of transactions and these will lead to a lot of bulk sql operations. I am more concerned towards inserts and updates.
  and these operations need not be real time there can be a bit lag...however processing them real time will be much helpful.</p>