title: WCF and Socket Data Processing - Anomaly Data Notification Module (ASP.NET MVC 3 + EF 4.0 + LINQ + WCF 4.0 + Sockets + SQL Server 2008 R2 + Azure)
body: <h3>Business scenario:</h3>
  <p>Imagine that there is <strong>Construction</strong>. Construction can be a new building, bridge etc.
  On the <strong>Construction Yard</strong> may occur variety of physical and the environment changes.
  These changes might be for example changes of temperature.</p>
  <p>We built application for our client that gather different type of data from construction yard, later on to:</p>
  <ol>
  <li>Show charts of changes in time</li>
  <li>Provide insights, alerts and notification for construction manager and other decision makers</li>
  </ol>
  <h3>Current status:</h3>
  <p>Application is built and working.</p>
  <ol>
  <li>Ad. 1 - All data is gathered and visualized with charts</li>
  <li>Ad. 2 - <strong>We are still strangling with proper design for this Module</strong></li>
  </ol>
  <h3>Issue:</h3>
  <blockquote>
  <p>How to properly design Anomaly Data Notification Module that gather data using WCF Service and Socket
  Process?</p>
  </blockquote>
  <h3>Overall Architecture Design:</h3>
  <p><img src="https://i.stack.imgur.com/oHgd6.png" alt="Architecture Design" /></p>
  <p>We have 4 main pieces of our puzzle:</p>
  <ol>
  <li>Backend database - SQL Server 2008 (Azure SQL)</li>
  <li>Frontend web application - ASP.NET MVC 3 (WebRole for Azure).</li>
  <li>WCF 4.0 Service that is called by ZigBee devices that send us data (WebRole for Azure).</li>
  <li>Windows Process that with configured frequency call Rfid Readers using Socket IP (WorkerRole for Azure).</li>
  </ol>
  <p>In database we have 2 separate tables for Rfid Readings and for ZigBee Readings.
  The table that stores results is deliberately de-normalized for performance: faster querying and grouping by multiple indexes. We are using Entity Framework and Linq with Projection for data access in Frontend application and it works well.</p>
  <h3>Anomaly Data Notification Module:</h3>
  <p>Our notification module functional requirements are:</p>
  <ol>
  <li>Send sms and email for notification subscribers of specific
  construction yard when:</li>
  <li>Value of measured data is above threshold x times of occurrence</li>
  <li>Parameters are: Min Value, Max Value, Threshold, Repetition Count, Repetition Time Window, Sensor type</li>
  </ol>
  <p>For example we added notification / alert for specific construction yard. This alert is configured:</p>
  <ul>
  <li>Min Value = -5</li>
  <li>Max Value = 40</li>
  <li>Threshold = 5</li>
  <li>Repetition Count = 5</li>
  <li>Repetition Time Window = 5min</li>
  <li>Sensor Type = Temperature</li>
  </ul>
  <p>So now when ZigBee WCF or Rfid Windows Process gets data, notification / alert can be triggered.
  For example:</p>
  <ul>
  <li>Date: 2012-08-19 10:00 Value: 30 OK</li>
  <li>Date: 2012-08-19 10:01 Value: 28 OK</li>
  <li>Date: 2012-08-19 10:02 Value: 40 OK (still below Max + Threshold)</li>
  <li>Date: 2012-08-19 10:03 Value: 41 OK (still below Max + Threshold)</li>
  <li>Date: 2012-08-19 10:04 Value: 39 OK</li>
  <li>Date: 2012-08-19 10:05 Value: 46 OK (above Threshold, Repetition Count 1)</li>
  <li>Date: 2012-08-19 10:06 Value: 39 OK</li>
  <li>Date: 2012-08-19 10:07 Value: 38 OK</li>
  <li>Date: 2012-08-19 10:08 Value: 39 OK</li>
  <li>Date: 2012-08-19 10:09 Value: 38 OK</li>
  <li>Date: 2012-08-19 10:10 Value: 39 OK</li>
  <li>Date: 2012-08-19 10:11 Value: 38 OK (Last Anomaly Occurrence &gt; Repetition Time Window, Repetition Count 0)</li>
  <li>Date: 2012-08-19 10:12 Value: 41 OK (still below Max + Threshold)</li>
  <li>Date: 2012-08-19 10:13 Value: 46 OK (above Threshold, Repetition Count 1)</li>
  <li>Date: 2012-08-19 10:14 Value: 47 OK (above Threshold, Repetition Count 2)</li>
  <li>Date: 2012-08-19 10:15 Value: 46 OK (above Threshold, Repetition Count 3)</li>
  <li>Date: 2012-08-19 10:16 Value: 47 OK (above Threshold, Repetition Count 4)</li>
  <li>Date: 2012-08-19 10:17 Value: 46 OK (above Threshold, Repetition Count 5) -&gt; ALERT!</li>
  </ul>
  <h3>Architecture Design Requirements:</h3>
  <blockquote>
  <p>All logic must be inside Business Layer that means we can't use Stored Procedures neither Triggers</p>
  </blockquote>
  <h3>Possible solution 1:</h3>
  <p>Our first thought was to attach Module logic for both ZigBee and Rfid Services however this may result in performance jam. If we call DB for last check every request of Service we generate a lot of  computing transactions that could even end with deadlock. We could use here WCF Queue mechanism, however still we don't see any benefits that might resolve this issue.</p>
  <p>Any recommendation for this solution is appreciated.</p>
  <h3>Possible solution 2:</h3>
  <p>Second idea was to deploy another Windows Process that would in Background scan database with configured frequency, and if &quot;Above Threshold&quot; occurred save this information into separate &quot;Cache&quot; table and also &quot;Mark&quot; this row in database that it has been check for specific notification / alert.
  So later on it will not be added into scanning.</p>
  <p>This implicates:</p>
  <ol>
  <li>Another &quot;Cache&quot; table</li>
  <li>Latency of delivered notification / alert in we configure &quot;Scanner&quot; for 30min frequency</li>
  <li>Some way to &quot;Mark&quot; row for &quot;Specific Notification&quot;, we can't add simply Bool flag as a result that 1 row off reading may be taken into calculation for many defined notifications (3-4 notifications for construction yard)</li>
  <li>How many &quot;Process&quot;, &quot;Background Workers&quot; should be working the same time? The same as existing notification configurations, or one single for all of them, working with a stack ?</li>
  </ol>
  <p>Any recommendation for this solution is appreciated.</p>
  <h3>Conclusion:</h3>
  <p>If you have any ideas, experience, know existing solutions (dlls), patterns that handle this type of problems and share with us, we really appreciate it.</p>