title: Request method and threading issue?
body: <p>I'm developing an app that scrapes a website for Google+1's, Facebook shares and Tweets. I have a request method that accepts a URL and then goes off and fetches the count for each social media type.</p>
  
  <p>It's logic is as follows: -</p>
  
  <ul>
  <li>Take URL</li>
  <li>Do requests through local/default IP until we get rate limited/non-500 response</li>
  <li>On error
  <ul>
  <li>Call <code>SelectNewProxy()</code> which iterates over a list of proxies and returns one at random to try (a nice way of avoiding request limits for your IP)</li>
  <li>Remove bad proxy from list to avoid selecting it again</li>
  <li>Start a timer which increments every second</li>
  </ul></li>
  <li>When the timer == 600 (10 minutes)
  <ul>
  <li>Create new <code>WebProxy</code> and try to the requests behind our local/default IP again</li>
  <li>Reset timer</li>
  </ul></li>
  </ul>
  
  <p>Rinse and repeat</p>
  
  <p>The code is as follows:</p>
  
  <pre><code>public string Request(string action)
  {
        HttpWebRequest req;
  
        OnStatusChange(new MyArgs() { Message = "Status: Requesting..." });
        string response = string.Empty;
  
        while (response.Equals(string.Empty) &amp;&amp; proxy != null)
        {
               try
               {
                    req = (HttpWebRequest)WebRequest.Create(action);
                    req.Proxy = proxy;
                    HandleUIMessages(action, proxy);
                    response = new StreamReader(req.GetResponse().GetResponseStream()).ReadToEnd();
                }
                catch
                {
                    //OnProxyChange(new MyArgs() { ProxyMessage = string.Format("Proxy: {0}", proxy.Address.ToString()) });
                    RemoveProxy(proxy);
  
                    if (!timer.Enabled)
                    {
                        timer.Interval = (int)TimeInterval.OneSecond;
                        timer.Elapsed += new System.Timers.ElapsedEventHandler(timer_Elapsed);
                        timer.Enabled = true;
                        timer.Start();
                    }
                    WebProxy reset = new SelectNewProxy();     
                    proxy = counter &gt;= 600 ? reset : proxy = SelectNewProxy();
                 }
        }
        return response;
  }
  </code></pre>
  
  <p>It's worth mentioning that I'm using <code>ThreadPool</code> and each request is running in it's own thread. It seems like it would work but I don't get the desired effect, the counter reaches '600' and sets <code>proxy = reset</code> but it appears to only do it very briefly, possibly only for the first thread that hits it? Then <code>timer_Elapsed</code> is called and <code>counter</code> is reset. Could it be that a thread is hitting it, assigning <code>proxy = reset</code> and then because <code>counter</code> has now been reset (no longer >= 600), all subsequent queued up threads call <code>SelectNewProxy()</code>? Feel like I'm rambling but hopefully someone can make sense of what I'm trying to say and if I'm right in my guess, how can I ensure that all threads get <code>proxy = reset</code> and retry our inital IP?</p>
  
  <p>Any help is much appreciated!</p>
  
  <p>Thankyou</p>