title: Is it possible to limit the number of web request per second?
body: <p>Hi i am spidering the site and reading the contents.I want to keep the request rate reasonable. Up to approx 10 requests per second should probably be ok.Currently it is 5k request per minute and it is causing security issues as this looks to be a bot activity.
  How to do this? Here is my code</p>
  
  <pre><code>protected void Iterareitems(List&lt;Item&gt; items)
  {
      foreach (var item in items)
      {
          GetImagesfromItem(item);
  
          if (item.HasChildren)
          {
              Iterareitems(item.Children.ToList());
          }
      }
  }
  
  protected void GetImagesfromItem(Item childitems)
  {
      var document = new HtmlWeb().Load(completeurl);
      var urls = document.DocumentNode.Descendants("img")
                  .Select(e =&gt; e.GetAttributeValue("src", null))
                  .Where(s =&gt; !string.IsNullOrEmpty(s)).ToList();
  }
  </code></pre>