title: Curl fails after following 50 redirects but wget works fine
body: <p>I have an experimental web crawler and I noticed that it cannot read some pages, for example on some particular domains curl says it failed after following 50 redirects but wget reads that same domain just fine:</p>
  
  <p><code>curl 'netflix.com' -L -o 'output.txt'</code></p>
  
  <p><strong>Result:</strong></p>
  
  <p><code>curl: (47) Maximum (50) redirects followed</code></p>
  
  <p>No data in output.txt file.</p>
  
  <p><strong>While this command works fine:</strong></p>
  
  <p><code>wget netflix.com</code></p>
  
  <p>Any ideas on what can cause this? I doubt that remote server handles requests based on the two different user agents.</p>