title: Facebook Not Scraping URL Properly (Blocked URL) On Fresh URL's
body: <p>I have started to experience a really strange problem with Facebook link scraping/caching which seems random in nature.</p>
  
  <p>On brand new posts when I go to check: <a href="https://developers.facebook.com/tools/debug/" rel="nofollow noreferrer">https://developers.facebook.com/tools/debug/</a> with my new URL I get a message saying 'Cannot scrape: URL blocked'</p>
  
  <p>We have literally been sharing URL's to Facebook for YEARS with no problem whatsoever. All open graph tags are present and working fine.</p>
  
  <p>What is really weird is if I just manually press the 'scrape again' button it grabs all the bits properly and says everything is fine. It's always just the first attempt.</p>
  
  <p>I have also tried manually calling the scrap API instead of letting Facebook do it when someone shares the URL for the first time and still sometimes (not always) it returns empty (just 'id' and 'url' returned) indicating the URL was blocked.</p>
  
  <p>What's going on here it's driving me mad? We are nowhere near our API limits if maybe that was the issue...</p>
  
  <p>Why would it say blocked first go on the debug page then literally 2 seconds later you press 'scrape again' and it works fine.</p>
  
  <p>Just to note sometimes when it says blocked it still shows maybe the image in the link preview but no title or description. It's like it's grabbed some tags but then given up. Very strange...</p>
  
  <p>I have checked our access logs and I can see 'Facebot' accessing our site fine.</p>
  
  <p>Hope someone can help, I am really hoping this is some weird Facebook bug as I can't see any change in our code base that would effect this.</p>
  
  <p>Facebook does not return anything helpful when this happens even with the API. I have no way in contacting Facebook direct either. If we are somehow getting blocked I would love to know why and fix that issue. But it's impossible to debug at the moment.</p>
  
  <p>Just to note I have 3 sites all running the same code base (WordPress environment). They are all experiencing this issue. All been running fine for many years.</p>
  
  <p>Cheers</p>