title: Multilevel web spider with regex match?
body: <p>I need a web spider to find certain links with regex.</p>
  
  <p>The spider would visit a list of websites, find links that match a regex pattern list, visit those matched links and repeat until the configured depth level.</p>
  
  <p>I was about to code this on php but im not very good with threads on php and I need threads for this application.</p>
  
  <p>So, what do you think is the best solution? </p>
  
  <p>Maybe theres some existing app/code I could configure to create this spider.</p>