title: Where to put a Crawler script in Laravel project?
body: <p>I have created a really simple PHP crawler, which I want to implement in a Laravel project. I don't know where to put it tho.. I want to start the script and just run it while the application is up.</p>
  
  <p>I know that it should not be in the Controllers, or in the Cron schedule, so any suggestions where to set it up?</p>
  
  <pre><code>$homepage = 'https://example.com';
  $already_crawled = [];
  $crawling = [];
  
  
  function follow_links($url){
    global $already_crawled;
    global $crawling;
  
    $doc = new DOMDocument();
    $doc-&gt;loadHTML(file_get_contents($url));
  
    $linklist = $doc-&gt;getElementsByTagName('a');
  
    foreach ($linklist as $link) {
      $l = $link-&gt;getAttribute("href");
      $full_link = 'https://example.com'.$l;
  
      if (!in_array($full_link, $already_crawled)) {
        $already_crawled[] = $full_link;
        $crawling[] = $full_link;
        echo $full_link.PHP_EOL;
        // Insert data in the DB
      }
    }
  
    array_shift($crawling);
    foreach ($crawling as $link) {
      follow_links($link);
    }
  }
  
  follow_links($homepage);
  </code></pre>