title: Blocking Task on Java web application, and request timeout on Heroku server
body: <p>I am new to Java web programming, I'm trying to make a web crawler, Using the Crawler4j <a href="https://code.google.com/p/crawler4j/source/browse/src/test/java/edu/uci/ics/crawler4j/examples/basic/BasicCrawler.java" rel="nofollow">sample code</a></p>
  
  <p>My problem is that when I submit the repost request, the Crawling task ( which is a blocking task) takes some time to get done, Heroku hosting has a request timeout of 3 seconds, which makes it impossible to run a synchronous crawling task, The same program works just fine on my local machine.</p>
  
  <p>From what I read, it's not possible to change Heroku's timeout with the basic/free offer.</p>
  
  <p>I was wondering if it was possible to launch this as an asynchronous task ( well I do know it is possible using the CrawlerController.startNonBlocking() function) and wait for it to finish so that I can show the results of the crawling operation.</p>
  
  <pre><code>@Override
  protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException{
  
  
      String url = request.getParameter("url");
  
  
  
  CrawlConfig config = new CrawlConfig();   
  
  String crawlStorageFolder = "/tmp/temp_storage";
  config.setCrawlStorageFolder(crawlStorageFolder);    
  
  
  int numberOfCrawlers = 1;
  
  config.setPolitenessDelay(1);
  
  config.setMaxDepthOfCrawling(2);
  
  config.setMaxPagesToFetch(5);
  
  config.setResumableCrawling(false);
  
  PageFetcher pageFetcher = new PageFetcher(config);
  RobotstxtConfig robotstxtConfig = new RobotstxtConfig();
  RobotstxtServer robotstxtServer = new RobotstxtServer(robotstxtConfig, pageFetcher);
  CrawlController controller = null;
  
  try {
  controller = new CrawlController(config, pageFetcher, robotstxtServer);
  } catch(Exception e){
  e.printStackTrace();
  }
  
  controller.addSeed(url);
  
  
  controller.start(Crawler.class, numberOfCrawlers);
  // Methods showing the results of the crawling ...
  }
  </code></pre>