def urlDispatcher(self):
        logger.log(logging.INFO, "Starting server urlDispatcher")

        while self.isActive:
            try:
                url = urlPool.get(True)
                if url not in urlVisited:
                    urlVisited[url] = True
                    #logic if static crawling will come here
                    urlToVisit.put(url)
                    scrappedURLlist.append(url)
            except:
                exc_type, exc_value, exc_traceback = sys.exc_info()
                message = "\n" + ''.join(traceback.format_exception(exc_type, exc_value, exc_traceback))
                logger.log(logging.ERROR, message)
Reads from the urlPool, makes sure the url has not been visited and adds it to the urlToVisit Queue
dataset/ETHPy150Open Diastro/Zeek/src/server.py/Server.urlDispatcher