def sitemap_urls_from_robots(robots_text):
    for line in robots_text.splitlines():
        if line.lstrip().lower().startswith('sitemap:'):
            yield line.split(':', 1)[1].strip()
Returns the set of all nodes that are after the given node.
dataset/ETHPy150Open scrapy/scrapy/scrapy/utils/sitemap.py/sitemap_urls_from_robots--crossover with--dataset/ETHPy150Open nltk/nltk/nltk/tgrep.py/_after