title: Can I use WGET to generate a sitemap of a website given its URL?
body: <p>I need a script that can spider a website and return the list of all crawled pages in plain-text or similar format; which I will submit to search engines as sitemap. Can I use WGET to generate a sitemap of a website? Or is there a PHP script that can do the same? </p>