human
What is a “web crawler”?
assistant
A web crawler is a program that automatically scans the internet and indexes content. It is used to search for information on the web, such as documents, images, videos, and other files.