Search engine scraping Search engine scraping Search engine scraping is the process of harvesting URLs, descriptions, or other information from search engines such as Google, Bing or Yahoo. This is a specific form of screen scraping or web scraping dedicated to search engines only. Most commonly larger search engine optimization (SEO) providers depend on regularly scraping keywords from search engines, especially Google, to monitor the competitive position of their customers' websites for relevant keywords or their indexing status. Search engines like Google do not allow any sort of automated access to their service but from a legal point of view there is no
Dragonfly (search engine) inconsistent with our mission, providing no information is more inconsistent with our mission," referring to the alternative of not servicing Chinese users at all. Google also downplayed the extent of the new search engine's censorship, reminding users that it also removes search results from its German, French, and U.S. platforms in order to comply with local government regulations in those countries. Ultimately, Google.cn received tepid acceptance: some commentators even praised the search engine with the logic that Chinese citizens, through conducting searches and observing which results had been removed, could better their understanding of what it was their government did