Downloads the file; (3) Computes its SHA256 checksum; (4) Archives the file. To check that an app has not been already downloaded, we first identify a unique identifier for APKs in the market associated to the crawler, and store in a database an entry market name–app identifier. As a consequence, and because it is impossible to determine that two files from two markets are the same unless both are downloaded and compared, the deduplication is local to one market, meaning that one file from one market is downloaded exactly once, regardless of whether or not it has already been downloaded from another market.

By far, we are collecting apps from around ten app markets or repositories. We now describe some representative ones

-   Google Play. The official market of Android is a website that allows users to browse its content through a web browser. Apps cannot however be downloaded through a web browser. Instead, Google provides an Android app2 that uses a proprietary protocol to communicate with Google Play servers. No app, however, can be downloaded from Google Play without a valid Google account (not even free Apps). Both issues thus outlined were overcome using open-source implementations of the proprietary 