*podemos* ‘we can’ vs. *Podemos* political party). These phenomena are specially frequent in social media, where language rules are often ignored.

2.  [**Language specific keywords**]{}. A word that is a very good keyword in a language can be a source of noise in another, e.g. *mendia*, ‘mountain’ in Spanish, is unambiguously referring to ‘Idoia Mendia’, a Basque politician, in our context, while in Basque it is clearly ambiguous.

3.  [**Anchor terms**]{} usually define the general topic (e.g. election campaign) to monitor. If the user specifies that a keyword requires an anchor, then in order to accept a message containing that keyword the message must also contain at least one anchor term. Anchor terms may be keywords or not.

4.  [**Long paragraphs are split**]{} before looking for keywords in the case of messages coming from news sites. First, it looks if any keyword appears in a candidate article. If so, it looks for keywords sentence by sentence, and those sentences are considered as the message unit.

Language Identification (LID) {#isv2018:dc:crawler:lid .unnumbered}
-----------------------------

LID is indispensable in order to apply the corresponding NLP analysis. LID is integrated into the crawling process as part of the MSM system. There are two main reasons 