Shared Task

Organizer's paper with an overview of the task results:

Jakub Piskorski, Laska Laskova, Michał Marcińczuk, Lidia Pivovarova, Pavel Přibáň, Josef Steinberger, Roman Yangarber. The Second Cross-Lingual Challenge on Recognition, Normalization, Classification, and Linking of Named Entities across Slavic Languages


   author = "Piskorski, Jakub  and Laskova, Laska  and Marci{\'n}czuk, Micha{\l}  and Pivovarova, Lidia  and P{\v{r}}ib{\'a}{\v{n}}, Pavel  
   and Steinberger, Josef  and Yangarber, Roman",
   booktitle = "Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing",
   month = aug,
   year = "2019",
   address = "Florence, Italy",
   publisher = "Association for Computational Linguistics",
   url = "",
   pages = "63--74"

Complete final ranking




The 2019 edition of the shared task covers four languages:

and focuses on recognition of five types of named entities including:









Detailed task description and system response guidelines


Data Sets

Sample data

Sample data consisting of raw documents and corresponding annotations are available HERE.

Training data

Training data are available HERE.



Test data

Test data are available HERE.

The test data set consist of two sets of documents, each related to a specific Topic (revolving around an entity or event), which are different from the topics in the training data set.

The format used is exactly the same as for training data.

Consistency Check

A Java script that checks consistency of the annotation files (including format, valid entity types, id assignment, etc.) can be found HERE.

Evaluation Metrics

A Java script that was used for evaluation can be found HERE. Please read 'readme.txt' inside the archive for more details. Do not hesitate to contact us in case you find any problems.

Evaluation is carried out on the system response returned by the participants for the test corpora.

Named entity recognition (exact case-insensitive matching) and lemmatization tasks are evaluated in terms of precision, recall, and F1 scores. In particular, as regards named entity recognition, two types of evaluations are carried out:



The document-level and cross-language entity matching task are evaluated using the Link-Based Entity-Aware metric (LEA)  introduced in (Moosavi and Strube, 2016).

Please note that the evaluation is case-insensitive. That is, all named mentions in system response and test corpora are lower-cased.

Publication and Workshop


Important Dates