AbstractMotivationMachine learning methods can be used to support scientific discovery in healthcare-related research fields. However, these methods can only be reliably used if they can be trained on high-quality and curated datasets. Currently, no such dataset for the exploration ofPlasmodium falciparumprotein antigen candidates exists. The parasiteP.falciparumcauses the infectious disease malaria. Thus, identifying potential antigens is of utmost importance for the development of antimalarial drugs and vaccines. Since exploring antigen candidates experimentally is an expensive and time-consuming process, applying machine learning methods to support this process has the potential to accelerate the development of drugs and vaccines, which are needed for fighting and controlling malaria.ResultsWe developedPlasmoFAB, a curated benchmark that can be used to train machine learning methods for the exploration ofP.falciparumprotein antigen candidates. We combined an extensive literature search with domain expertise to create high-quality labels forP.falciparumspecific proteins that distinguish between antigen candidates and intracellular proteins. Additionally, we used our benchmark to compare different well-known prediction models and available protein localization prediction services on the task of identifying protein antigen candidates. We show that available general-purpose services are unable to provide sufficient performance on identifying protein antigen candidates and are outperformed by our models that were trained on this tailored data.Availability and implementationPlasmoFABis publicly available on Zenodo with DOI 10.5281/zenodo.7433087. Furthermore, all scripts that were used in the creation ofPlasmoFABand the training and evaluation of machine learning models are open source and publicly available on GitHub here:https://github.com/msmdev/PlasmoFAB.