Atrial fibrillation (AF) is an arrhythmic cardiac disorder with a high and increasing prevalence in aging societies, which is associated with a risk for stroke and heart failure. However, early detection of onset AF can become cumbersome since it often manifests in an asymptomatic and paroxysmal nature, also known as silent AF. Large-scale screenings can help identifying silent AF and allow for early treatment to prevent more severe implications. In this work, we present a machine learning-based algorithm for assessing signal quality of hand-held diagnostic ECG devices to prevent misclassification due to insufficient signal quality. A large-scale community pharmacy-based screening study was conducted on 7295 older subjects to investigate the performance of a single-lead ECG device to detect silent AF. Classification (normal sinus rhythm or AF) of the ECG recordings was initially performed automatically by an internal on-chip algorithm. The signal quality of each recording was assessed by clinical experts and used as a reference for the training process. Signal processing stages were explicitly adapted to the individual electrode characteristics of the ECG device since its recordings differ from conventional ECG tracings. With respect to the clinical expert ratings, the artificial intelligence-based signal quality assessment (AISQA) index yielded strong correlation of 0.75 during validation and high correlation of 0.60 during testing. Our results suggest that large-scale screenings of older subjects would greatly benefit from an automated signal quality assessment to repeat measurements if applicable, suggest additional human overread and reduce automated misclassifications.