Image quality assessment (IQA) is significant for current techniques of image-based computer-aided diagnosis, and fundus imaging is the chief modality for screening and diagnosing ophthalmic diseases. However, most of the existing IQA datasets are single-center datasets, disregarding the type of imaging device, eye condition, and imaging environment. In this paper, we collected a multi-source heterogeneous fundus (MSHF) database. The MSHF dataset consisted of 1302 high-resolution normal and pathologic images from color fundus photography (CFP), images of healthy volunteers taken with a portable camera, and ultrawide-field (UWF) images of diabetic retinopathy patients. Dataset diversity was visualized with a spatial scatter plot. Image quality was determined by three ophthalmologists according to its illumination, clarity, contrast and overall quality. To the best of our knowledge, this is one of the largest fundus IQA datasets and we believe this work will be beneficial to the construction of a standardized medical image database.