BackgroundThe Epstein-Barr virus (EBV) is a prevalent oncovirus associated with a variety of human illnesses. BGLF5, an EBV DNase with alkaline nuclease (AN) activity, plays important roles in the viral life cycle and progression of human malignancies and has been suggested as a possible diagnostic marker and target for cancer therapy. Methods used conventionally for the detection of AN activity, radioactivity-based nuclease activity assay and DNA digestion detection by gel electrophoresis, are not suitable for screening AN inhibitors; the former approach is unsafe, and the latter is complicated. In the present study, a fluorescence-based nuclease activity assay was used to screen several natural compounds and identify an EBV DNase inhibitor.ResultsFluorescence-based nuclease activity assays, in which the DNA substrate is labelled with PicoGreen dye, are cheaper, safer, and easier to perform. Herein, the results of the fluorescence-based nuclease activity assay were consistent with the results of the two conventional methods. In addition, the PicoGreen-labelling method was applied for the biochemical characterisation of viral nucleases. Using this approach, we explored EBV DNase inhibitors. After several rounds of screening, emodin, an anthraquinone derivative, was found to possess significant anti-EBV DNase activity. We verified the efficacy of emodin using the conventional DNA-cleavage assay. Furthermore, using comet assay and micronucleus formation detection, we confirmed that emodin can inhibit DNase-induced DNA damage and genomic instability. Additionally, emodin treatment inhibited EBV production.ConclusionsUsing a PicoGreen-mediated nuclease activity assay, we successfully demonstrated that emodin has the potential to inhibit EBV DNase nuclease activity. Emodin also inhibits EBV DNase-related biological functions, suggesting that it is a potential inhibitor of EBV DNase.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12985-023-02107-x.