ObjectivesWe aimed to present the state of the art of CT- and MRI-based radiomics in the context of ovarian cancer (OC), with a focus on the methodological quality of these studies and the clinical utility of these proposed radiomics models.MethodsOriginal articles investigating radiomics in OC published in PubMed, Embase, Web of Science, and the Cochrane Library between January 1, 2002, and January 6, 2023, were extracted. The methodological quality was evaluated using the radiomics quality score (RQS) and Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Pairwise correlation analyses were performed to compare the methodological quality, baseline information, and performance metrics. Additional meta-analyses of studies exploring differential diagnoses and prognostic prediction in patients with OC were performed separately.ResultsFifty-seven studies encompassing 11,693 patients were included. The mean RQS was 30.7% (range − 4 to 22); less than 25% of studies had a high risk of bias and applicability concerns in each domain of QUADAS-2. A high RQS was significantly associated with a low QUADAS-2 risk and recent publication year. Significantly higher performance metrics were observed in studies examining differential diagnosis; 16 such studies as well as 13 exploring prognostic prediction were included in a separate meta-analysis, which revealed diagnostic odds ratios of 25.76 (95% confidence interval (CI) 13.50–49.13) and 12.55 (95% CI 8.38–18.77), respectively.ConclusionCurrent evidence suggests that the methodological quality of OC-related radiomics studies is unsatisfactory. Radiomics analysis based on CT and MRI showed promising results in terms of differential diagnosis and prognostic prediction.Critical relevance statementRadiomics analysis has potential clinical utility; however, shortcomings persist in existing studies in terms of reproducibility. We suggest that future radiomics studies should be more standardized to better bridge the gap between concepts and clinical applications.Graphical abstractSupplementary InformationThe online version contains supplementary material available at 10.1186/s13244-023-01464-z.