PurposeThis study aims to identify the mechanism of Inhibin Subunit Beta B (INHBB), a member of the transforming growth factor-β (TGF-β) family involved in the regulation of human endometrial stromal cells (HESCs) decidualization in recurrent implantation failure (RIF).MethodsRNA-seq was conducted to identify the differentially expressed genes in the endometria from control and RIF patients. RT-qPCR, WB, and immunohistochemistry were performed to analyse the expression levels of INHBB in endometrium and decidualised HESCs. RT-qPCR and immunofluorescence were used to detect changes in the decidual marker genes and cytoskeleton after knockdown INHBB. Then, RNA-seq was used to dig out the mechanism of INHBB regulating decidualization. The cAMP analogue (forskolin) and si-INHBB were used to investigate the involvement of INHBB in the cAMP signalling pathway. The correlation of INHBB and ADCY expression was analysed by Pearson’s correlation analysis.ResultsOur results showed significantly reduced expression of INHBB in endometrial stromal cells of women with RIF. In addition, INHBB was increased in the endometrium of the secretory phase and significantly induced in in-vitro decidualization of HESCs. Notably, with RNA-seq and siRNA-mediated knockdown approaches, we demonstrated that the INHBB-ADCY1-mediated cAMP signalling pathway regulates the reduction of decidualization. We found a positive association between the expression of INHBB and ADCY1 in endometria with RIF (R2= 0.3785,P= 0.0005).ConclusionsThe decline of INHBB in HESCs suppressed ADCY1-induced cAMP production and cAMP-mediated signalling, which attenuated decidualization in RIF patients, indicating that INHBB is an essential component in the decidualization process.Supplementary InformationThe online version contains supplementary material available at 10.1007/s10815-023-02762-7.