BackgroundThe effectiveness of the long-term care service in Japan has been unclear, and most of the relevant studies of this service have been limited to a single region and relatively small samples, necessitating large-scale studies. We examined the associations between long-term care service use and the service/care-need level progression at the national scale in Japan.MethodsWe conducted a nationwide retrospective cohort study using data from the Japanese Long-Term Care Insurance Claims database. Individuals aged ≥ 65 years and newly certified as being at the support-need level 1 or 2 or the care-need level 1 between April 2012 and March 2013 were included. We first conducted 1:1 propensity score matching and then examined the associations between service use and the progression in support-need or care-need levels by using Kaplan–Meier survival curves and log-rank tests.ResultsThe final sample consisted of 332,766 individuals. We observed that service use was associated with a faster decline in the support/care-need level, although the differences in the subjects' survival rate diminished; the log-rank test showed significance (p< 0.001). When stratified for urban–rural classifications or regions of Japan, the results were similar to the primary analysis in all of the stratified groups, and no clear regional variations were observed.ConclusionWe did not observe a clear beneficial effect of receiving long-term care in Japan. Our results suggest that Japan's current long-term care service may not be effective for the recipients of these services. Considering that the system is becoming a financial burden, a re-examination of the service to provide more cost-effective care may be advisable.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12913-023-09615-0.