The lack of response to therapy in pancreatic ductal adenocarcinoma (PDAC) patients has contributed to PDAC having one of the lowest survival rates of all cancer types. The poor survival of PDAC patients urges the exploration of novel treatment strategies. Immunotherapy has shown promising results in several other cancer types, but it is still ineffective in PDAC. What sets PDAC apart from other cancer types is its tumour microenvironment (TME) with desmoplasia and low immune infiltration and activity. The most abundant cell type in the TME, cancer-associated fibroblasts (CAFs), could be instrumental in why low immunotherapy responses are observed. CAF heterogeneity and interactions with components of the TME is an emerging field of research, where many paths are to be explored. Understanding CAF–immune cell interactions in the TME might pave the way to optimize immunotherapy efficacy for PDAC and related cancers with stromal abundance. In this review, we discuss recent discoveries on the functions and interactions of CAFs and how targeting CAFs might improve immunotherapy.