Boron nitride nanosheet (BNNS) and graphene oxide (GO) as a single filler can effectively improve the thermal conductivity of the composites, and the synergistic mechanism of BNNS and GO was investigated in this paper. In this study, BNNS was first surface-functionalized and the multi-sized (50 nm, 200 nm, 500 nm) modified BNNS (A-BN) were attached to GO through non-covalent bonding interactions to form a cross-linked structure. Then, A-BN and GO were used as thermal fillers and support material adsorption eutectic phase change materials (PCMs) to prepare composite phase change material (CPCM). Characterization results show that small-size A-BN was more likely to form dense thermal networks with good compatibility and interface connectivity between PCMs, A-BN, and GO, ensuring that PCMs can be stored in the network without leaking. When the size of the BNNS was greater than 200 nm, the advantage of thermal conductivity obtained by A-BN was no longer obvious, and the phase change behavior of CPCM was inhibited. In general, the prepared CPCM has the ideal thermal response and thermal stability, which is very suitable for energy storage and thermal management applications.