With the development of miniaturized, highly integrated, and multifunctional electronic devices, the heat flow per unit area has increased dramatically, making heat dissipation a bottleneck in the development of the electronics industry. The purpose of this study is to develop a new inorganic thermal conductive adhesive to overcome the contradiction between the thermal conductivity and mechanical properties of organic thermal conductive adhesives. In this study, an inorganic matrix material, sodium silicate, was used, and diamond powder was modified to become a thermal conductive filler. The influence of the content of diamond powder on the thermal conductive adhesive properties was studied through systematic characterization and testing. In the experiment, diamond powder modified by 3-aminopropyltriethoxysilane coupling agent was selected as the thermal conductive filler and filled into a sodium silicate matrix with a mass fraction of 34% to prepare a series of inorganic thermal conductive adhesives. The thermal conductivity of the diamond powder and its content on the thermal conductivity of the adhesive were studied by testing the thermal conductivity and taking SEM photos. In addition, X-ray diffraction, infrared spectroscopy, and EDS testing were used to analyze the composition of the modified diamond powder surface. Through the study of diamond content, it was found that as the diamond content gradually increases, the adhesive performance of the thermal conductive adhesive first increases and then decreases. The best adhesive performance was achieved when the diamond mass fraction was 60%, with a tensile shear strength of 1.83 MPa. As the diamond content increased, the thermal conductivity of the thermal conductive adhesive first increased and then decreased. The best thermal conductivity was achieved when the diamond mass fraction was 50%, with a thermal conductivity coefficient of 10.32 W/(m·K). The best adhesive performance and thermal conductivity were achieved when the diamond mass fraction was between 50% and 60%. The inorganic thermal conductive adhesive system based on sodium silicate and diamond proposed in this study has outstanding comprehensive performance and is a promising new thermal conductive material that can replace organic thermal conductive adhesives. The results of this study provide new ideas and methods for the development of inorganic thermal conductive adhesives and are expected to promote the application and development of inorganic thermal conductive materials.