It is still extremely challenging to endow epoxy resins (EPs) with excellent flame retardancy and high toughness. In this work, we propose a facile strategy of combining rigid–flexible groups, promoting groups and polar phosphorus groups with the vanillin compound, which implements a dual functional modification for EPs. With only 0.22% phosphorus loading, the modified EPs obtain a limiting oxygen index (LOI) value of 31.5% and reach V-0 grade in UL-94 vertical burning tests. Particularly, the introduction of P/N/Si-containing vanillin-based flame retardant (DPBSi) improves the mechanical properties of EPs, including toughness and strength. Compared with EPs, the storage modulus and impact strength of EP composites can increase by 61.1% and 240%, respectively. Therefore, this work introduces a novel molecular design strategy for constructing an epoxy system with high-efficiency fire safety and excellent mechanical properties, giving it immense potential for broadening the application fields of EPs.