To meet the demand for more extensive applications of Mg alloys, a Mg-5Al-2Ca-1Mn-0.5Zn alloy without RE was prepared in this paper, and its mechanical properties were further improved by conventional hot extrusion and subsequent rotary swaging. The results show that the hardness of the alloy decreases along the radial central region after rotary swaging. The strength and hardness of the central area are lower, but the ductility is higher. The yield strength and ultimate tensile strength of the alloy in the peripheral area after rotary swaging reach 352 MPa and 386 MPa, respectively, while the elongation remains at 9.6%, exhibiting better strength–ductility synergy. The grain refinement and dislocation increase caused by rotary swaging promoted strength improvement. The activation of non-basal slips during rotary swaging is an important reason for the alloy to maintain good plasticity while improving strength.