Asbestos has been confirmed as a major pollutant in asbestos-mining areas that are located in western China. In general, asbestos-fibre dust will is released into the environment due to the effect of intensive industrial activities and improper environmental management, such that the health of residents in and around mining areas is jeopardised. A typical asbestos mining area served as an example in this study to analyse the content and fibre morphology of asbestos in soil and air samples in the mining area. The effects of asbestos pollution in and around the mining areas on human health were also assessed based on the U.S. Superfund Risk Assessment Framework in this study. As indicated by the results, different degrees of asbestos pollutions were present in the soil and air, and they were mainly concentrated in the mining area, the ore-dressing area, and the waste pile. The concentration of asbestos in the soil ranged from 0.3% to 91.92%, and the concentration of asbestos fibres in the air reached 0.008–0.145 f·cc−1. The results of the scanning-electron microscope (SEM) energy suggested that the asbestos was primarily strip-shaped, short columnar, and granular, and the asbestos morphology of the soils with higher degrees of pollution exhibited irregular strip-shaped fibre agglomeration. The excess lifetime cancer risk (ELCR) associated with the asbestos fibres in the air of the mining area was at an acceptable level (10−4–10−6), and 40.6% of the monitoring sites were subjected to unacceptable non-carcinogenic risks (HQ > 1). Moreover, the waste pile was the area with the highest non-carcinogenic risk, followed by the ore dressing area, a residential area, and a bare-land area in descending order. In the three scenarios of adult offices or residences in the mining area, adults’ outdoor activities in the peripheral residence areas, and children’s outdoor activities, the carcinogenic-and non-carcinogenic-risk-control values in the air reached 0.1438, 0.2225 and 0.1540 f·cc−1, and 0.0084, 0.0090 and 0.0090 f·cc−1, respectively. The results of this study will lay a scientific basis for the environmental management and governance of asbestos polluted sites in China.