As one of the most popular edible fungi in the market, the quality ofAgaricus bisporuswill determine its sales volume. Therefore, to achieve rapid and nondestructive testing of the quality ofAgaricus bisporus, this study first built a portable spectrum acquisition device forAgaricus bisporus. The Ocean Spectromeper was used to calibrate the spectral data of the device, and the linear regression analysis method was combined to analyze the two. The results showed that the Pearson correlation coefficient of significance between the two was 0.98. Then, the spectral data ofAgaricus bisporuswere collected, the spectral characteristic wavelength ofAgaricus bisporuswas extracted by the SPA and PCA algorithms, and the moisture content and whiteness prediction models based on a BP neural network and PLSR, respectively, were built. The parameters of the BP neural network model were optimized by SSA. The R2values for the final moisture content and the predicted whiteness were 0.95 and 0.99, and the RMSE values were 5.04% and 0.60, respectively. The results show that the portable spectral acquisition and analysis device can be used for the accurate and rapid quality detection ofAgaricus bisporus.