Why does only the risk impacting the market portfolio matter for any asset?
Since all investors hold the same market portfolio, the only risk that matters is how an asset affects the market portfolio's risk.