In August last year, ANA said it would pay some $25m (2.5bn yen; Â£16m) for a stake in the Burmese carrier.
ANA Holdings is the parent company of Japan's All Nippon Airways.
In a statement, ANA said rising competition in Myanmar was one reason for cancelling the deal.
"Competition between new and old airlines in Myanmar has intensified," ANA said, "bringing rapid changes in the external environment, and calling into question the assumptions made at the time of the original decision."
It also said it had been unable to reach an agreement on capital participation with AWA.
The deal was part of ANA's plans to expand its business internationally.
At the time, it would have been the first investment in a Myanmar-based commercial airline by a foreign airline.
AWA, which is based in Yangon, was launched in 2011 and operates domestic flights within Myanmar.
It also offers flights to Chiang Mai in Thailand and has plans to further expand its international service.
Since political reforms have led to the easing of international sanctions in Myanmar, many foreign firms have looked to the country for business opportunities.
After a 12-year hiatus, All Nippon Airways resumed its service between Japan and Myanmar in October 2012.
Japan's ANA Holdings (ANA) has cancelled its plan to buy a 49% stake in Asian Wings Airways (AWA), an airline based in Myanmar, also known as Burma.