The race at the Sepang International Circuit has been an annual fixture on the F1 calendar since 1999.
However, ticket sales and TV viewing figures have been in decline and Razlan Razali, chief executive of the circuit, questioned the race's "economic value".
Red Bull's Daniel Ricciardo won at Sepang in October but only 45,000 fans attended the 120,000-capacity circuit.
"From the longer term perspective, it's probably the correct decision because it's more expensive and no longer that attractive," said Nazri Abdul Aziz, the country's culture and tourism minister.
Malaysia will stop hosting a Formula 1 grand prix after their current agreement expires in 2018.