Former justice secretary Gove said it had "nothing to do with the genius" of  JMW Turner, the artist the competition was named after.
Mr Vaizey, an ex-culture minister, said the prize "celebrates brilliant contemporary artists".
Their comments were made in advance of Helen Marten winning the prestigious Â£25,000 award.
Mr Gove acknowledged Marten's win after the announcement, tweeting: "Congratulations to Helen Martin but #honestly is this = to Turner, Ruskin, even Holman Hunt - of course not #modishcrap."
He previously stated that the prize "celebrates ugliness, nihilism and narcissism" and "the tragic emptiness of now".
Others to weigh in on the issue included journalist Miranda Sawyer, who responded: "I've been a judge on the Turner Prize and those are not the attributes we looked for."
She added: "Some artists feel their job is to comment/engage/challenge the 'now', tragically empty or not."
Marten, who trained at the Ruskin School of Art and works using sculpture, screen printing and writing, pledged to share the prize money with her fellow nominees.
She won the Â£30,000 Hepworth Prize for sculpture in November and also pledged to share the prize with the other nominees.
Mr Gove's initial comments were prompted by a BBC News tweet inviting viewers to follow live coverage of this year's awards ceremony.
Other commentators were more receptive to Marten's victory over fellow nominees Michael Dean, Anthea Hamilton and Josephine Pryde.
Art critic Adrian Searle wrote in The Guardian that the Macclesfield-born artist "thinks differently from the rest of us" and praised her "playful and inventive way with language".
The Telegraph's Mark Hudson agreed that Marten is a "worthy winner" but claimed she was "not the people's choice".
Dean's installation on living on the poverty line, he wrote, "won him the voluble support of gallery goers who have stuck their opinions on post-it notes, at Tate's invitation, on the walls outside the exhibition".
The Turner Prize ceremony was broadcast live on the BBC News Channel and BBC World and is now available online.
Follow us on Facebook, on Twitter @BBCNewsEnts, or on Instagram at bbcnewsents. If you have a story suggestion email entertainment.news@bbc.co.uk.
Politicians Michael Gove and Ed Vaizey have had a Twitter spat about the merits of the Turner Prize.