Researchers found 75% of men would opt for the big screen version of a story, while 30% admitted they had not picked up a book since they were at school.
Being too busy, not enjoying reading or spending time online were all blamed for reading less.
Men also tended to be slower readers and less likely to finish books.
Of those questioned about their reading habits, 63% of men said they did not read as much as they thought they should, with 46% saying they now read fewer books now than they did in the past.
However, one in five men also confessed to pretending they had read a specific title, in order to appear more intelligent.
Researchers from OnePoll spoke to 2,000 British men and women ahead of World Book Night on 23 April, when volunteers gift hundreds of thousands of free books to "reluctant readers" in their communities.
"We know reading is really important, so we've got to get more people in general, particularly men, to pick up a book," said Sue Wilkinson, CEO of the Reading Agency, which commissioned the survey.
"It seems that men recognise the value of reading books but admit that they don't do it as much as they might for several reasons.
"TV shows and films, and the internet, are competing for people's time these days, especially that of young men, and our focus is to remind them of the pleasure that can be derived from reading a book as well," she said.
A third of men in the study said they preferred to spend their spare time using the internet, while 30% engaged more with film and TV.
Organisers said this year's World Book Night was focusing on targeting men to "show them what they are missing".
Wilkinson encouraged people to think of a man in their community who may benefit from being given a book, adding "reading really can change your life".
The 20 titles being distributed on World Book Night are as follows;
Men prefer to watch film adaptations of books than read the original novel, according to a new study, which found the opposite is true for women.