Cricket has not been played at the Summer Games since Great Britain beat France to win gold in 1900.
But the ICC believes an Olympic Twenty20 tournament is the key to growing the game.
"I think the majority of the members - and certainly myself - think the time is right," said Richardson.
"We've come to the conclusion that the overall benefit to the game, in terms of globalising and growing it, outweighs any negatives.
"We need to make a decision by July so we can make an application in time for September, when, as I understand it, the (International Olympic Committee) will consider new sports for 2024."
The 2024 Games will be hosted by either Paris or Los Angeles, with a decision due in September 2017.
Find out how to get into cricket with our inclusive guide.
With growing concerns about the cost of staging the Games, the IOC has been reluctant to go much beyond 11,000 athletes at an Olympics, although it did add five new sports to the programme for Tokyo 2020 without removing any.
Richardson, who made 164 appearances for South Africa in Tests and one-day internationals, said preliminary talks with the IOC have already taken place and the discipline to be included for consideration must be a format played at international level.
"They haven't said [an existing] sport would have to go [to make room], but they said when taking any decision on a new sport they've got an overall limit on the number of athletes," he said.
"So as a team sport we would only fit six to eight teams. T20 is the ideal format and we'd say even better than rugby sevens as it's actually one of the mainstream formats of cricket."
He said that he did not think there would be any opposition from ICC members England or West Indies due to the fact that they would not be able to compete at a Games under those banners.
He suggested England would be relaxed about playing along with other home nations as Great Britain.
The "time is right" for cricket to apply for a place at the Olympics, according to International Cricket Council chief Dave Richardson.