The outbreak of the mosquito-borne disease in South America is being linked to severe birth defects.
Last week the World Health Organization (WHO) declared the virus a global public health emergency.
Kipchoge Keino said: "We are not going to risk taking Kenyans there if this Zika virus reaches epidemic levels."
Brazilian authorities insist there will be no risk to athletes and spectators, except pregnant women, when the Rio Games take place in August.
The International Olympic Committee says it is in "close contact" with organisers, while Olympic venues will be inspected daily in the lead-up to and during the Games.
Kenya topped the medal table at last year's World Athletics Championships and the country's absence would be severely felt on the track as they boast many of the best middle and long-distance runners in the world.
They won 11 Olympic medals at London 2012, all in athletics, and David Rudisha, one of the sport's biggest stars, would be denied an opportunity to defend his 800m Olympic title.
Meanwhile, the United States Olympic Committee has told American sports federations that athletes and staff concerned for their health should consider not going to the Games.
The Australian and New Zealand Olympic Committees have also said they would support athletes should they withdraw.
Kenya could pull out of this summer's Rio Olympics if the Zika virus in Brazil reaches "epidemic levels", says the head of Kenya's Olympic committee.