The man barricaded himself in the house and injured one police officer during the late-night battle.
Police say they have recently been investigating individuals suspected of links with foreign terror groups.
In January a Muslim cleric suspected of recruiting for so-called Islamic State was shot dead in Kigali.
Police identified the suspect killed on Wednesday night as Channy Mbonigaba, from the western district of Rubavu.
A terror suspect was shot dead in Rwanda after a three-hour shoot-out in an affluent suburb of the capital, Kigali, police say.