Six other people were wounded in the attack that began at Ammunition Hill.
The gunman fired at a tram stop, then drove off. He stopped after a police pursuit and opened fire again, killing a policeman before being shot dead.
Prime Minister Benjamin Netanyahu praised the "very fast, very determined action of the Israeli police".
The police said the attacker was a Palestinian from East Jerusalem.
The tram station is also close to a police headquarters.
The Jerusalem Post said the dead were a 60-year-old woman and a police special forces operative.
A pedestrian and a policeman have died after a Palestinian gunman opened fire from his car in an attack in Jerusalem, Israeli police say.