Neither candidate gained the 50% needed to win outright in the first round. Mr Duda won 34.7% of the vote, just ahead of Mr Komorowski on 33.7%.
Mr Komorowski hopes to win a second term in office.
Opinion polls before the election had put him comfortably in the lead. The turnout was 48.8%.
Mr Komorowski, 62, took office five years ago after his predecessor, Lech Kaczynski, died in a plane crash.
He is an independent allied with the centre-right Civic Platform, which has been in government since 2007.
Mr Komorowski said the election result was "a serious warning for the entire team in power''.
Mr Duda is from the right-wing opposition Law and Justice party, which is led by former President Kaczynski's twin brother, Jaroslaw.
He welcomed the result of the first round and looked forward to "two weeks of hard work" before the second round on 24 May.
"I deeply believe victory is still ahead of us," he said.
Mr Duda attracted most support in the more conservative eastern regions of the country, near the border with Ukraine and Belarus.
Rock musician Pawel Kukiz came third in the election, gaining just over 20% of the vote. There were another eight candidates standing.
The president has limited powers, but is head of the armed forces and can veto new laws.
The presidential vote comes ahead of parliamentary elections this autumn, and may give pointers to Civic Platform's chances of retaining power for a third term.
Conservative challenger Andrzej Duda will face President Bronislaw Komorowski in a run-off in Poland's presidential election.