The move was a "serious breach of Iraqi sovereignty" Prime Minister Haider al-Abadi's office said in a statement.
About 150 Turkish soldiers have been deployed near in the town of Bashiqa to train Iraqi Kurdish forces, earlier reports said.
Mosul has been under the control of the Islamic State (IS) group since last year.
The statement called on Turkey to "respect good neighbourly relations and to withdraw immediately from the Iraqi territory".
Turkey enjoys close relations with autonomous Kurdish regions in Iraq, although it views Syrian Kurdish groups over the border as hostile, analysts say.
The fall of Mosul was a key moment in the rise of IS and a Iraqi government offensive to retake the city has been repeatedly put back.
This week saw the UK carry out its first air strikes against IS in Syria and Germany voting to send military support to the coalition fighting IS in the country.
The Iraqi government has demanded that Turkey withdraw troops it sent to an area near the northern city of Mosul.