Intan Olivia Marbun was severely burned after an attacker threw petrol bombs at a group of small children playing outside the church on Sunday.
Three other children were injured.
Police arrested a suspect after he was captured by local people following the attack.
The attacker has been identified as a militant linked to local extremists supporting the so-called Islamic State (IS) group.
"The man passed by the church and threw what we suspected was low-explosive  cocktail bombs, causing light injuries to four children who were playing there," local police spokesman Fajar Setiawan told the AFP news agency.
Police say the attacker was wearing a t-shirt emblazoned with the word "jihad", the agency adds.
"Marbun suffered extensive burn injuries and respiratory problems and doctors could not save her," Mr Setiawan said.
The other children suffered "less serious injuries and were still being treated in hospital" but would likely be discharged soon, according to Reuters, which quoted a hospital spokesman.
Indonesia has the world's largest Muslim population and is home to significant numbers of Christians, Hindus and Buddhists.
The country has suffered a series of attacks inspired by the IS militant group in recent times.
A two-year-old girl has died from injuries she sustained in an attack at a church on the island of Borneo in Indonesia.