China says 300,000 civilians were massacred when the city was occupied by Japan's troops in 1937, although some Japanese nationalists dispute this.
President Xi told survivors that to deny a crime was to repeat it but insisted the ceremony was to promote peace, not prolong hatred.
Relations between the two countries have been strained in recent years.
They have clashed over island territory in the East China Sea as well as over Japan's insistence on honouring its war dead, including convicted war criminals, at the Yasukuni shrine.
The ceremony, which came on the 77th anniversary of the massacre, is part of three new public holidays intended to mark the conflict between the two countries.
A crowd of about 10,000 people attended the event in Nanjing, taking part in a minute's silence to honour those killed. They included survivors of the massacre, as well as soldiers and students.
The "Rape of Nanjing", as it's often referred to in China, is an exceptionally sensitive issue in the frequently tense relations between the two countries.
China and Japan have been locked in a bitter territorial dispute - although relations have improved somewhat after the two leaders held their first talks last month.
That said, the countries continue to spar over Japan's brutal occupation of China during World War Two.
Beijing says Tokyo has never properly apologised or atoned for its wartime past. It was furious after Japan's Prime Minister Shinzo Abe visited a shrine that honours war criminals among the country's dead.
And while China says tens of thousands of civilians were slaughtered in Nanjing, some Japanese politicians and nationalists deny a massacre even took place.
In a speech at the event, Mr Xi criticised Japanese nationalists for denying the atrocity took place.
"Anyone who tries to deny the massacre will not be allowed by history, the souls of the 300,000 deceased victims, 1.3 billion Chinese people and all people loving peace and justice in the world," Mr Xi said.
But he added that China should not "bear hatred against an entire nation just because a small minority of militarists launched aggressive wars," according to the state-run Xinhua news agency.
Millions of Chinese people were killed when Japan occupied China in the 1930s and 1940s.
Chinese President Xi Jinping has presided over his country's first state commemoration of the Nanjing massacre.