A report by Public Health England estimates that up to 35,000 adults with a learning disability are being prescribed an antipsychotic, an antidepressant or both without appropriate clinical justification.
A letter is being sent to patients and professionals as a call to action.
Anyone concerned about a prescription is urged to consult their doctor.
People should not stop their medication unless they have been advised to do so by their clinician, experts advise.
The issue first came to light following a review into an abuse scandal at a private hospital. The government report into the events at Winterbourne View, noted "deep concerns" about the over-use of medicines in people with learning disabilities and autism.
When used appropriately and kept under review, the drugs can be useful.
But sometimes they are used wholly inappropriately, as a "chemical restraint" to control behaviour, in place of other more appropriate treatments, despite guidelines recommending against this.
NHS England commissioned three reports - from the Care Quality Commission, Public Health England and NHS Improving Quality - which found a much higher rate of prescribing of medicines associated with mental illness among people with learning disabilities than the general population.
NHS England will be hosting an urgent summit on the issue on 17 July to formulate a plan.
A delivery board will then be established to drive through the necessary changes.
Dominic Slowie, NHS England's national clinical director for learning disabilities, said: "This is a historic problem, but one that nobody knew the true scale of; that's why we worked with patients, carers and professionals to get to the bottom of the issues once and for all.
"We are determined to take action to protect this group of patients from over-medication."
The Royal College of Psychiatrists condemned inappropriate use of antipsychotic medication, warning: "There will not only be very few benefits to the individual, but a strong likelihood of undesirable neurological and metabolic side-effects."
NHS England says it is taking urgent measures to tackle the over-medication of people with learning disabilities.