A study in the BMJ looked at GP data spanning over a decade and found that more than a quarter of 33,000 adults with learning difficulties had been prescribed antipsychotics, often with no obvious clinical justification.
The drugs are designed to treat severe mental illness, not tricky behaviour.
NHS England has already warned prescribers about the problem.
In July, it sent a letter to patients and professionals saying these powerful medicines should not be used as a "chemical restraint".
A report by Public Health England estimates that up to 35,000 adults with a learning disability are being prescribed an antipsychotic, an antidepressant or both without appropriate clinical justification.
NHS England advises: "If you are worried, either for yourself or someone you know, about the medicines being taken, speak to the person responsible for prescribing them. This will usually be a GP, psychiatrist, specialist doctors, pharmacist or nurse prescriber."
It says medicines used to treat mental illness can be very effective in treating some people with learning disabilities when used appropriately.
People with learning difficulties in the UK are being inappropriately over-medicated, patient records suggest.