The Marie Curie charity and the Bevan Foundation looked at figures from 2012, which also showed one in four hospital beds were used by patients in their last year of life in Wales
Older terminally ill people were much less likely to have palliative care.
The Welsh government said there should be equal access to good care.
The report highlighted "huge numbers" of admissions to hospitals and days spent in hospital by people in the last year of life.
Out of all people who died in 2012, only a total of 17% accessed palliative care services.
Of people with cancer aged over 85, only 35% used the services, compared to 58% of those aged up to 44.
The report recommends the Welsh government and NHS Wales:
Simon Jones, head of policy at Marie Curie Wales, acknowledged that Wales had a higher level of specialist palliative care services than England or Northern Ireland, saying improvements had been made including better medicine, round-the-clock clinical nurse specialists and consultant support.
However he added: "Our report is a stark reminder of the work that still needs to be done to ensure that all those who need and would benefit from specialist palliative care get it.
"For example the recorded level specialist palliative care given to people dying from dementia is as low as 3.8%."
He said while some people would need the specialist services only available in a hospital, many could be cared for in other settings including hospices and patients' own homes.
Bevan Foundation director Victoria Winkler said: "Our analysis doesn't only reveal the short-fall in provision of specialist palliative care, it also shows that the NHS has a rich source of data at its fingertips which it can use to gain insight into its activities."
A Welsh government spokesman said it announced Â£6.4m funding in August to support hospices and palliative care across Wales.
"No matter where you live in Wales, people approaching the end of their lives should have access to consistently good care and support," he said.
"With advances in modern medicine and treatment, more people live longer with incurable diseases."
Less than half of people who died from cancer and only 5% of those dying from other conditions received specialist palliative care, a report has found.