Data quality issues in the public sector

The public sector collects a lot of data.  Huge quantities.  But how good is it?  It is used by management and central government to monitor workloads and critical success factors.  Important policy decisions are taken, budgets are set, all based on the ‘data’ (and the information managers think they gleen from it), but should it be trusted?

Data quality in the public sector is a huge issue.  The old adage “garbage in, garbage out” is as true today as it ever was.

Of course, this is not a problem solely for the public sector, but perhaps the unique culture of local authorities makes it harder to remedy.  There may be less impetus to put measures in place to get to the root of the problem.

Sometimes rather than tackle the data quality issues at source, a team of quality checkers are put in place to check data quality, clean the data and correct mistakes.  This means that the originator of the data, the team or person, who input it, knows that data quality is not their responsibility.  This team may have targets of how many data items are added daily, but not the quality of them.

Departments and organisations may lack a sense of collective responsibility – staff are rarely reminded that they are all working towards the same objectives (looking after vulnerable residents perhaps) and that they should work together to achieve this.  A blame culture can develop where one team blame another for the low data quality.

Or more likely managers turn up their hands and think there is nothing to be done.

Examples from a local authority might be:

  • Customer service staff not appreciating that the failure to double check a postcode on a customer record could mean that a member of staff in another department was sent on a fool’s errand to the wrong address, costing the council time and money whilst also providing poor customer service and adversely affecting the council’s reputation. It is such a simple thing, but regularly missed and has tangible impacts.
  • Several teams fail to add relevant information on a council tax or housing benefit record to enable debt collection in court later. It is not relevant to them, but it is vital to the Debt Recovery team and potentially means debt being written off.
  • Social workers are reluctant to fully complete a case record beyond what they personally need to manage the case, however a few seconds more, adding only two or three more pieces of information could save hours of data cleansing work for vital management reporting.

By employing extra staff to clean the data, we are solving the wrong problem, we are adding to it.

So what is the problem and how can it be solved?

  1. Entering clean data takes longer and this can feel onerous
  2. It is human nature to do as little as possible. Many public sector organisations don’t have a culture of “doing the right thing first time”; the culture is more likely to be “it’s not my job”.
  3. Managers who realise there is a problem; often try to fix it by providing a remedy, such as data quality teams to correct the issues – which is not the right approach. Managers need to look for the root cause of the problem (or get a business analyst to do so for them).
  4. It is possible that the cause of the problem is a lack of responsibility at the point of data entry. If the staff member checked the work at that point, there would be no problem.  It would be correct, first time.  However, they know that someone else is going to fix it.  In some cases staff may not realise how important a particular field is for someone else in the organisation.  If something is unimportant for their own job, they may omit the data.  It needs to be stressed to these staff the importance of completing the task.   A few seconds more could equate to an hour for someone trying to sort out the data at the other end, this is costly.  The process needs to be transparent and joined up from end to end.
  5. It should be part of everyone’s job to enter clean data. Ensure the manager demonstrates that clean data is important.  Doing it right the first time, even if it takes longer, is always, the least resource in the long run.
  6. The manager / supervisor really need to understand the importance of staff being allowed enough time to enter clean data and to impart that to staff every day.
  7. Staff need to be supported and trained adequately.  If targets have been set around numbers of cases logged a day (for instance) staff should be allowed to spend the necessary time to add the data correctly (often not much longer).  A change in mind-set from the manager from quantity to quality, will effect a change in mind-set for the team / department.
  8. Data entry screens can be complicated and this should be analysed to ensure that data entry is as simple as possible.
  9. Sometimes data is being duplicated in different places. At present staff can feel their data entry time is wasted if they are double or even triple logging data.  These issues need addressing.
  10. If members of staff are responsible for adding clean data, then it becomes the data quality teams role to help them, not to correct them. An organisation may still need someone who randomly quality checks and who compiles reports, but their role would change (and far fewer would be required).   These staff would work with the managers to understand the issues and cooperatively work on ways to address them.
  11. It is vital that the organisation changes to a culture of collective responsibility, everyone working towards a common goal where jobs complement each other and all staff know, whatever level they are at in the organisation, that they are all working towards the same objective and that it is vital to ‘get it right first time’. It is public money; we have a duty not to waste it.

Working towards the same aim…

At some councils, every member of staff know how their job; however back office it is; affects the residents.  An IT project manager for Children’s Services knows that their job facilitates the education of the children in the area for instance.  This is similar to the anecdotal story about the cleaner (janitor) at NASA who told President Kennedy that his job was “putting a man on the moon”.  Only 20% of employees have such a belief in the purpose of their work, but it makes such a difference in delivery.


Leave a Reply


You are commenting using your account. Log Out /  Change )

Google photo


Twitter picture


Facebook photo


Connecting to %s
