Introduction
Collecting more data does not necessarily produce better evidence. When records are incomplete, measurements are inconsistent or errors go undetected, even large datasets can lead to misleading conclusions and poor decisions.
Organisations increasingly rely on data to monitor programmes, evaluate interventions, allocate resources and demonstrate results. Digital surveys, management information systems, administrative databases and dashboards have made it possible to collect and process information at a scale that would have been difficult to achieve only a few years ago.
Yet the ability to collect more data does not automatically translate into a better understanding of reality.
A survey may reach thousands of respondents but contain inconsistent answers. A monitoring system may record activities across multiple locations but count the same participant several times. An administrative database may appear complete while systematically excluding people who face the greatest barriers to accessing a service.
These problems can remain hidden until the data are used to calculate indicators, compare groups, assess programme performance or inform resource allocation.
The consequences can be significant. Poor-quality data can distort performance reports, weaken research findings, misdirect resources and undermine confidence in an organisation’s evidence.
The central challenge is therefore not simply to collect data. It is to ensure that the data are sufficiently accurate, complete, consistent and appropriate for the decisions they are expected to support.
1. What does data quality actually mean?
Data quality refers to the extent to which data are fit for their intended purpose. It is not a single characteristic, and a dataset that is suitable for one analysis may be inadequate for another.
Several dimensions are particularly important.
Accuracy concerns whether recorded values correctly represent the underlying information. An incorrectly entered household size or an inaccurately recorded expenditure amount can distort subsequent analysis.
Completeness concerns whether the required information is present. Missing observations may reduce analytical power or introduce bias when the missingness is systematic.
Consistency concerns whether information is recorded and represented uniformly across records, locations, periods and systems. Different definitions of an indicator can make comparisons misleading.
Validity concerns whether data conform to the rules, ranges and definitions established for collection. For example, an age field containing an impossible value may indicate an entry error, although unusual values should be investigated rather than automatically deleted.
Uniqueness concerns whether records that should represent distinct observations have been duplicated. Duplicate records can inflate participation figures or distort summary statistics.
Timeliness concerns whether data are available and sufficiently current for their intended use. Accurate information delivered too late may be of limited value for an urgent management decision.
Relevance concerns whether the information collected actually addresses the question being investigated.
These dimensions are related but distinct. A dataset can be complete but inaccurate, consistent but irrelevant, or timely but insufficiently validated.
Data quality assessment should therefore begin by identifying how the information will be used and which quality dimensions matter most for that purpose.
2. How poor data quality develops
Data quality problems rarely originate from a single source. They can emerge at any stage of the data lifecycle, from initial design and collection to processing, analysis and reporting.
Poorly defined indicators and questions
When concepts are not clearly defined, different data collectors may interpret the same question differently.
For example, a programme may seek to measure the number of people who have received business advisory services. If one location counts every advisory session and another counts unique individuals, the resulting figures are not directly comparable.
The problem is not necessarily the data collection tool. It is the absence of a consistent operational definition.
Inadequate data collection procedures
Errors can arise when questionnaires are poorly designed, enumerators receive insufficient training, skip patterns are incorrect or respondents misunderstand questions.
Digital collection tools can prevent certain mistakes through validation rules and required fields. However, they cannot automatically ensure that respondents understand the questions or that enumerators record truthful and accurate information.
A completed form is not necessarily a correct form.
Inconsistent identifiers and duplicate records
When individuals, households, businesses or facilities cannot be identified consistently, the same entity may appear multiple times in a database.
This can inflate counts, complicate follow-up and undermine analysis of change over time.
Duplicate detection is not always straightforward. Two records with similar names may represent different people, while records with different spellings may refer to the same person. Reliable identification therefore requires appropriate identifiers, documented matching rules and careful review.
Weak data management and integration
Organisations often collect information through several systems, spreadsheets and reporting templates. Differences in variable names, coding conventions, dates, geographic identifiers and indicator definitions can introduce inconsistencies when these sources are combined.
Without clear documentation and version control, analysts may also work with outdated datasets or apply different transformations to the same information.
Inadequate quality assurance
When data are checked only at the end of a reporting period, errors may be difficult or expensive to correct.
For example, an invalid location code discovered months after fieldwork may require contacting the original data collector or revisiting source records. If the source information is no longer available, the error may remain unresolved.
Quality assurance is more effective when built into the entire process rather than treated as a final cleaning exercise.
3. Why large datasets can still produce misleading findings
Large datasets can create a false sense of confidence. When thousands of records are available, users may assume that the results are automatically reliable.
However, sample size does not eliminate systematic error.
Suppose a programme reports that 8,000 people received a service. If the records include substantial duplication, the number of unique recipients may be considerably lower. The reported figure may be calculated correctly from the database while still misrepresenting the programme’s reach.
Similarly, a large survey can produce precise estimates of a poorly measured concept. If respondents interpret a question differently, increasing the sample size will not necessarily correct the underlying measurement problem.
Selection problems create another challenge. If people who cannot access a service are also less likely to appear in the programme’s records, analysis of those records alone may overstate how accessible the service is to the wider target population.
These examples illustrate an important distinction: computational accuracy is not the same as evidential reliability.
Statistical software can calculate a mean, estimate a regression or generate a dashboard correctly while using data that do not adequately represent the phenomenon under investigation.
Reliable analysis therefore requires examining how the data were generated, what they represent and which limitations affect the conclusions that can reasonably be drawn.
4. Data cleaning is necessary, but it is not enough
Data cleaning is an important part of data management. It involves identifying and addressing problems such as inconsistent formats, invalid entries, duplicate records and missing values.
However, cleaning should not become a process of making a dataset appear complete or convenient.
Consider a household expenditure survey in which several respondents report unusually high spending. An analyst may suspect that the values are errors and remove them before calculating average expenditure.
But the values may represent genuine differences in household circumstances. Removing them without investigation could systematically understate expenditure among particular households.
Likewise, replacing missing values with averages may simplify an analysis while concealing meaningful patterns in the missingness.
A defensible data cleaning process should follow several principles.
First, investigate before changing. Identify unusual values and determine whether they reflect errors, valid observations or limitations in the collection process.
Second, document every material transformation. Record the original issue, the rule applied, the justification and the effect on the analysis where relevant.
Third, preserve the source data. Retain an unchanged original dataset and conduct cleaning on a separate working copy or through a reproducible processing workflow.
Fourth, assess the implications of missing data. Examine how much information is missing, whether missingness varies across groups and whether the analytical approach appropriately accounts for it.
Finally, validate the cleaned dataset. Confirm that the changes have resolved the identified problems without introducing new inconsistencies.
The objective is not to produce perfectly tidy data at any cost. It is to produce a transparent, defensible dataset that remains faithful to the underlying observations.
5. Build data quality into the entire data lifecycle
The most effective approach to data quality is preventive rather than purely corrective. Organisations should establish quality controls at each stage of the process.
Before collection: define and design
Begin by clarifying the purpose of the data, defining indicators and documenting the required variables.
Questionnaires should use clear wording, appropriate response options and logical skip patterns. Data collection tools should be piloted under realistic conditions, and enumerators should understand the purpose of each question.
Where digital tools are used, validation rules should flag implausible values, enforce appropriate constraints and reduce avoidable entry errors.
During collection: monitor and correct
Field teams should receive regular feedback on completeness, unusual response patterns, interview duration, geographic coverage and other relevant quality indicators.
Supervisors should investigate anomalies rather than relying exclusively on automated alerts. A short interview, for example, may indicate a problem, but it may also reflect a legitimate difference in interview circumstances.
Early checks allow teams to correct misunderstandings and collection problems before they affect large numbers of observations.
During processing: validate and document
Data processing should include checks for missingness, duplicates, inconsistent coding, invalid values and unexpected patterns.
When datasets are combined, analysts should confirm that records use compatible definitions, identifiers and units of measurement.
Changes should be documented so that the path from the original observations to the analytical dataset remains transparent and reproducible.
During analysis: test robustness and interpret carefully
Analysts should examine whether findings change under reasonable alternative assumptions or data treatment decisions.
Where missingness, measurement error or possible selection bias could materially affect results, these limitations should be assessed and reported.
Conclusions should reflect the quality and scope of the available evidence. A dataset with substantial limitations may still be useful, but its limitations should inform how confidently its findings are interpreted.
During reporting and use: maintain accountability
Reports and dashboards should present indicators with clear definitions, relevant time periods and appropriate denominators.
Users should understand what the figures represent, where the data came from and which limitations affect interpretation.
Data quality is not solely the responsibility of the analyst. Programme managers, data collectors, system administrators and decision-makers all influence how information is generated, maintained and used.
6. A practical data quality review checklist
Before using a dataset for reporting, research or decision-making, organisations can conduct a structured review.
- Purpose: Is the dataset appropriate for the question being investigated?
- Definitions: Are variables and indicators clearly defined and consistently applied?
- Completeness: How much information is missing, and does the pattern of missingness matter?
- Accuracy and validity: Are values plausible and supported by the available source records?
- Duplicates: Are records unique where they should be, and are matching decisions defensible?
- Consistency: Do dates, codes, units and categories align across records and data sources?
- Representativeness: Does the dataset adequately cover the population or cases about which conclusions will be drawn?
- Documentation: Are the data sources, collection procedures, transformations and limitations recorded?
- Reproducibility: Can another analyst understand and reproduce the steps used to prepare the data?
- Decision relevance: Are the remaining limitations acceptable for the intended decision?
The review should be proportionate to the risks involved. A preliminary internal monitoring summary may require a different level of assurance from a national evaluation, a public performance report or an analysis that informs the allocation of substantial resources.
Where data quality is insufficient, the appropriate response may be to correct the data, collect additional information, revise the analytical approach, qualify the findings or reconsider whether the intended decision can be supported.
7. From data quality to organisational confidence
Data quality is often treated as a technical issue assigned to data officers or research teams. In practice, it is also a management and governance concern.
Organisations need clear responsibilities for data collection, validation, access, correction, documentation and approval. They also need realistic timelines and budgets that allow quality assurance to take place.
A system that rewards rapid reporting but provides no time for verification can encourage the production of numbers without adequate scrutiny. A system that treats identified errors as opportunities for improvement can support more reliable reporting and better institutional learning.
Data quality also requires proportionality. Not every imperfection makes a dataset unusable, and not every analytical task requires the same level of precision. What matters is understanding which errors could change the interpretation of the results or lead to a poor decision.
Confidence in evidence is built when users can trace where the information originated, understand how it was processed and judge whether the conclusions are justified.
That confidence cannot be created by a dashboard, a large sample or sophisticated software alone. It depends on sound design, disciplined implementation, transparent analysis and responsible interpretation.
Conclusion: Better decisions begin with trustworthy data
Data can help organisations understand programme performance, identify emerging problems, assess inequalities and allocate resources more effectively. But the value of that information depends on the quality of the processes through which it is collected, managed, analysed and interpreted.
More observations cannot automatically correct biased selection, inconsistent measurement or systematic recording errors. Nor can statistical sophistication compensate for data that do not adequately represent the problem being investigated.
A stronger approach treats data quality as a continuous responsibility. It begins with clear definitions and sound collection procedures, continues through validation and documented processing, and extends to analysis, reporting and decision-making.
The goal is not simply to produce clean datasets. It is to establish whether the available information is sufficiently trustworthy for the conclusions being drawn and the actions being considered.
At DAREL Consulting, we connect data quality, research, analytics and evaluation to help organisations move from collecting information to generating credible evidence. Because better decisions require more than more data. They require confidence in what the data actually tell us.
