Data Quality / Accuracy
Over the years, data quality has been a constant challenge with almost every industry. The list
of common data quality issues stretches from manual data entry errors to lack of
completeness and from duplicate data to data transformation errors.
O'Reilly, a technology and business training provider, surveyed the state of data quality in
2020.
According to the 1900 survey participants, it was the huge number of growing data sources
that was the single most common data quality issue. More than 60% of respondents selected
'Too many data sources and inconsistent data,' followed by 'Disorganized data stores and lack
of metadata,' which was selected by just under 50% of respondents."
For McKinsey, data quality was a time-consuming challenge.
With real-time data on the rise, what is the answer? One solution -- born out of the O'Reilly
survey -- was the formation of data teams. The survey results suggested that a dedicated
data quality team helped to foster the use of AI-enriched tools. In fact, a dedicated team was
much more motivated to invest in learning to use these tools as well while few analysts and
data engineers had the time or capacity to fully master these tools.
THE NEED FOR SPEED | 12
"Data processing and cleanup can consume more than half of an analytics team's
time, including that of highly paid data scientists, which limits scalability and
frustrates employees. Indeed, the productivity of employees across the organization
can suffer." Respondents to their 2019 Global Data Transformation Survey reported
that an average of 30 percent of their total enterprise time was spent on non-value-
added tasks because of poor data quality and availability.
