How to improve your data quality

Organizations lose an average of $12.9 million per year due to poor data quality. In addition to the immediate impact on revenue, poor data quality increases the complexity of the data ecosystem in the long run, leading to poor decision making.

As enterprises increasingly use data analysis to help drive business decisions, they are increasingly paying attention to the data quality (DQ) in their systems. Gartner predicts that by 2022, 70% of organizations will use indicators to strictly track data quality levels and improve data quality by 60%, thereby significantly reducing operational risks and costs.

Data quality is directly related to the quality of decision making. High-quality data can provide better customer leads, deeper understanding of customers, and better customer relationships. Data quality is a competitive advantage that data and analytics (D&A) leaders need to continuously improve.

1.Determine how data quality improvements will impact business decisions

Identify clear links between business processes, key performance indicators (KPIs), and data assets. List existing data quality issues facing the organization and how they impact revenue and other business KPIs. After establishing a clear link between data as an asset and improvement requirements, data and analytics leaders can begin to develop a targeted data quality improvement plan with a clearly defined scope, a list of stakeholders, and an initial investment plan.

2.Defining the criteria for “good enough” data

To improve data quality, first understand how the organization defines “best fit” data quality. The organization is responsible for defining what is “good.” Data and analytics (D&A) leaders need to stay on top of expectations by having regular discussions with business stakeholders. Different lines of business using the same data (e.g., customer master data) may have different standards and, therefore, different expectations for data quality improvement initiatives.

3.Establish enterprise-wide data quality standards

Data and analytics leaders need to establish data quality standards that all business units in the organization adhere to. Different stakeholders within the enterprise are likely to have different business sensitivities, cultures, and maturity levels, so they may meet data quality implementation requirements in different ways and at different speeds.

This will enable stakeholders across the enterprise to understand and execute their operations as per defined and agreed upon data quality standards. Enterprise-level data quality standards will help educate all stakeholders and enable seamless adoption.

4.Use data profiling early and often

Data quality profiling is the process of examining data from existing sources and summarizing the information in the data. It helps determine the corrective actions that need to be taken and provides valuable insights that can be presented to the business to drive the ideation process for improvement initiatives. Data profiling helps determine which data quality issues must be addressed at the source and which can be addressed later.

But this should not be a one-time activity. Organizations should perform data profiling as often as possible, depending on resource availability, data errors, etc. For example, analysis may reveal that some key customer contact information is missing, which may directly lead to a large number of customer complaints and make it difficult for the organization to provide good customer service. At this point, data quality improvement activities will be a very high priority.

5.Design and implement data quality dashboards for monitoring key data assets such as master data

The data quality dashboard provides all stakeholders with a comprehensive snapshot of data quality, including past data, to help design future process improvements by identifying trends and patterns. It can be used to compare the performance of data critical to key business processes over a period of time, enabling organizations to make the right business decisions and achieve expected business goals based on reliable, high-quality data.

The data quality dashboard also reflects the impact of improvement activities, such as incorporating new data practices into operational business processes. The dashboard can be customized to meet the specific needs of the enterprise and display the trustworthiness of the data.

6.Moving from a reality-based semantic model to a trust-based semantic model

Data does not always originate internally where data quality can be controlled and maintained from the outset. In some cases, data assets originate externally where data quality rules, authors, and levels of governance are often unknown. Therefore, a “trust model” is better than a “truth model”.

This means that critical enterprise data is not absolute, and organizations must also consider its origin, jurisdiction, and governance, and therefore its usability in decision making. When trustworthiness cannot be maintained, data and analytics leaders can take mitigating measures.

7.Put data quality on the agenda for data and analytics governance committee meetings

Data and analytics leaders need to tie data and governance initiatives to business outcomes, which will help track investments in data governance improvements against business goals. To gain the attention of the governance committee, the impact of data quality improvements must be communicated to the governance committee in the language they understand best: impact on business and revenue. The governance committee needs to have a clear understanding of the data quality improvement process and challenges, and they need to be informed of this information on a regular basis.

8.Define the data quality responsibilities and operational procedures of the data steward role

The data steward is responsible for ensuring the quality and fitness for purpose of the organization's data assets, including the metadata for those data assets. In mature organizations, the data steward role also includes advocating for good data management practices and monitoring, controlling, or escalating data quality issues when they occur.

Data analytics leaders need to embed this role into their data and analytics strategy so that data quality is measured and maintained systematically and regularly. A governance scope and stakeholder map should be created to provide a clear understanding of how data quality issues will be managed.

9.Establish a data quality task force across business units and IT under the leadership of the Chief Data Officer team or equivalent

Organizations can invest significant time and resources in forming a dedicated team comprised of representatives from the business, IT, and the office of the chief data officer to collaborate on improving data quality. Such collaboration can enable organizations to better manage risk while also creating more opportunities to reduce operating costs and promote growth through shared, unified best practices.

10.Establish data quality review as a "gateway" for release management

Conduct timely corrections and inspections through review and update progress, and identify and disseminate impactful best practices as the organization increases in maturity in addressing data quality initiatives.

11.Regularly communicate the benefits of improved data quality to the business

Data and analytics leaders need to measure the impact of improvement initiatives and communicate results regularly. For example, customer service executives can serve customers better and faster because they have high-quality, trusted data, so every 10% improvement in customer data quality will increase customer responsiveness by 5%.

Data and analytics leaders must not only draw the attention of the governance committee to data quality improvements, but also make it a long-term practice and, more importantly, regularly communicate the benefits to the governance committee.

12.Leverage external/industry peer groups, such as user groups from vendors, service providers, and other established forums

Data and analytics leaders can connect the enterprise with data quality peer groups and promote maturity in the organization in this area, enabling both parties to exchange additional perspectives on best practices and insights into how others have addressed similar challenges.