Unified Metamodel for Data Executives

Executive Summary

This webinar covers the importance of metadata, understanding FAIR, and developing a CDO self-measurement tool for effective data access and management. Howard Diesel makes the case for integration, shared understanding, and trust in data management, and works through change control, effective knowledge management, and learning measurement. It also covers fair data principles, advanced data management practices, and the challenges and principles of open data, along with the different types of analytics and decision-making approaches and strategies for effective business decision-making. Howard closes by stressing how important comprehensive assessments are for understanding organisational progress and decision-making.

Webinar Details

Title: Unified Metamodel for Data Executives
Date: 05 October 2023
Presenter: Howard Diesel
Meetup Group: Data Executives
Write-up Author: Howard Diesel

Unifying Data for Effective Access and Measurement

Howard outlined the challenge of unifying data from various sources into a comprehensive view, and the need for a self-measurement tool for CDOs tasked with implementing data management offices. He was after advice on how CDOs can effectively measure and communicate their progress, sharing an example of using statistics and graphs to report it, and also considered new measures like data literacy, AI literacy, governance literacy, and number of staff trained as indicators of CDO progress. He closed with an anecdote about setting up a sales leaderboard in a hydraulic supermarket, which was initially met with excitement but led to some employees manipulating sales data once the measurement was actually implemented.

Figure 1 DM Unified Metamodel Deep-Dive

Figure 2 Data Executive Requirements

The importance of metadata and understanding FAIR

The challenge of accurately measuring humans came up early on, and Howard stressed how important comprehensive data management really is. He worked through metadata and its various roles, including denoting entities and serving as a “love note to the future” about the data, and pointed to the growing value of metadata collection in the age of generative AI. He then introduced FAIR, Findability, Accessibility, Interoperability, and Reusability, and was curious how familiar the audience already was with it.

Figure 3 Metadata Definitions

Developing a CDO Self-Measurement Tool

Howard discussed how established FAIR is in the scientific community, and how absent it is in business. He’s been tasked with creating a self-measurement tool for a Chief Data Officer to track their data management office’s progress, and identified sustainable data management capabilities like maintaining knowledge management and embedding data management into development processes. He also pointed to the common integration gap between data quality and business intelligence departments, and the need for guidance on how different DMBOK capabilities can actually work together.

Figure 4 What does GOOD look like?

The Importance of Integration, Shared Understanding, and Trust in Data Management

Effective data management matters a great deal for businesses to ensure information stays accurate and reliable, but the lack of an overall system gets in the way of integrating different components of data management. Overcoming that takes shared understanding and common behaviour, understanding data models and business glossaries included, and trust in the data matters too, since decision-making depends on that data being accurate and reliable. Communication plays a big role here, keeping information effectively shared and easily accessible, and findability matters just as much, requiring clear, accessible documentation rather than a vague “it’s on Confluence”.

Change Control, Effective Knowledge Management, and Measurement of Learning

The webinar covered several aspects of organisational management: change control, effective communication, and employee understanding among them. The Unified Metamodel is a genuinely important part of that, giving easy access to information, and maturity assessments have shown organisations often lack evidence, policies, procedures, and data strategies, which can hurt overall performance.

Change management, knowledge management, and collaboration all matter for organisational success, though cultural differences can complicate implementation. Learning measurement should go beyond tracking hours too, to include recognised learning and value actually delivered, and the “one metric that matters” approach, Spotify’s listening hours, for instance, can be a genuinely meaningful gauge of organisational performance and progress.

Figure 5 Integrated, Embedded and Sustainable DM Capabilities

Figure 6 Measurement scorecard OMTM (The One Metric That Matters): Trustworthiness

Improving Listening Hours for Customers

Data management and business decision-making culture came up as key themes. Howard stressed the importance of understanding each department’s contribution to the overall data picture, building trust in data for informed decisions, and addressing data trustworthiness directly. He also pointed to how effective data management drives ROI, predictable deliverables, unified and aligned data, and data maturity. Building a data-driven culture takes time and depends on trust in the data, and encouraging people to actually make data-based decisions is what creates a genuinely data-driven culture in a business.

Figure 7 Climbing the Ladder

Figure 8 Measurement Metrics

Figure 9 Trusted Data Cake Layer Metrics

Figure 10 Trusted Data Cake Layer Metrics Continued

Decision-Making and Hierarchy in Business

Effective decision-making matters a great deal in business, and understanding its hierarchy matters just as much, lower-level decisions tend to be easier to make than high-level strategic ones. Building that hierarchy means rolling up numbers and analysing the decision-making process, and ratios get used here too, distributed across multiple metrics, though different areas won’t necessarily contribute equally to the final metric that matters.

There are two domains worth considering: culture and change management, and decision-making itself. Quantifying each decision-making level is genuinely hard, since subjective opinions need replacing with actual scores, but understanding that process and its hierarchy is what makes decision-making effective in business.

Figure 11 Trusted Data Cake Layer

Data Management Principles and Scales

The office assesses employees both quantitatively and qualitatively using six scales: trustworthiness, propensity to trust, the fair data maturity model, data management, data culture, and strategic alignment maturity model. The strategic alignment maturity model measures how well strategies actually align with data and business.

The Australians have principles called Fair and Care that focus on data management for data and for people respectively, and working with DMMA involves six scales too: strategic alignment, collaboration, business decision-making, fair, and change and data management metrics. The principles of open and fair, lastly, focus on purpose and data respectively.

The Importance of FAIR Data Principles

FAIR principles treat data and metadata equally and tie them to proper provenance, emphasising applicable language, qualified references, and evaluating a dataset’s capabilities. They ensure datasets are findable, accessible, interoperable, and reusable, and while they originated in academic publishing, they’ve proven effective for correcting mistakes and improving measure implementation across various domains, a good demonstration of what accessible data and metadata can do in research and beyond.

Figure 12 FAIR Data Principles

Impressed by Advanced-Data Management Practices and the FAIR Framework

Howard was genuinely impressed with the researchers in Dublin for their advanced understanding and measurement of information, noting how much arduous work effective data management, metadata included, actually takes. The Dublin researchers have made real strides in taxonomies, knowledge graphs, crosswalks, and canonical models, and the FAIR framework, which prioritises data survival and understandability, came up as a result. Universities now have to submit publications per the FAIR model, spanning maturity levels from single-use data through identifiable data, described standardised data, and systematically typed data, with a regulation on the data management repository rounding things out.

Figure 13 FAIRification Framework

Figure 14 FAIRplus Dataset Maturity (DSM) Model

Figure 15 DSM Maturity Levels

Challenges and Principles of Open Data

Storing data somewhere reliable matters a great deal for ensuring future access, regardless of whether the software itself gets discontinued. Various formats came up for expressing and describing data, PDF, Excel, CSV, RDF, linked open data, and linked open data is the recommended choice for facilitating efficient processing. The open data maturity level covers principles around indigenous representation, government-level governance, and the benefits commercial entities get from open data publishing, and fair principles built around data, people, and purpose highlight different aspects of open data more broadly.

Figure 16 FAIR Data CMM

Figure 17 Data Principles: People, Purpose and Data-Oriented

Reflecting on the Use and Ethical Considerations of Data Sets

Managing and using data sets fairly and ethically matters a great deal, and Howard stressed the importance of keeping data sets findable, accessible, interoperable, and reusable (FAIR) across different formats. An Excel template for assessing FAIRness lets data sets get ranked by essentiality, priority, importance, and usefulness, with that priority ranking helping assess evidence and determine compliance. Howard also acknowledged the real progress made in data management and metadata within the research community, and pointed to how much metadata and knowledge matter for AI systems, along with the challenges generative AI still faces in understanding and categorising information accurately.

Figure 18 Open, FAIR & CARE

Figure 19 FAIR Assessment

Observations on Collaboration and Decision-Making Process

Howard discussed how important it is to shape information properly for trustworthy, precise decision-making, even where machines are involved. He introduced a collaboration scale that starts with communication and progresses through coordination, cooperation, and collaboration, stressing how important it is to actually measure collaboration levels. He presented a “network, cooperate, coordinate, share, and understand” process for building that collaboration, noting that decision-making tends to slow down as collaboration deepens, since consensus takes time, and acknowledged that things get even more complex once multiple internal and external parties are involved.

Figure 20 Measuring Collaboration

Figure 21 Collaboration Scale

Figure 22 Scales of Collaboration

OODA Process and Decision-Making Techniques

The OODA process, developed by fighter pilot John Boyd, runs through observing, orienting, deciding, and acting, and gets used in decision-making with root cause analysis helping understand both wins and losses. Effective communication with business departments matters here too, and an assessment has been built to evaluate department collaboration, communication, and role clarity. Decision-making sometimes calls for being authoritative and sometimes for seeking consensus, depending on the situation, and while faster decision-making is often preferred, weighing other factors matters just as much for making genuinely better decisions.

Figure 23 Collaboration Assessment

Digital Decisioning Strategy and Process

Implementing digital decisions means carefully deciding which decisions actually suit machine-made choices, which takes a comprehensive inventory of all decisions and an honest assessment of the information available on each. The OODA loop process helps here, working through key questions: what happened, why it happened, what will happen, what should be done, and what’s still unknown. Index weights can prioritise decisions, and different types of analytics can help answer these questions, a process that shows up commonly throughout the DMBOK.

Figure 24 Dynamic Data-Driven Knowledge (D3K)

Figure 25 Digital Decisioning Strategy and Process

Understanding the Different Types of Analytics and Decision-Making Approaches

Decision-making draws on several different approaches and tools. Traditional Business Intelligence answers what happened and why, Predictive Analytics predicts likely outcomes, and Prescriptive Analytics suggests actions to reach desired outcomes. Operational Research focuses on finding the most effective approach, while Blind Spot Analysis looks at what might get missed due to uncertainty.

The KFAN diagram measures uncertainty and helps categorise decisions into quadrants based on how well cause and effect are understood. Digital decision-making suits situations with clear cause-and-effect relationships, though drift in understanding can lead to bad decisions.

Chaos represents emergencies where data is scarce and decision-making comes down to assigning someone or a team to just make the call, and in genuine disorder there’s no clean answer, the decision itself may need unmaking. Decisions need treating according to the level of information and understanding available, since not all of them can be handled the same way, unknown unknowns, known unknowns, and known knowns all call for different approaches.

Figure 26 Context Frame: North Star & OMTM Layers

Figure 27 D3K Scale

Figure 28 Knowledge-Driven (Smart) Business Decisions

Figure 29 Understanding the Different Types of Analytics and Decision-Making Approaches

Strategies and Methods for Effective Decision-Making

Effective decision-making draws on a range of approaches and considerations. Confirmatory and exploratory analytics both matter for analysing decisions, and classifying decisions by their nature helps guide the process too. Seeking too many opinions can actually backfire, particularly around scheduling and training costs, and research shows that prompt, well-informed decision-making is part of what drives success in top companies.

Decision-making gets harder when data is lacking or the team disagrees, though, and different approaches, the theory of constraints or the “just do it” approach, for instance, suit different situations. Choosing the right method for each type of decision matters, and sometimes the right call is not deciding at all until there’s enough information. The “OO” Loop involves observing market trends and competitor activity, with ongoing observation, decision-making, and adaptation needed to keep up with market change.

Figure 30 State of Knowing for Business Decisions

Figure 31 Business Decision Method

Quick decision-making and strategy alignment in business

The Orient approach means identifying competitors’ unique strategies and checking for gaps in your own. Timely decision-making matters for staying ahead of competitors and avoiding getting stuck cycling between observation and orientation, and the OO scale quantifies how many opportunities get missed for every successful decision made.

The PDCA framework helps analyse and understand those missed opportunities and improve decision-making from there. Strategy alignment matters too, keeping business strategies tied to overall business objectives, and capability and data life cycle assessments can identify areas for improvement in business processes and strategy that help reach those broader goals.

Figure 32 OODA Loop Measurement

Figure 33 PCA/OODA Assessment

Comprehensive View of Program Operation

A comprehensive view of a program’s operation matters for assessing its effectiveness and progress properly. Focusing on just one aspect risks a biased view, and a purely technical answer won’t necessarily capture how well a program is actually performing. How well data supports decision-making matters too, and decision-making speed can vary between individual teams and the organisation as a whole. Combining various perspectives and approaches in program management is really what gets you a well-rounded view.

Different Types of Organisations and Decision-Making

Howard introduced different types of organisations: inventory-oriented, customer-oriented, and product-oriented. Inventory-oriented organisations focus on inventory terms and operate through centralised decision-making, while customer-oriented organisations let ground-level employees make decisions while interacting with customers directly.

Product-oriented organisations prioritise creating quality products that lead the market. Howard stressed the importance of decision-making processes and how hard it is to eliminate subjectivity from them.

Importance of Data and Maturity Assessments in Decision Making

Data matters enormously for decision-making and reaching organisational goals, and understanding the cause-and-effect relationship, along with identifying known knowns, unknown knowns, and known unknowns, is central to effective analysis. Decision-makers often overlook how much information quality affects their decisions, though. Tom Redman stressed embedding data and people culture into the organisation, and Howard shared his own experience doing maturity assessments in South Africa, where organisations are often sceptical of assessments, seeing them as a way to extract more money without any tangible benefit.

Importance of Comprehensive Assessments in Understanding Organisational Progress

Assessing an organisation’s performance properly means looking at multiple elements rather than relying on a single assessment or maturity check. The goal is really educating CDOs on how to do their jobs well. Not unlike the difficulty of building a logical data model, it takes gathering the right people, understanding the organisation’s current state, and making the assessment process genuinely enjoyable. Organisations often make the mistake of asking for time estimates on reaching a certain maturity level, though progress varies enormously and can take years to reach the higher levels.

Importance of Assessment before Project Quoting

Assessment matters a great deal before quoting on a project, helping understand the organisation’s requirements and surface potential hurdles to effective delivery. Breaking the assessment process into smaller components, the DMAA framework (Define, Measure, Analyse, and Act), for instance, helps quantify effort and gives a clearer sense of the project’s feasibility. Determining a project’s success ultimately takes assessing a wide range of aspects rather than leaning on a single measure.