The Conceptual Modelling Journey Map for Data Managers

Executive Summary

This webinar outlines key topics and challenges in developing a robust Data Modelling course. It focuses on the critical aspects of data integration, governance, and organisational strategy. Howard Diesel emphasises the significance of defining concepts in database design, covering essential frameworks such as the Zachman Framework and the development of system models.

Lastly, the webinar addresses the necessity of choosing the right database schema and outlines the distinction between Relational and Dimensional data models. Howard then highlights the role of data managers in navigating the complexities of conceptual modelling and the importance of clarity for effective implementation.

Webinar Details:

Title: The Conceptual Modelling Journey Map for Data Managers
Date: 08 August 2024
Presenter: Howard Diesel
Meetup Group: Data Managers
Write-up Author: Howard Diesel

The Challenges of Developing a Data Modelling Course

Howard opens the webinar with mention of his experience so far of developing a conceptual modelling course aimed at data citizens and higher-level individuals. This, he shares, is in response to requests for a more focused Data Modelling curriculum. The course will address key real-world challenges, particularly in stakeholder alignment and the establishment of a sustainable implementation framework, which includes principles, policies, procedures, and technology relevant to Data Modelling. Howard also shares that he is interested in gathering insights on the technologies currently used for Data Modelling, especially beyond conventional tools, to enhance our approach to validation and accuracy.

Figure 1 Title Slide

Figure 2 The Conceptual Modelling Journey Overview

Figure 3 Conceptual Modelling: Learning Path

Figure 4 Data Manager Journey Map

Data Modelling and The Challenges of Data Integration

The operating model for data management involves clearly defined roles and responsibilities related to Data Modelling, including identifying who provides inputs to, supplies, and consumes the data models. It’s crucial to ensure that Data Modelling is scalable and sustainable to adapt to evolving business needs.

Continuous revisiting and updating of the data model are essential, particularly after system changes. Effective data integration poses significant challenges, especially when dealing with data from multiple sources; having a clear model of the data and target systems is vital for successful integration. Addressing these aspects is critical for any data manager.

Data Modelling in the Organization

The key takeaway for data managers is the importance of understanding and communicating the benefits of conceptual modelling within an organisation. Many organisations seem to show a decreased interest in Data Modelling. Data Modelling focuses on the structure and relationships of data which distinguishes it from economic or statistical modelling.

Data Modelling in Data Governance

Data Modelling plays a crucial role in Data Governance by providing a foundation for effective communication and shared understanding among stakeholders. This involves writing clear business definitions and creating conceptual models that enable informed, data-driven decision-making for data stewards and professionals alike.

Howard shares that the upcoming discussions will focus on enhancing data literacy, exploring advanced modelling techniques such as data vault and temporal modelling, and ensuring compatibility between models and master data. Additionally, there will be a focus on how to effectively utilise models for data analytics and insights, ultimately driving business value and fostering innovation.

Figure 5 Data Manager Journey Map Focus

Figure 6 Data Citizen Journey Map Focus

Figure 7 Data Professional Journey Map Focus

Figure 8 Data Executive Journey Map Focus

Conceptual Modelling: Exploring Conceptual Modelling

Howard shares that he will explore several forms of conceptual modelling in the webinar. These include entity-Dimensional modelling, focal point modelling, fact-oriented modelling, object-oriented modelling, time-based modelling, NoSQL modelling, and semantic modelling. Initially, the focus will be on three key concepts.

It’s essential to grasp four critical terms: conceptual schema, concept model, conceptual model, and conceptual modelling. The progression occurs from the concept model to conceptual modelling, which builds the conceptual model, ultimately leading back to the conceptual schema. Understanding these areas is crucial for effective conceptual modelling.

Figure 9 Essential Concepts: Conceptual Modelling Journey

Figure 10 Essential Concepts Overview

Figure 11 Conceptual Modelling Terminology: Controlled Vocabulary for Data Modelling

Figure 12 Conceptual Modelling Critical Terms

Understanding the Definition and Implications of Concepts in Database Design

The concept of a conceptual schema varies significantly across definitions. Wikipedia describes it as a high-level description of the informational needs that guide database design, emphasising classification. In contrast, Cambridge defines it as a drawing representing an idea, suggesting a focus on conceptual representation rather than database specifics. This highlights a potential overlap in terminology, where a conceptual schema serves to define the ontology of concepts as perceived and articulated by users. Essentially, it encapsulates what exists conceptually from the user’s perspective, blending elements of both design and representation.

Figure 13 Dictionary: Conceptual Schema

The Importance of Definition in Data Modelling

Howard emphasises the importance of identifying users, particularly subject matter experts (SMEs), for whom data models are being developed. He references Roland Ross, known for his expertise in taxonomies, which Ross described as a word map aiding in achieving precision within a subject area, or “universe of discourse.” The process begins with defining business terms before constructing conceptual models, as industry experts like Steve Hoberman outlined. Howard then notes that this approach ensures clarity and consistency in model development, reinforcing the notion that well-defined business definitions lead to more precise and effective conceptual representations.

Figure 14 Dictionary: Concept Model

The Importance of Definition and Model Development in Data Modelling

A discussion starts on the importance of defining terms clearly before modelling, highlighting that many organisations may use terms with vague or internal definitions that hinder effective communication. It goes on to suggest that attempting to write out definitions can reveal instances where multiple concepts are conflated, necessitating separate terms for clarity. Howard references the experiences of Terry and Mark Atkins, from a data warehousing background, who observed that rushing past definitions often lead to inappropriate relationships in Data Modelling. He explains this as breaking definitions into single statements for business approval, allowing for disambiguation and clearer communication. This structured approach enhances understanding and aligns with various levels of clarity in language and concept, moving from dictionaries to ontologies for better precision.

Defining Concept Model, Conceptual Model and Concept Map

A concept model is defined by a set of definitions representing various concepts and their connections through verbs or verb phrases, serving as a foundation for logical validation. In contrast, a conceptual model evolves from a conceptualisation process, outputting elements that initiate engineering, such as capabilities, processes, and locations.

Concept maps serve as inputs in this process, aiding in the clarity of undefined elements. While a concept model focuses on an inventory of defined words and is primarily a knowledge representation, it encompasses broader applications beyond Data Modelling, including architectural frameworks. This emphasises the importance of understanding and structuring real-world references graphically.

Figure 15 Dictionary: Conceptual Model

Figure 16 Conceptual vs. Concept

Importance of Concepts in Organizational Strategy

An attendee mentions their process in developing a conceptual model for an alignment framework app, emphasising the importance of understanding an organisation’s purpose and desired outcomes. This process involves defining key terms and their relationships, which initially proved complex, prompting the team to utilise R Key for modelling. Through this exploration, Howard found that he could simplify the conceptual model, focusing on high-level understanding rather than creating a robust data model.

Howard moves on to underscore the need to clarify activities (verbs) alongside structural concepts. He then stresses the significance of documenting business rules to derive entities and relationships effectively. This approach aligns with principles seen in frameworks like FCOIM and case talk, where enumerating fundamental facts is crucial for establishing relationships and structures within the model.

Conceptual Modelling: Development of System Models

Conceptual modelling involves creating a high-level system representation through classification, including objectives, inputs, and outputs. This can be visually depicted using graphical tools like causal loop and activity diagrams. The key focus is on abstraction, where concepts are simplified for clearer communication rather than merely the act of typing. For instance, the term “customer” serves as an abstract representation rather than a specific individual. Overall, conceptual models are valuable communication tools that emphasise user-centred design and versatility. Further details will be shared through slides.

Figure 17 Dictionary: Conceptual Modelling

Conceptual Modelling: Classification and Database Representation

Conceptual modelling emphasises that a conceptual schema is a visual representation of ideas, while a concept model provides a set of definitions and logical frameworks. Key elements include identifying classifications, creating a purpose statement for clear communication, and enhancing disambiguation. The ontological aspect involves establishing a controlled vocabulary relevant to the problem domain, which aids in common understanding without delving into excessive details.

Additionally, conceptual modelling is the process of crafting a high-level representation of the main components, distinguishing them from the more abstract notions that can lead to misunderstandings. Overall, this iterative approach focuses on clarity and effective communication within the modelling framework.

Figure 18 Conceptual Modelling Critical Terms

Figure 19 Conceptual Modelling Critical Terms Expanded

Conceptual Modelling: The Zachman Framework

The discussion then moves onto conceptual modelling and utilising the Zachman Framework as a foundational element akin to a periodic table for data management. The framework outlines different levels of data representation, starting from the executive level, which identifies entities and subject areas, down to the engineering and development levels, where data definitions and instances are established.

Howard shares that it is crucial to understand that the modelling process is not only vertical but also horizontal; as the conceptual data model evolves, it must incorporate functional models that aggregate data alongside logistics, roles, responsibilities, scheduling, and business planning. This dual approach enhances the overall development and clarity of data management strategies.

Figure 20 Conceptual Modelling Concept Map: Using Zachman

Figure 21 Conceptual Modelling 5W1H

Data Modelling: Concepts and the Use of an Ontology

Howard outlines a structured approach to inventory identification through the development of an ontology, which includes an executive perspective via an operating model, a conceptual data model, and a business management perspective. He emphasises the importance of analysing existing data—often difficult when examining extensive Excel spreadsheets and databases—by focusing on a conceptual model that identifies entities, processes, and tools for representation and validation. To facilitate the understanding of Data Modelling for trainees, explaining the fundamental concepts of entities, attributes, relationships, and their constraints is crucial. Grasping these building blocks is essential for effective conceptual modelling and system development.

Figure 22 Data Model Vertical & Horizontal Relationships

Figure 23 Conceptual Model for Conceptual Modelling

Figure 24 Ontology for Data Modelling

Data Modelling: Concepts, Subject Areas, and Relationships

An overview of Data Modelling would include several levels, starting with the conceptual and subject area models, which could be further broken down into a logical model, often represented as an enterprise data model. For applications, both logical and physical models are utilised, including a conceptual data model (CDM), logical data model (LDM), and application physical data model (APDM).

Each subject area, such as product design or sales, contains high-level abstract concepts and relationships—expressed as verbs—indicating how different elements relate, like a product belonging to a product group. It’s critical to ensure that concepts remain mutually exclusive and collectively exhaustive (MECE), meaning that a concept cannot span multiple subject areas. Challenges surrounding business glossaries often stem from improper subject area modelling and ownership, highlighting the importance of establishing clear definitions and governance before defining common terminology.

Figure 25 Data Modelling Overview

Figure 26 Modelling Levels: Subject Area Model Example

Data Modelling: Schemes and Database Choices

Howard touches on various Data Modelling schemes, including Relational, Dimensional, object-oriented, fact-based, time-based, and NoSQL, and their implications for data storage. He then highlights the importance of aligning the chosen Data Modelling approach with the target database type, such as a Relational database.

Figure 27 Conceptual Scheme – Database X-Ref

Figure 28 Scheme to Database Cross-Reference

Choosing the Right Database Schema

A discussion centres on the importance of choosing the appropriate database model based on the specific use case. Howard emphasises that when selecting a NoSQL database, one should not attempt to fit it into a Relational or Multidimensional framework. Instead, one should outline a hierarchy of data models, starting from a conceptual model applicable to all storage types, followed by a logical model pertinent to various structures (like object, document, or column databases), and culminating in a physical data model, which varies by database type. Additionally, understanding the right schema for a given business problem is crucial for effective Data Modelling and quality assessment.

Figure 29 Scheme to Database Cross-Reference Two

Database Schema and Aggregation Strategies

When building a database to model business rules, a Relational schema is often preferred due to its clear structure and integrity, particularly when dealing with sequential processes. A Dimensional model is more suitable for aggregating transactions, allowing for efficient data analysis.

The choice of schema directly influences the database type used for storage; for instance, a Dimensional model can target either a Relational or Multidimensional database, while document or NoSQL databases may require data flattening to fit into Relational or Dimensional structures for effective analytics. Overall, the specific business problem dictates the appropriate schema and storage solution.

Relational and Dimensional Data Models

It is essential to distinguish between Relational and Dimensional models in Data Modelling. A logical Dimensional model is preferred for business analytics needs, while a Relational model, often used in traditional data warehouses, serves a different purpose. Mixing these two types within the same logical model generally does not work effectively.

The Relational model can function as a source for the Dimensional model, which is structured around hierarchies and navigation paths towards fact tables. Each model serves its specific role: a Dimensional model supports self-service BI and data marts, while a Relational model typically uses a third normal form suited for data warehousing.

Data Managers: Challenges of Implementing Conceptual Modelling

A data manager faces several core challenges in implementing conceptual modelling within an organisation, including stakeholder alignment, the right implementation framework, scalability, and data integration. To overcome these challenges, it is essential to establish a shared understanding and commitment to the purpose of data management. Utilising the ADKAR change management framework, which emphasises Awareness, Desire, Knowledge, Ability, and Reinforcement, is key. Before any training or policy enforcement, stakeholders must be aware of the benefits of conceptual modelling and recognise how it can enhance decision-making. Ultimately, fostering desire amongst team members is crucial for successful adoption and collaboration in data management initiatives.

Figure 30 Conceptual Modelling for The Data Manager

Figure 31 Conceptual Modelling Data Manager Overview

Figure 32 Conceptual Modelling Real-world Challenges

Figure 33 Stakeholder Alignment

Data Modelling: The Importance of Disambiguation

The importance of disambiguation in organisations is highlighted by the need for a common understanding of key terms, such as “customer.” Howard refers to a case study where a CEO realised the necessity of Data Modelling after discovering multiple definitions of “customer” from his team. This awareness is the first step toward achieving a shared understanding within the organisation, which is essential for effective data conceptual models and process frameworks. Emphasising vertical lineage in metadata before horizontal lineage is crucial for addressing data quality issues; understanding the processes that generate data (state reconstruction) helps clarify the subsequent data flow. Establishing a consistent vocabulary and commitment to purpose ultimately fosters better organisational communication and data practices.

Figure 34 Conceptual Modelling Critical Terms

Figure 35 Commitment to Purpose

Conceptual Modelling and Simulation (Digital Twin)

The conceptual modelling process involves several key stages. It starts with problem description, then defines project goals and develops a conceptual model through refinement and verification using tools like concept maps. Simulations or digital twins can further benefit verification by testing the credibility of the model against established goals.

A digital twin integrates sensors into the system to offer real-time feedback and predictive capabilities, enhancing optimisation efforts in environments like smart cities. Principles to guide conceptual modelling include abstraction, consistency, clarity, relevance, and validation, ensuring that models can be extended without losing critical elements while effectively meeting requirements.

Figure 36 Stages of Conceptual Modelling and Simulation (Digital Twin)

Figure 37 Stages of Conceptual Modelling and Simulation (Digital Twin) Two

Figure 38 “Principles”

Data Modelling: Implementation Framework

The implementation framework involves a series of procedures designed to refine, validate, and implement a model while emphasising the importance of maintenance and evaluation. Key considerations include scalability to meet business needs, the ability to handle diverse data types (volume, variety, and velocity), and the integration of multiple data sources through canonical models and ontologies, which have proven successful in projects like Gene Ontology.

Effective team modelling aligns data structures with business objectives and clarifies core concepts, relationships, and business rules. Additionally, establishing a clear operating model with designated data owners for specific subject areas helps prevent overlap and ensures accountability within Data Governance.

Figure 39 Implementation Framework

Figure 40 “Scalability”

Figure 41 Data Integration

Figure 42 Data Integration: Canonical Model & Ontology

Figure 43 “Key Takeaways”

Figure 44 Data Governance & other Knowledge Areas

Figure 45 Generic Data Governance Model

Figure 46 Generic Data Governance Model Expanded

The Path to Implementing an Information Management Program

The process for establishing an information management program begins with deriving value drivers from business architecture, which informs the business strategy. This strategy leads to the development of a capability map, identifying data subject areas. With these areas outlined, a data strategy SWOT analysis is conducted to pinpoint data-related challenges within the business. A heat map is then created to visualise these issues, followed by constructing a capability map focused on addressing the identified challenges. Subsequently, an operating model is established, and ownership is assigned, marking the commencement of Data Governance. This structured approach ensures foundational elements are in place prior to launching the Data Governance program.

Data Governance in Business Architecture

Howard moves into the final stage of the webinar by emphasising the importance of conceptual subject areas within business architecture, particularly the need for a clear information model. This model is essential for aligning strategy with a heat map to prioritise focus areas. A capability map can derive data subject areas by identifying relevant nouns and verbs.

Challenges in Data Governance are also highlighted, including the need to understand what data is necessary, its meanings, locations, and whether it meets quality standards. Effective communication and a robust implementation framework are crucial for achieving shared understanding and behavioural alignment.

Figure 47 Information Management Program Initiation

Figure 48 Critical Information about Data

Figure 49 Communication & Change

Data Modelling and Course Development

The webinar concludes with a discussion of an ongoing course focused on Relational and Dimensional Data Modelling. Lastly, Howard shares his plans to expand into other areas, including fact-oriented, time-based, and NoSQL modelling and ontology development to enhance data representation.

Executive Summary

This webinar outlines key topics and challenges in developing a robust Data Modelling course. It focuses on the critical aspects of data integration, governance, and organisational strategy. Howard Diesel emphasises the significance of defining concepts in database design, covering essential frameworks such as the Zachman Framework and the development of system models.

Lastly, the webinar addresses the necessity of choosing the right database schema and outlines the distinction between Relational and Dimensional data models. Howard then highlights the role of data managers in navigating the complexities of conceptual modelling and the importance of clarity for effective implementation.