Data & AI Governance Unification for Data Citizens

Executive Summary

This webinar outlines key considerations for the application of Artificial Intelligence (AI) in the insurance industry, with a focus on claim processing and customer service. Howard Diesel emphasises the importance of the European AI Act and a national AI index in guiding governance and compliance. He then highlights the role of technology in enhancing dialogue with clients while ensuring human oversight, particularly in areas such as entity resolution and anomaly detection. It also discusses the significance of classification models in risk assessment, the ethical implications of biases in AI training, and the need for robust data collection and management practices. By addressing the intersection of AI Governance, data flow monitoring, and corporate data records management, this webinar underscores the critical balance between automation and human intervention in optimising insurance operations.

Webinar Details

Title: Data & AI Governance Unification for Data Citizens
Date: 2025-08-21
Presenter: Howard Diesel
Meetup Group: African Data Management Community
Write-up Author: Howard Diesel

European AI Act and the National AI Index

Howard Diesel opened the webinar and underscored critical developments in AI Governance, specifically regarding the European AI Act and its relationship with the GDPR framework. He emphasised the importance of the National AI Index, recently published by the NDMO and SDAIA, which demonstrates the maturity of AI initiatives within the government and builds on the earlier National Data Index focused on data management standards. Overall, the progress in AI Governance and responsibility is promising, and additional resources are available for those interested in exploring these topics further.

Figure 1 Unification of Data, Records & AI Governance

Figure 2 Governance Unification Overview

Practical Application of AI in Insurance Claim Processing

In response to an attendee’s request for a more practical approach to the discussions on frameworks, Howard prioritised case studies for this webinar. He also shared that he is taking the lead by preparing a review on AI literacy, which aims to ensure that all participants feel included and fully understand the data literacy components. By illustrating key AI concepts through a specific case study on insurance claim processing, Howard hopes to provide valuable insights that enhance our understanding of the topic. Ultimately, this approach will foster a more profound comprehension of AI applications in real-world scenarios.

Figure 3 AI Literacy

AI in Insurance Claim Processing and Prompt Engineering

Insurance claim processing is a well-defined business process that features various data exchange standards and message flows between brokers and insurers, particularly when a claim is initiated following a loss. This involves the transfer of information about the policy and the claim itself between multiple entities.

 A key component of this process is FNOL, or First Notice of Loss, which serves as the initial communication regarding a claim. There is growing interest in developing an AI literacy program related to these processes, prompting discussions about integrating machine learning and statistical models within the insurance industry.

When an individual encounters a loss, theft, or injury, they can file a notice of loss with their insurance broker as part of their insurance policy. Leveraging advanced AI tools such as Copilot and various chatbots, users can enhance their understanding of the claims process. To streamline reporting, it would be advantageous to implement a chatbot or virtual agent that guides users through the necessary steps, prompting them for specific information, such as police reports and required documentation, thus simplifying the claims process and improving efficiency.

Prompt engineering has been a hot topic recently, with some individuals voicing the opinion that it’s becoming obsolete due to advancements in AI models, such as ChatGPT, which can now assist users in formulating prompts. A practical approach to prompt engineering involves breaking down prompts into key components: task, instruction, context, reason, and clarification.

This structure enables more precise interactions with AI, with refinement occurring through ongoing dialogue with the model. By clearly specifying what you want the AI to create or answer, you can enhance the overall effectiveness of your inquiry.

To effectively engage with an AI model and minimise the risk of receiving inaccurate information (hallucination), it’s essential to provide comprehensive context and specific instructions for the task at hand. Users should clearly articulate their viewpoint, the reasons behind their inquiries, and any relevant references to guide the model. Additionally, fostering an open dialogue by asking the model what information may be lacking can enhance the quality of responses. By prompting the AI to identify missing elements, users can refine their questions and obtain more accurate answers, ensuring a productive interaction.

Figure 4 AI Literacy Overview

Figure 5 AI Literacy

Automation and Governance in Insurance Claims

The objective of automating the insurance claims process is to enhance efficiency and improve the overall customer experience from the initial notice of loss to the feedback stage. This streamlined approach will utilise various API agents and models to facilitate the collection of necessary documents, conduct assessments, and process payments seamlessly. By optimising each stage of the claims journey, we aim to provide a more effective and satisfying experience for our customers, ultimately culminating in a proactive request for their feedback after the payment has been made.

The business needs to evaluate current processes and improve efficiency, particularly in claims management, where one organisation reported a 50% reduction in processing time for low-risk claims through direct online payments. Understanding industry frameworks is crucial, as highlighted in recent discussions, particularly regarding the purpose of initiatives, such as GDPR principles. Emphasising the importance of defining the purpose behind automation is key, as it allows for a balance between enhancing operational efficiencies and delivering customer value.

The integration of AI chatbots and automated voice recognition systems presents significant challenges that often compel customers to seek human assistance. While the goal is to streamline processes and reduce customer burden, this is particularly crucial in high-stakes situations where human input is necessary. Implementing a “human in the loop” approach for critical decisions, such as evaluating claims or processing payouts, not only ensures that customers are informed about any discrepancies but also helps address concerns about fraud. This balanced solution aims to enhance efficiency while maintaining the essential human oversight required to foster customer trust and improve the overall experience.

The integration of human oversight in governance is essential for ensuring clarity and accountability, especially when it comes to interpreting events or data. Without a human in the loop, agents risk making assumptions due to insufficient information, which can lead to inaccuracies or “hallucinations.” This discussion highlights the critical role of providing detailed context in prompts, as it helps elicit accurate responses and minimises the likelihood of incorrect outputs. Ultimately, fostering human involvement in these processes enhances the reliability of governance and decision-making.

Figure 6 Advanced Prompt Formula for Prompt Engineering

Figure 7 Advanced Prompt Formula for Prompt Engineering pt.2

Role of Technology in Conversation and AI Governance

In comparing human communication and technological interactions, Howard highlighted a key distinction in how information is structured and presented. While humans typically follow a logical sequence of reason, context, and tasks, technology can function effectively by reversing that order, demonstrating a more flexible approach to information delivery. Howard then recounts his personal preference for drafting thoughts in Word before submitting them all at once, a strategy designed to circumvent the frustration associated with chat boxes that may submit incomplete messages. Ultimately, this emphasises the principle of “ready, fire, aim,” underscoring the necessity of establishing a clear context before conveying information.

Adequate conversation preparation is crucial, as it enables individuals to anticipate responses and validate the information they receive. A key aspect of this process is refining discussions, particularly in relation to topics like AI Governance, which may not be initially addressed. By continually revisiting and questioning the provided information, one can engage in an iterative dialogue with a copilot, ensuring a comprehensive understanding. For example, a recent interaction highlighted the utility of chatbots and mobile intake for handling notices of loss, showcasing innovative approaches to data and records management.

In the event of a motorbike or car accident, the initial step involves taking and submitting photographs, which can be facilitated by a mobile intake system that works in conjunction with a chatbot to gather additional details. Following the first notice of loss, the claim processing proceeds through several stages: triage, document review, adjudication via a rules engine, and ultimately settlement, where feedback is solicited from the customer.

Human involvement is crucial, particularly during document review, to ensure the quality and proper tagging of submitted documents for effective records management. Additionally, robust Data Governance is necessary to ensure that pertinent data is extracted from the FNOL to enhance decision-making throughout the claims process.

Figure 8 Advanced Prompt Formula for Prompt Engineering pt.3

Figure 9 AI Governance in the Insurance Claim Process

Entity Resolution and Human Involvement in Chat Bots

Entity resolution plays a crucial role in enhancing a chatbot’s functionality, especially during customer interactions. As users engage with the chatbot, it is essential for the system to accurately identify and reconcile various entities, such as multiple products, policies, and any conflicting sub-clauses or conditions that may arise.

This process involves three primary domains: the customer domain, the contractor policy domain, and pertinent legal terminology. The resolution process extends to include eyewitness accounts in the event of an incident, underscoring the chatbot’s ability to manage complex information effectively. In conclusion, effective entity resolution not only streamlines customer interactions but also ensures a comprehensive understanding of intricate details within the conversation.

The integration of human involvement in document review and decision-making processes is essential for ensuring transparency and fairness. By utilising tools like sentiment analysis on customer surveys, human evaluators can accurately assess customer dissatisfaction and provide necessary insights. This human oversight is also vital in identifying potential fraud cases, thereby enhancing the reliability of automated systems. Ultimately, this approach establishes a robust framework that strikes a balance between automation and human critical judgment.

In the UK, addressing the first notice of loss typically involves human interaction due to the emotional intelligence required to support individuals during distressing situations. While automated systems may initiate the process, they often provide the option to speak with a human representative for added support. Despite initial frustrations, many users become accustomed to handling such scenarios after experiencing them a few times, as illustrated by an example involving an accident where prompt documentation and communication were effectively managed.

Application of AI Techniques and Governance in Customer Service

The refinement loop for processing claims begins with the first notice of loss stage, where natural language processing (NLP) techniques are applied to convert speech to text via a chatbot. This approach aims to extract structured data from customer interactions, whether through voice calls or chat, reflecting a shift in communication preferences, such as the increasing use of voice messages over text in platforms like WhatsApp.

The integration of NLP and chatbots not only enhances data extraction efficiency but also facilitates a more seamless user experience during the claims process. Further developments include creating a matrix that links specific AI techniques to governance frameworks, ensuring the responsible implementation of these techniques.

The governance framework encompasses corporate governance, Data Governance, records management, and AI Governance, ensuring ethical customer interactions and effective escalation processes. Corporate governance facilitates humane engagement, particularly when dealing with frustrated or emotionally invested customers, necessitating human intervention when required. Data Governance validates input data and applies appropriate metadata to submitted information, such as photographs and policies.

Records management involves logging all conversations, distinguishing between documents and records, ensuring transparency and monitoring for bias in interactions. Additionally, speech-to-text technology must enable the accurate transcription of verbal communications, allowing for auditability while safeguarding voice logs with necessary privacy measures to validate model fairness and accuracy.

When submitting a first notice of loss, it is essential to understand that all information provided will be utilised in the claims process. This is crucial because there are instances where individuals later dispute the accuracy of their initial statements regarding injuries or circumstances presented. Therefore, it is essential to be truthful and consistent from the outset to avoid complications later on.

In a discussion about insurance claims, it was emphasised that clear communication is crucial, especially when interacting with agents, including AI representatives. There is a need to inform individuals when they are interacting with an AI agent and ensure that all conversations are recorded. This transparency is vital, as failure to communicate important context can hinder the claims process and lead to disputes over evidence presented by the insurance company. Overall, maintaining accurate records of interactions helps protect the interests of all parties involved.

Effective implementation of various techniques and stages requires a strong focus on several key factors: corporate governance, data management, records management, and AI Governance. It is crucial to connect each technique to its relevant context, ensuring that all governance aspects are aligned and integrated throughout the process. Attention to these elements will enhance the overall effectiveness and reliability of the strategies employed.

Figure 10 AI Techniques

Data Collection and Processing for CFCL Engagement

The development of a structured data set for the First Notice of Loss (FNOL) stage marks a significant advancement in the claims process. This initiative focuses on extracting essential information from custom input methods, such as voice calls and chatbots, by implementing a robust data dictionary that encompasses key elements like customer identity, policy details, incident information, claim type, evidence consent, and legal compliance. Emphasising the need for user consent and adherence to records management requirements, the process ensures that every interaction is transformed into structured data, thus enhancing the efficiency of claim processing. In conclusion, the ability to systematically compile and organise this information not only streamlines operations but also elevates the overall effectiveness of claims handling.

Data Analysis and AI Techniques in Claims Prioritisation

In the claims processing workflow, various techniques are employed to enhance efficiency and accuracy. Key methods include classification models, anomaly detection, and risk scoring algorithms, which are used to assess claim severity, detect fraud, and prioritise claims based on urgency and potential damage.

The claim triage process provides insights, including claim category estimation, estimated loss, and initial severity assessment, based on sentiment analysis. It utilises advanced technologies such as OCR, computer vision, and document AI for extracting data from scanned documents, photos, and videos. Additionally, adjudicated rules, facilitated by decision trees and explainable AI, enhance transparency in the decision-making process. Meanwhile, automation tools such as RPA and smart contracts streamline payment processing, complemented by customer feedback through sentiment analysis.

Figure 11 FNOL Data Dictionary

The Role and Functioning of Classification Models in Risk Assessment and Claims Processing

The objective of the classification model is to automatically categorise insurance claims into predefined risk categories, such as high risk, low risk, and suspected fraud, using historical data patterns. This is achieved through supervised learning methods, such as logistic regression and random forest grading, which utilise structured input features including claim amount, incident type, location, policy details, customer profile, and history.

By training the model on labelled historical claims data, it learns to predict risk classifications for new claims by analysing similar features. While the model relies on extensive data for accuracy, there is an ongoing tension between GDPR guidelines that advocate for minimising data usage and the requirements of AI that often necessitate maximising data for effective learning.

Figure 12 AI Techniques for Claim Triage

Figure 13 Classification Model

Figure 14 Classification Model pt.2

Understanding Classification Models and Big Data Streaming in Insurance

A classification model is essential for categorising data and training algorithms to produce accurate outputs. In the insurance industry, companies increasingly utilise big data streaming to gather public information related to claims, such as social media content and photos. This practice helps them assess claims more effectively and potentially reduce payouts. Additionally, insurers are seeking external evidence from bystanders and witnesses to corroborate claims, enhancing their ability to determine the validity and timing of incidents.

The classification of information collected from photographs is essential for addressing privacy concerns, especially regarding Personally Identifiable Information (PII). This process occurs soon after data collection, significantly influencing how organisations manage their workflows. By routing high-risk information through established procedures, organisations can ensure efficient traffic management and enhance their overall operational effectiveness. Ultimately, a clear understanding of this classification system not only underscores its importance but also supports more responsible and effective data handling practices.

Accord plays a crucial role in enhancing the efficiency of data exchange in the insurance claims process. By providing standardised solutions, Accord facilitates seamless communication among claimants, brokers, and insurers, which is vital for timely responses. This connectivity is especially advantageous in urgent situations, such as car theft claims, where rapid data sharing can significantly improve response times and lead to better overall outcomes. Ultimately, Accord’s innovative approach streamlines the claims process and enhances collaboration among all parties involved.

Anomaly Detection and Risk Scoring in Data Analysis

Anomaly detection is a crucial process used to identify claims that significantly deviate from standard patterns, potentially indicating fraud, errors, or unusual circumstances. This method employs unsupervised learning models, such as Isolation Forest, which works by isolating anomalies rather than profiling normal data. The model achieves this by analysing data points and determining that anomalies require fewer splits to isolate, highlighting their distinctiveness from the standard patterns. This approach effectively facilitates the detection of fraudulent activity by allowing these outliers to stand out, much like finding a rare tree in a forest.

An isolation forest is a method for identifying anomalies by detecting outliers, akin to spotting a distinct red tree in a forest. Statistical deviations can be quickly identified, and autoencoders play a vital role in this process by compressing data into a standardised format and attempting to reconstruct it; however, they struggle with reconstructing outliers, making them useful for anomaly detection.

Box plots, such as box and whisker plots, help visualise deviations from normal data by highlighting points that fall significantly outside the expected range. Additionally, one-class Support Vector Machines (SVMs) define the boundaries of normal data and flag any anomalies that lie outside these boundaries, relying on vector-based approaches similar to those used in Generative AI.

The process involves analysing the normal distribution of claims through features collected to identify outliers based on statistical deviation, which aids in detecting new patterns or rare events. This triggers a manual review, incorporating human insights to gain a deeper understanding of specific situations. Following this, a risk scoring algorithm assigns a numerical score that indicates the likelihood of a claim being complex, fraudulent, or high cost.

Utilising Excel spreadsheets for each assessment area, the scoring is compiled into dashboards for administrative review, enabling the identification of conversion patterns and facilitating the dynamic prioritisation of customer scoring models with weighted features. This comprehensive approach employs three algorithms for claim triage, allowing effective monitoring and assessment of claims.

Figure 15 Anomaly Detection

Figure 16 Risk Scoring Algorithms

Risk, Bias, and Ethics in AI Training

In evaluating risks, biases, and ethics related to data training, several potential issues emerge, including data bias, algorithm bias, and automation-related decision-making risks that could lead to a loss of empathy. For instance, pushing individuals through processes too quickly, such as completing a notice of loss, can result in poor engagement, as they may feel rushed and unable to articulate their challenges. Additionally, concerns like privacy violations and feedback loop management arise, necessitating a careful review of customer feedback. Misinterpretation of sentiment due to emotional tone or cultural context can lead to inappropriate escalations and false positives, underscoring the importance of combining model outputs with human review to ensure the accuracy of sentiment analysis.

Effective Data Governance is crucial for ensuring fair representation of historical claims data, particularly in urban areas like Cape Town, where biases can lead to unfair claim denials or delays. Howard highlights the necessity of a human-in-the-loop approach to address these biases, emphasising the need for diverse training data that accurately reflects critical data elements. By utilising robust reference data sets, organisations can enhance corporate governance, reduce decision-making errors, alleviate customer distress, and mitigate regulatory risks, ultimately fostering a commitment to identifying and addressing biases within governance practices.

Figure 17 Bias / Ethics

Figure 18 Harm Assessment

AI Harm Assessment and the Role of Human Intervention

The evaluation of harm related to AI systems is crucial for identifying potential risks such as injury, denial of services, loss of privacy, and social manipulation. Understanding how data collection practices can disrupt individuals’ private lives is essential, especially in scenarios where personal information is shared without consent. For instance, photographing a car with individuals inside who have not given their consent raises significant ethical concerns regarding privacy infringement. Ultimately, it is vital to consider the implications of data handling practices, as contributions from various sources can lead to inferred information that jeopardises personal privacy and necessitates stringent ethical considerations.

Howard shared that a previous presenter had developed an innovative AI model that automatically blurs faces in photographs and videos when consent is not obtained, enhancing privacy protections. This advancement emphasises the need for harm assessment within the broader context of Data Governance, particularly as it intersects with AI, corporate responsibilities, and record management. The approach suggests integrating human oversight, especially for complex or emotional cases, ensuring that sensitive situations are handled by humans rather than solely by AI algorithms. This entails systems such as claim triage and AI-based severity scoring to balance automation and human judgment effectively.

In situations involving ambiguous or high-risk claims, such as those with unclear evidence or information disputes, it is essential to involve humans in the decision-making process. This includes claim adjudication, interpreting legal terms, processing payments, and managing large payouts. Additionally, when customer feedback is negative, or escalations occur—such as requests to speak with a manager—human intervention is necessary to resolve the issue. Understanding when to integrate human oversight is critical, as it should occur at various points throughout the process, not solely at the decision-making stage.

Figure 19 Humans In the Loop

Regulations of AI Systems

The AI Act outlines a risk-based framework for regulating AI systems. Low-risk applications, such as video games and spam filters, are exempt from these obligations. For systems with a higher risk of manipulation or deceit, such as chatbots and emotion recognition technologies, transparency is required to inform users that they are interacting with AI.

The Regulations also highlight the potential harm from faulty or misused AI in sensitive areas, such as employment, law enforcement, product safety, and critical infrastructure. Additionally, the act prohibits subliminal manipulation and social scoring, necessitating a careful assessment of risk to determine appropriate regulatory measures.

Figure 20 AI Act: AI Risk Levels

Corporate Data Records Management and AI Governance

The checklist for corporate data records management emphasises the importance of logging and retaining all AI-generated decisions, including detailed records of orders, versioned training datasets, and retention schedules applied to voice AI. Howard highlighted the necessity of a tested governance model that ensures bias, fairness, and explainability for each AI technique, tailored to the associated risk level. Key elements include governance checks for computer vision, visual evidence adjudication, linking image metadata to claims, retention of image analysis outputs, and ensuring fairness in damage assessment models.

Figure 21 Governance Checklist

Figure 22 Governing AI Techniques

Monitoring and Addressing Data Flow and Bias in AI Models

To effectively monitor AI classification models, it is essential to track various types of drift through comprehensive dashboards. Key metrics include model performance indicators such as accuracy, error rate, precision, and recall, alongside data distribution to identify data drift. By comparing histograms of training data and live data, discrepancies can be detected, indicating shifts that require attention. Additionally, monitoring for bias and fairness is crucial, which includes demographic bias alerts and analysis of human override logs to assess decision-making processes. Ultimately, compliance metrics are essential for effective governance and ensuring adherence to regulatory standards.

Figure 23 Model Drift Monitoring

AI Governance and Data Management

The evolving landscape of AI Governance necessitates a closer integration with corporate governance, particularly in critical sectors such as healthcare and finance. Participants in the discussion emphasised the importance of rigorous scrutiny for AI tools, arguing that they should be evaluated more stringently than human actions to ensure ethical usage. This highlights the crucial role of corporate governance in establishing transparency and oversight in AI applications, ensuring that acceptable practices are clearly defined and followed by all stakeholders involved. Overall, while frameworks such as King III and King IV provide valuable guidance, a notable gap remains in understanding the implications of King V within this context.

King V, despite being a draft document, effectively addresses critical trends in data management and thought leadership, particularly in relation to AI and process automation. The discussion highlighted the essential role of Data Governance, emphasising that even sophisticated AI workflows rely heavily on the quality of Master Data records. Furthermore, it became clear that a robust foundational framework is necessary to successfully implement and utilise AI solutions, demonstrating that the underlying principles are vital for maximising the benefits of technological advancements. In conclusion, King V serves as a valuable resource for navigating the complexities of data management in an increasingly automated world.

The Role of Data Management and Human Intervention in AI Development

Data management plays a crucial role in the successful implementation of AI systems. The conversation underscores the importance of governance and established procedures in navigating the challenges posed by AI technology, as evidenced by a recent webinar that shed light on trainers’ struggles in this area. Without proper data management practices, AI systems can generate unreliable results, underscoring the critical need for effective Data Governance to harness AI’s full potential and ensure its effectiveness. Thus, prioritising data management is essential for the successful deployment and performance of AI technologies.

The successful development of models, particularly for risk classification, relies heavily on understanding and identifying critical data elements. A goal-oriented approach is essential, as selecting the right features is crucial for achieving reliable outcomes. Without a strong data management practice, the integrity and quality of the data can suffer, leading to poor results, as evidenced by the concerning statistic of only 45% success in pilot projects. Therefore, data management professionals need to familiarise themselves with relevant terminology and demonstrate how their support contributes to practical risk assessment and ongoing data quality monitoring.

The classification of data is crucial for ensuring efficient workflows; without it, organisations risk creating complex, unmanageable processes resembling “spaghetti workflows.” Effective implementation of AI should enhance operational efficiency, but failure to classify data correctly can undermine this goal. It is essential to maintain a “fit for purpose” approach, ensuring that historical claims data is accurately interpreted to avoid biases in both data and algorithms. Furthermore, the concept of “human in the loop” emphasises the crucial role humans play in leveraging emotional intelligence and intuition in conjunction with AI’s capabilities, marking a shift in responsibilities within the process.

The evolution of technology has significantly transformed the job landscape, reduced the prevalence of manual and traditional clerical work, and simultaneously created new opportunities in emotionally and intuitively driven roles. This shift mirrors the industrial revolution, where the decline of manual jobs led to the rise of essential clerical positions. A critical aspect of this discussion is the “reasonableness dimension” in data quality, which highlights the need for human judgment in evaluating anomalies detected by machines.

Despite advancements in Artificial Intelligence, its inability to make nuanced assessments of normalcy highlights the ongoing importance of human intervention in data analysis. Ultimately, as the job market continues to evolve, the integration of human insight will remain indispensable in navigating the complexities of data.

The effectiveness of anomaly detection technologies is increasingly recognised for their ability to identify and isolate outliers within large data sets, much like finding a single tree in a sprawling forest. Participants in the discussion expressed optimism about advancements in data analysis, emphasising the complex relationships among various Data Governance elements and the crucial role of human reasoning in interpreting the insights derived from these technologies. Adam highlighted that as data continues to evolve, so too does the role of humans in this analytical process, fostering a hopeful perspective for future developments in the field. In conclusion, the integration of technology with human intellect holds promise for the future of data analysis and anomaly detection.

If you would like to join the discussion, please visit our community platform, the Data Professional Expedition.

Additionally, if you would like to watch the edited video on our YouTube please click here.

If you would like to be a guest speaker on a future webinar, kindly contact Debbie (social@modelwaresystems.com)

Don’t forget to join our exciting LinkedIn and Meetup data communities not to miss out!