AI Governance for Data Executives

Executive Summary

This webinar looks at where artificial intelligence and data governance meet, giving data executives practical insight into machine learning implementation, bias detection, model deployment, and governance frameworks. It covers how to manage AI systems responsibly without losing sight of business value or regulatory compliance.

Webinar Details

Title: AI Governance for Data Executives
Date: 21 October 2025
Presenter: Howard Diesel
Meetup Group: African Data Management Community
Write-up Author: Howard Diesel

Machine Learning and Climate Science in a Virtual Meeting

Howard Diesel opens the webinar by covering some schedule changes and explaining why this topic matters. He draws on a previous climate science webinar to show how machine learning gets applied to weather patterns and environmental challenges more broadly, pointing to both the bad news, retreating glaciers in the Swiss Alps, disappearing permafrost, and the good news, like advances in solar panel technology.

Framing it around these contrasting outcomes makes the case for how much machine learning matters across different domains, and it sets up the more technical discussion to come about its various applications in addressing climate issues. The goal for the webinar is to leave participants with a solid understanding of how machine learning can genuinely help our response to environmental challenges.

Figure 1 Machine Learning for ABOK

Figure 2 What is Machine Learning?

Machine Learning: Challenges, Approaches, and Applications

Machine learning increasingly pulls together statistics, computer science, and operations research to tackle real business problems. Operational research in particular is showing up more in areas like queuing systems and resource allocation in healthcare. Three types of analytics sit at the centre of this: descriptive, predictive, and prescriptive, together giving a framework for data-driven decisions.

Understanding how data actually flows also matters for machine learning to work well. Data moves through systems like data lakes, lakehouses, and warehouses before it ever reaches the models powering tools like insurance portals, chatbots, and fraud detection systems. Getting a handle on that flow helps make sense of machine learning’s role in the business, and it’s foundational for AI governance too.

Figure 3 Machine Learning in Insurance and Healthcare

Figure 4 Three Machine Learning Paradigms

Figure 5 Supervised Learning: Learning from Labelled Data

Figure 6 Linear Regression and Logistic Regression

Figure 7 Unsupervised Learning: Discovering Hidden Patterns

Evolution and Impact of Machine Learning Techniques

Howard shares that he will cover the three main categories of machine learning: supervised, unsupervised, and reinforcement learning. Supervised learning uses human-labelled datasets to predict numbers or classify data into categories. Howard’s preferred approach starts with unsupervised learning to find patterns and clusters first, what he calls “listening to the data”, which helps with density estimation and spotting opportunities for dimensionality reduction.

Feature engineering comes up a lot here too, streamlining features by matching similarities and correlations to make training more efficient. There’s an ongoing debate in the machine learning community about feature reduction versus deep learning approaches, which tend to keep all the features rather than trim them down. Both strategies have their place in getting the best out of a model, depending on the situation.

Figure 8 Reinforcement Learning: Learning through Interaction

The Importance of Data Preparation and Governance in Machine Learning

Good data preparation makes or breaks a machine learning project. The end-to-end process starts with identifying the business problem and moves through data collection, feature engineering, model training, evaluation, and deployment. Data preparation eats up around 60% of that time, with only 25% going to modelling and 15% to deployment, which says a lot about how much it matters for the quality of the end result.

Data preparation is a genuine burden, too: data scientists spend 60-70% of their time just cleaning and organising data, which is part of why they’ve been jokingly called “data garbage men.” That reality is exactly why good data governance and collaboration matter, since poor governance can stall machine learning initiatives altogether. In short, getting data preparation and governance right is what clears the path to successful machine learning outcomes.

Figure 9 End-to-End Machine Learning Workflow

Figure 10 Data Acquisition and Cleaning: the Foundation of Machine Learning

Feature Engineering and Standardisation in Policy Analysis

Feature engineering, creating, transforming, and selecting the input variables that actually matter, is a key step in building an effective machine learning model. That might mean generating a derived feature like “days since policy inception” from a date field, turning raw data into a continuous numerical value that’s easier to work with.

Additionally, feature transformation also deals with scale differences between variables, using normalisation and standardisation based on the mean and standard deviation. That keeps high-value features like claim amounts from drowning out lower-value ones like policy duration in years. Get this right and you avoid imbalanced models where a handful of variables dominate everything else, which improves performance and cuts the risk of biased outcomes.

The importance of feature engineering is further underscored by the visual examples of histograms presented during discussions. By showcasing these histograms, the presenter demonstrates how unstandardized features can lead to imbalanced models, ultimately highlighting the detrimental effects of allowing certain variables to dominate the analysis.

Left unaddressed, that imbalance leads to weaker model performance, since it fails to capture the relationships across all the relevant features properly. Getting feature engineering right, in other words, is what builds a model that’s actually robust, fair, and reliable.

Figure 11 Feature Engineering: the Art of Machine Learning

Feature Selection and Machine Learning Techniques in Predictive Modelling

Feature selection matters because it identifies the variables that actually move the needle, often narrowing things down to a “top 20.” Good feature engineering often has more impact on a model’s effectiveness than the choice of algorithm itself. That said, it needs care: demographic or location-based variables, postal codes, for example, can introduce bias and lead to unfair discrimination if you’re not careful.

Building an effective predictive model means assembling what you might call an “all-star” team: correlation analysis, variance assessment, principal component analysis. Testing multiple algorithms, logistic regression, decision trees, random forests, neural networks, support vector machines, lets you build ensemble models that combine the strengths of each approach, which tends to produce more accurate and fairer predictions than relying on any single one.

Figure 12 Feature Engineering: the Art of Machine Learning

Figure 13 Model Training and Selection: Finding the Best Performer

Challenges and Considerations in Machine Learning Model Deployment

Deploying a machine learning model is different from deploying a traditional BI report. A BI report can go live right after testing, but a machine learning model needs extra validation before it’s ready for production, covering infrastructure challenges like API compatibility across operating systems and security considerations that don’t really come up in traditional BI work.

The main question in deploying a machine learning model is when it’s actually trained and accurate enough for the real world, which takes balancing model complexity against performance and having the right governance frameworks in place. Deployment readiness should go beyond accuracy metrics too, factoring in bias detection and potential business impact. Get all of that right, and organisations can improve their odds of deploying a model that holds up on both technical and ethical grounds.

Figure 14 Deployment and Monitoring: Production Machine Learning

Machine Learning and Predictive Modelling in Power BI

Adding educational videos to a Power BI dashboard is a good way to make complex machine learning concepts land for governance professionals. One video in particular walks through bias and variance using the example of predicting mouse weight from height, showing how linear regression models, the straight lines, have high bias but low variance, while more flexible models, the squiggly lines, have low bias but can swing toward high variance.

That’s foundational knowledge for governance professionals evaluating model performance and AI deployment readiness. Understanding the bias-variance trade-off is what lets them judge whether a given modelling approach is actually appropriate, and having that grounding is what makes effective oversight and monitoring of AI systems possible in the first place.

Figure 15 Understanding Bias and Variance Tradeoff

Figure 16 Videos

Figure 17 Machine Learning Fundamentals, https://www.youtube.com/watch?v=EuBBz3bI-aA

Bias in Machine Learning Algorithms

Bias and variance are two concepts that really shape model performance and governance. Bias comes from oversimplified model assumptions; variance comes from a model being too sensitive to fluctuations in the training data. How these two interact with model complexity is central to building effective AI systems: simple models tend toward high bias and low variance, while complex models tend toward low bias and high variance.

Getting that balance right is what minimises total error and keeps model performance robust. Overfitting is the real danger here: it happens when a model memorises noise in the training data instead of learning patterns that generalise, and it performs poorly once it sees new data. This matters for anyone making AI governance decisions, since it shapes validation, testing, and deployment strategy. Deploy an overfitted model too early, and it can mean poor real-world performance and real business losses.

Data Training and Validation: Challenges and Strategies

Training-validation splits matter for model performance, and a 70/30 training-to-validation ratio is a common starting point, though it may need adjusting depending on how the model performs. How you select the data for these sets matters just as much: random selection can introduce bias if the data isn’t properly stratified, so it’s worth thinking about how the data is distributed across clusters.

Unsupervised learning clustering techniques can help here, giving useful insight into how the data is distributed so the training and testing sets stay representative when you sample properly from each cluster. It’s a similar idea to data quality sampling methods more broadly: aim for consistency and thoroughness rather than arbitrary selection, and definitely don’t just grab the first or last records. Done carefully, this keeps data integrity intact and gives the model a better shot at generalising well.

Error Reduction and Data Selection in Machine Learning

Error reduction in machine learning tends to follow a pattern worth knowing if you’re doing governance work: error rates start high and drop as the model trains, but keep training too long and they can start climbing again as the model overfits, becoming too tailored to the training data to generalise well. That’s why model performance needs watching, to find the right balance between how long to train and how complex to let the model get.
Understanding the total error formula, bias, variance, and irreducible noise together, matters for good governance here. Once teams recognise the point where a model shifts from learning genuine patterns to memorising noise, they can set clear criteria for when it’s actually ready. That way, deployment decisions rest on real performance metrics rather than just training being finished, which produces more robust, reliable applications.

Machine Learning Clustering and Data Quality Assessment

Clustering techniques help a lot with data selection and quality assessment. Using clustering to look at data density and distribution supports better decisions about training and validation datasets, promoting sampling that’s actually representative across data segments rather than random or sequential selection that risks bias.

It involves identifying cluster centroids and sampling within appropriate distances, similar to statistical sampling methods used elsewhere for large-dataset quality assessments. This kind of systematic approach keeps models trained on genuinely representative data while preserving the diversity a model needs to generalise well, which is really where data governance and machine learning intersect: get the selection process right, and model performance follows.

Machine Learning Models for Anomaly Detection

Machine learning models matter a lot in production, particularly for prediction and decision-making tasks like anomaly detection and fraud identification. A big part of getting this right is the distinction between false positives and false negatives, both of which have real business consequences. False positives mean extra human validation, but they also give you a chance to catch and fix errors early. False negatives are worse in a sense: they let unrecognised threats through and allow incorrect approvals to go unchecked.

Good risk assessment is what determines acceptable error rates and where human intervention should kick in. For insurance claims, this decides which claim amounts can be auto-approved and which need a human to look at them. A 30% error rate, for example, is a good illustration of why businesses need to balance operational efficiency against risk exposure, setting thresholds based on both the financial stakes and their own capacity to handle exceptions.

The Complexity and Trade-offs in Machine Learning for Insurance Claims

A confusion matrix is a useful tool for assessing how a model performs on insurance claim processing. Breaking results into true positives, false positives, true negatives, and false negatives gives a detailed picture of performance across a dataset of, say, 1,000 claims. Metrics like accuracy, precision, recall, and F1 score, all derived from the confusion matrix, show how well the model is actually predicting fraudulent claims.

The ROC curve shows the trade-off between true and false positive rates, another way of getting at model reliability. Below 60% accuracy, a model is essentially no better than random guessing, which sets a fairly clear floor for performance standards. It’s worth weighing the cost of human validation against the risk of missing fraudulent claims too, since that risk varies with claim amounts and how much capacity the organisation has to handle exceptions. Balancing all of this is what optimises both performance and operational efficiency.

Figure 18 Classification Metrics: Understanding Model Performance

Figure 19 ROC Curve and Threshold Tuning

Figure 20 ROC Curve and Threshold Tuning pt.2

Decision Trees and Anomaly Detection

Decision trees are genuinely useful in supervised learning because they’re transparent and interpretable. They generate tree-like models with clear if-then-else rules based on data patterns, which cuts down the need for heavy input from subject matter experts. For insurance claims, that might mean a decision tree building rules from claim amounts, claimant age, and previous claim history.

Because decision trees are interpretable, experts can review and validate them properly before they go live, making sure the logic actually lines up with business objectives and regulatory standards. That transparency matters a lot for organisations focused on governance and compliance, since it builds confidence in automated systems. Being able to understand and validate how a decision was reached, especially in an industry as complex as insurance, is exactly why decision trees appeal to governance-conscious organisations.

Figure 21 Machine Learning Algorithm Landscape

Figure 22 Decision Trees: Transparent Rule-Based Learning

Deep Learning and Neural Networks in Fraud Detection

Neural networks and deep learning systems can process a lot of features at once, but that power comes at the cost of interpretability, they’re often called “black boxes” for a reason. They take inputs like age and claim amounts and produce binary outputs, legitimate versus fraudulent, say, without making the reasoning obvious. There are specialised techniques for unpacking neural network decision-making, though, so with the right methodology, these systems can be understood better than their reputation suggests.

That interpretability challenge is exactly why governance teams need solid frameworks for validating and monitoring these systems, which often means bringing in specialised explainability tools to keep deep learning deployments responsible in critical business processes. Get that framework in place, and organisations can use neural networks without giving up transparency or accountability.

Figure 23 Neual Networks: Deep Learning Power

Figure 24 The Essential Main Ideas of Neural Networks: https://www.youtube.com/watch?v=CqOfi41LfDw

Figure 25 Support Vector Machine: Finding the Optimal Decision Boundary

Figure 26 Time Series Forecasting: Predicting Temporal Patterns

Figure 27 Natural Language Processing: Understanding Text Data

Core Principles of AI Governance

AI governance rests on three pillars: fairness, transparency, and accountability. Fairness means detecting and correcting bias so decisions treat different demographic groups equitably, which comes down to rigorous fairness audits and training data that’s actually representative. Transparency means decision-making that can be explained, model logic that’s documented, and model cards that lay out the development decisions that were made. Audit trails matter here too, documenting every step for compliance and for anyone reviewing the decision later.

Accountability means clear ownership of model outcomes and frameworks for regulatory compliance, backed by continuous monitoring and incident response plans so AI systems keep operating responsibly. Together, fairness, transparency, and accountability protect users from bias and discrimination while still upholding business value and regulatory compliance. By bringing all three together, organisations can deploy AI responsibly while building trust and reliability into their automated systems.

Figure 28 Artificial Intelligence Governance and Ethics

Neural Networks and AI Techniques

Model explainability matters more and more as regulatory pressure for transparency grows. Different AI techniques vary widely in how interpretable they are: decision trees are easy to follow, neural networks much harder. Tools like SHAP (Shapley Additive Explanations) and LIME have been built to close that gap, giving useful ways to unpack the decisions coming out of complex models.

Understanding feature importance rankings matters too, for identifying the variables that actually drive predictions. For insurance claims, that might mean claim amount, policy duration, customer age, previous claims, location risk, vehicle age, and coverage type. As demand for explainable AI keeps growing, these tools help with both regulatory compliance and governance more broadly. Secondary explainability techniques give stakeholders the transparency they need to understand what’s behind an automated decision, which builds trust in the system overall.

Figure 29 Model Explainability: Making AI Decisions Transparent

Figure 30 Machine Learning Practices: Key to Successful Implementation

Feasibility and Application of the Medallion Architecture in Data Warehousing and Processing

The Medallion Architecture works well for data processing, with bronze, silver, and gold layers giving a systematic path for handling information. The bronze layer holds raw data from sources like claims and policyholder information. The silver layer then refines it through processing, calculations, and business rules to meet specific requirements. That structured pathway is what improves data quality and reliability along the way.

At the top sits the gold layer, the presentation layer where AI models, analytical databases, and front-end applications all come together. It supports advanced analytics while sticking to governance protocols around fairness, transparency, accuracy, and security. One live debate around this architecture is whether machine learning models should be fed from the bronze or silver layer, which comes down to a trade-off between data quality and processing efficiency in service of good AI governance.

Data Quality, Architecture, and Human Intervention in Machine Learning

Data quality is what makes or breaks an effective machine learning model, since accurate predictions depend on the input data meeting certain quality standards. The bronze-versus-silver debate really comes down to two different approaches to data quality control: improving the data before it enters the data lake, or cleaning it once it’s already there. Which one makes sense depends on latency requirements, business needs, and what the existing infrastructure can actually support.

With streaming data, real-time processing often wins out, giving immediate insight but requiring more careful handling of transformations. Batch processing, by contrast, allows for a more thorough quality pass, catching data issues more comprehensively. Whichever method you choose, upstream cleaning or in-lake processing, keeping data quality high is what reduces the irreducible noise that otherwise drags down model performance and reliability.

Balancing Risk and Business Impact in Business Process Analysis

Human intervention matters for managing the risks that come with automated decision-making. Figuring out when and how to bring humans into the loop takes a real analysis of business processes, potential consequences, and risk impact. For insurance claims specifically, that means weighing claim amounts, customer impact, processing speed requirements, and organisational capacity before deciding.

The EU AI Act gives organisations a useful guide here, requiring that automation decisions be justified by legitimate business interests rather than convenience alone. Risk assessments should weigh both sides: the cost of false positives, extra workload from human validation, against the risk of false negatives, like fraud cases slipping through. Clear thresholds and escalation procedures, based on financial exposure, operational capacity, and regulatory requirements, are what keep automation deployment responsible.

Role of AI and Human Factors in Decision Making and Risk Management

Edge AI is genuinely hard to oversee, especially in high-stakes settings like medical devices or time-critical applications where decisions have to happen immediately. Multi-model approaches, adversarial networks among them, can help with validation by checking the quality of the primary model’s outputs. A secondary AI system might run toxicity checks on a language model’s responses, for instance, filtering out harmful outputs before they reach users. That extra layer of scrutiny matters a lot in high-risk scenarios, where getting a decision wrong can have serious consequences.

Managing all of this means giving subject matter experts the right tools and explainability features to actually understand and validate AI-driven decisions. Comprehensive information and solid analytical capabilities let SMEs critically evaluate model outputs and catch problems before they escalate. Stronger human oversight in edge AI systems protects the business, but it also improves outcomes for customers by cutting the risk that comes with automated decisions.

Machine Learning Models in Insurance Claims Assessment

False negatives are genuinely hard to catch in machine learning, precisely because they’re subtle and often go without immediate human review. They happen when something gets approved that shouldn’t have been, which is why detection strategies need to be sophisticated. Multi-model approaches using different algorithms for cross-validation help spot them, and reinforcement learning models can monitor primary model decisions in real time and flag discrepancies. Parity checks and fairness audits add another layer, surfacing systematic bias across different demographic groups.

Addressing false negatives properly takes a systematic review process: secondary models, regular auditing, analysing decision-making patterns over time. That’s how organisations catch the underlying issues that signal a model needs retraining or adjustment. Staying proactive about monitoring and improving decision-making is really what keeps machine learning models robust and fair.

Data Bias and Machine Learning Challenges

Z-score-based anomaly detection is a useful complement to machine learning fraud detection. Values that exceed 3 standard deviations from the mean flag as potential anomalies, which then prompts human review to sort out false positives from legitimate outliers. It’s a statistical layer that adds real value on top of the model’s own validation.

Z-scores have their limits, though: false negatives can still slip through if multiple models miss the same anomalies because they’re short on relevant features or training data. Tackling that means a broader strategy, reviewing feature engineering, retraining models on updated datasets, running systematic audits. That’s the work that surfaces gaps in anomaly detection across different model types and validation strategies, and it’s what actually improves fraud detection over time.

Chaos Engineering and Data Quality Assessment

Chaos engineering strengthens system resilience by deliberately introducing controlled failures into data processing pipelines and seeing what happens. Testing hypotheses about system behaviour under these failure scenarios lets organisations check the integrity of data flows, from source systems through data lakes and warehouses to front-end applications. It’s a proactive way to catch vulnerabilities in the data infrastructure before they turn into real data quality or model performance problems.

The connection between chaos engineering and machine learning error management points back to the same governance protocols, fairness, transparency, accountability, that matter everywhere else. As organisations work through the complexity of data processing, it’s worth making sure failure points don’t quietly introduce bias or degrade model quality. Chaos engineering ends up supporting both good data governance and reliable AI operation, by keeping data flow integrity and quality standards intact.

Integration of Data, AI, and IT Governance

Effective AI governance means integrating several domains at once, data governance, records management, AI governance, data privacy, IT governance. Each of them has to work with the others for oversight to actually be holistic, instead of falling into isolated silos. Data governance, for example, is what keeps input data high-quality and fit for purpose, while records management handles documentation, images, and the audit trails that accountability depends on.

AI governance itself is what addresses model decisions and algorithmic fairness directly, while data privacy and IT governance cover security and infrastructure. The old “garbage in, garbage out” principle is as true as ever here: poor-quality inputs compound into increasingly severe errors down the line. That’s why organisations need a unified governance approach that treats these domains as genuinely interconnected, with clear responsibilities and collaborative oversight across development, deployment, and ongoing management.

Bias and Decision Making in Machine Learning

Machine learning can genuinely improve executive decision-making by surfacing bias in human judgment. Running executives and algorithmic models through the same scenarios in parallel is one way to expose systematic bias in how people actually decide. Explainability tools help too, showing which features and criteria drive an automated decision, which makes for a meaningful comparison against how a human made the same call.
That kind of comparison helps executives see their own blind spots and improves decision quality generally. False negatives remain a challenge, though: undetected bias can keep costing the business without anyone noticing right away. Which is why systematic bias-detection, across both human and machine decisions, backed by explainability tools, is worth building in, to catch weaknesses before they hurt business outcomes or fairness.