DEVOPS for Data Management with Ernest Matthysen
Executive Summary
This webinar provides an overview of the challenges and solutions associated with data management at Mediclinic. Ernest Matthysen touches on the implementation of a transition to an agile approach, focusing on agile adoption and Jira implementation, support within an agile framework, and utilising agile methodology in a business context. He emphasises adopting the agile framework and sprint process, managing delivery subtasks, and the importance of sprint planning and stand-up meetings. The webinar explores other key points including agile delivery and business engagement, time boxing unknowns and data management team involvement, sprint reviews and retrospectives, engaging with business and agile approach, benefits of data steward engagement with MDM process, and best practices for retrospectives in agile teams. Lastly, Ernest highlights the importance of developer skills and efficient processes, team support and agile development, minimal viable products, and the role of developers and business analysts in data modelling.
Webinar Details
Title: DAMA SA - DevOps for Data Management
Date: 08 February 2021
Presenter: Ernest Matthysen
Meetup Group: DAMA SA User Group
Write-up Author: Howard Diesel
Contents
Data Management at Mediclinic
Data Management Challenges and Solutions
Transition to Agile Approach
Agile Adoption and Data Management Challenges
Adoption of Agile Framework and Jira Implementation
Support in an Agile Framework
Utilising Agile Methodology in a Business Context
Agile Framework and Responsibilities
Agile Development and Sprint Process
Agile Project Management and Sprint Planning
Agile Sprint Planning and Delivery Process
Management of Delivery Subtasks
Importance of Sprint Planning and Stand-Up Meetings
Agile Frameworks and Team Communication
Agile Delivery and Business Engagement
Time Boxing Unknowns and Data Management Team Involvement
Sprint Reviews and Retrospectives
Importance of Engaging with Business and Agile Approach
Benefits of Data Steward Engagement with MDM Process
Best Practices for Retrospectives in Agile Teams
Analysing Disruptions in Agile Sprint Cycles
Importance of Developer Skills and Efficient Processes
Importance of Team Support and Agile Development
Importance of Minimal Viable Product and Agility in Software Development
Role of Developers and Business Analysts
Data Modelling in Agile Development
Data Management at Mediclinic
Ernest Matthysen, a management specialist and solutions architect at Mediclinic International with over 25 years of experience in the IT industry, will be presenting on agile principles and the agile scrum process and framework, covering their implementation in Mediclinic which has led to increased productivity and more realistic timelines for deliveries. Ernest shares that Mediclinic is a company established in 1983, listed on the London Stock Exchange, with three divisions in London, Switzerland, Dubai, and Abu Dhabi, and many clinics in Southern Africa. Its data management team has been developing solutions since 2013, providing services to both group and individual divisions and delivering solutions for master data management and bespoke data management solutions. Currently, there are 23 solutions supported, with more under development and a team of three developers using EBX 5 technology.
Figure 1 Mediclinic Company Overview
Figure 2 Mediclinic Data Management
Data Management Challenges and Solutions
Ernest notes that they use Jira for agile delivery management and Snowball for incident and service request management. They develop solutions for the entire group while facing data management challenges, with data quality, management reporting, and predictive analytics using machine learning and AI as key strategic objectives. The shift to agile and the scrum framework has been beneficial due to the competitive business landscape and the need to obtain data from IoT devices and non-standard sources. The impact of COVID-19 has affected revenue, leading to a greater need for cost-effective and efficient solutions. Regulatory pressures such as the Protection of Personal Information Act (POPIA), the General Data Protection Regulation (GDPR), and the Middle East Data Security and Privacy Information Act impact healthcare operations.
Transition to Agile Approach
The transition to an agile approach has been critical in improving healthcare operations. The IT delivery team needed to engage and work with stakeholders to meet business expectations. Adopting an agile framework has improved stakeholder relationships and created easy-to-use, intuitive, and mobile-friendly solutions. The move to an agile delivery mechanism has necessitated a transition from a pseudo-waterfall approach, and work has been done to create a platform to support this transition. Additionally, the importance of data privacy in delivering solutions to different divisions has been recognised, and front-line pressures in business have led to pushback on involvement in IT processes.
Agile Adoption and Data Management Challenges
Ernest notes the implementation of the agile methodology for delivering new solutions and operational support of existing solutions. However, data management challenges exist due to different standards used in various companies' international divisions. These standards differ due to amendments and regional differences, but the company plans to solve this by creating a solution to master it in one code. Agile adoption began in 2018, and the company underwent a training session comparing the pros and cons of the waterfall and agile framework. This training triggered a shift in mindset from waterfall to agile methodology.
Figure 3 Agile Adoption
Adoption of Agile Framework and Jira Implementation
The department adopted the Agile framework and provided training to all teams, which initially proved expensive. However, an in-house training solution was developed and implemented. Not all teams were on board with Agile, so the data management team implemented the Jira Agile Scrum framework. Jira was selected as the platform for Agile delivery, and adjustments were made to its implementation and training documentation after lessons were learned from the first month-long sprint. The documentation was amended to incorporate Jira functionality, helping new teams adopt the Agile approach. Additionally, development and operational support processes were also considered.
Support in an Agile Framework
To improve support processes in an Agile framework, a company initially faced tension when implementing Jira alongside Cherwell. However, they realised that integrating the two would be beneficial, with Cherwell serving as a pipeline for incidents, service requests, and changes. The company allocated 80% of its resources' time to new development or changes and 20% to support. After six months, they found that the 80-20 time allocation was accurate based on statistics pulled from Jira. The company emphasised the importance of drive and adoption in the Agile process, with the passion and commitment of the scrum master being key. Overall, team members who embraced the Agile process were more successful in implementing Jira and Cherwell.
Utilising Agile Methodology in a Business Context
Ernest discusses the implementation of Jira and Cherwell, utilising Agile methodology in a business context. He emphasises the importance of coaching scrum masters, developers, and product owners in understanding and implementing agile principles. Product owners play a critical role in this framework, and it is necessary to make optimal use of their time without business analysts and limited stakeholder availability. The use of Jira backlog for managing incidents, service requests, and requests for changes is also highlighted. Additionally, Ernest touches on the need for communication with the business to determine the feasibility of accommodating service requests in the sprint.
Figure 4 Agile Framework and Process
Agile Framework and Responsibilities
To ensure effective project management in a sprint-based framework, it is essential to prioritise tasks based on urgency and engage with stakeholders. The product owner represents the business and understands the project's vision and requirements, while the scrum master guides and coaches the team. The development team must possess strong analytical capabilities to communicate effectively with the business. Sprint planning, refinement, daily scrums, and execution are integral ceremonies.
Agile Development and Sprint Process
The sprint retrospective is a crucial meeting held at the end of a sprint to evaluate the project's performance and identify areas for improvement. It involves analysing the sprint and product backlogs and delivering a minimal viable product. The objective is to deliver and demonstrate a working product to the business, maintaining communication and demonstrating progress. The sprint planning session is held on Mondays to refine the sprint plan, including stories from the backlog and subtasks required for delivery. Throughout the two-week sprint, daily scrums are held to track progress and ensure product delivery. Feedback from the demonstration to the business and the sprint review is used to make changes to the product backlog. The sprint retrospective is held on Fridays before the next sprint planning session to evaluate the sprint and make improvements for the next sprint.
Agile Project Management and Sprint Planning
Agile project management and sprint planning involve using a product backlog, a living log of all business stories or requirements. Multiple sprints can be open simultaneously, with an active and inactive sprint. If a story cannot be completed in the current sprint, it may overflow to the next sprint or be ended, and a new story is created. Stories are rarely deleted from the backlog as they represent business requirements. The demand pipeline prioritises objectives for the year, which are allocated to team members based on expertise. Engaging with the business ensures visibility of new demands and updates to the sprint, enabling the team to iterate and improve in each sprint.
Figure 5 Sprint Planning
Agile Sprint Planning and Delivery Process
The delivery of functionality to the business is organised into sprints, which are planned using the demand pipeline, high-level plan, and feedback from demos. The sprint plan considers factors such as leave, training, and varying abilities of resources. A draft sprint plan is already in place before the sprint planning meeting, and developers commit to what they can deliver within the sprint. The 80/20 principle is used, only planning 80% of a resource's time to account for unknowns. The process flow of delivering a new solution is documented in a spreadsheet, categorising steps into stories. Incidents or service requests can impact the demand pipeline and high-level plan, requiring engagement with the business to discuss implications.
Management of Delivery Subtasks
To develop a vanilla solution concept, a template for new delivery includes stories and subtasks. A spreadsheet is created to select stories and subtasks and generate a CSV for Jira Import. The template is reusable for story and subtask creation, making it particularly useful for repeating stories and small subtasks. To prioritise and manage deliveries based on available resources, stakeholders negotiate the demand pipeline with regular updates and feedback sessions. This negotiation process helps manage clashes between functions and ensures good rapport with the business.
Importance of Sprint Planning and Stand-Up Meetings
Sprint planning and stand-up meetings are critical components of the agile approach in software development. Sprint planning should be done while considering future sprints, resource capabilities, and sprint execution. Stand-up meetings should never be skipped and follow a consistent rhythm to prevent the breakdown of the agile approach. During stand-up meetings, developers should discuss their current tasks, completed work, work for the day, and any impediments they face. Promoting valuable interaction during these meetings is important so the team remains engaged and the meetings are not reduced to mere box-ticking exercises.
Figure 6 Sprint Execution & Stand-up's
Agile Frameworks and Team Communication
Effective team communication and prioritising service requests from the business are key to successfully implementing an Agile framework. In a DevOps environment, remaining flexible and monitoring tasks that may overflow into the next sprint is important. Business availability can affect scheduled testing sessions, so being adaptable is necessary to ensure their needs are met. Optional catch-up sessions can also be implemented to ensure that the team remains engaged and that meetings are not reduced to mere box-ticking exercises.
Agile Delivery and Business Engagement
Agile methodology ensures consistent delivery of promised results to the business, leading to compounding success and better engagement. Successful delivery, in turn, stimulates the business, creating additional requirements and giving the team a reason for existence. Data management practitioners should prioritise delivering solutions that provide value to the business, even if they are bespoke solutions and not necessarily master data management. Maintaining focus and avoiding multitasking is essential, which can lead to slow progress and time loss from switching between tasks.
Time Boxing Unknowns and Data Management Team Involvement
In a development and operational support environment, unknowns can consume significant time, impacting sprint delivery and the ability to fulfil promises. Time boxing is essential for managing unknowns. The limited development team may need to get involved in operational support, especially when dealing with data steward requests for system extracts or bulk updates. Data issues are often a business responsibility and may be logged as incidents, requiring collaboration with the data steward to rectify the issue and ensure proper proof of request. A balancing act is required for the small team, but capacity issues may arise in the future due to the number of solutions that need to be handled.
Sprint Reviews and Retrospectives
To ensure the solution is error-free and functioning correctly, the team conducts peer design reviews and demonstrations before presenting the working solution to the business during sprint reviews. This process not only catches any issues early on but also creates a sense of trust and confidence in the team and solution. Additionally, the business can see the journey the team is going through, creating a sense of partnership.
Figure 7 Sprint Reviews & Retrospectives
Importance of Engaging with Business and Agile Approach
When building a solution for a business, it is important to involve the business to create a sense of partnership and ensure that the solution aligns with its specific requirements. This involvement also facilitates familiarity with the solution through training and user acceptance testing (UAT), making deployment more efficient. Adopting an agile approach allows for course correction based on feedback from the business, which is essential for the early detection of potential errors. Engaging with the business and showing them the solution early on leads to better value and feedback, which can help avoid long-term failures.
Benefits of Data Steward Engagement with MDM Process
Engaging data stewards in the MDM (Master Data Management) process has resulted in various benefits. The delivery pipeline has improved by building better relationships with data stewards, and urgent requests are now prioritised based on sprint timelines. This has helped avoid long-term failures and led to better value and feedback. The relationship with businesses has also evolved due to the MDM process.
Best Practices for Retrospectives in Agile Teams
Ernest discusses the importance of retrospectives in Agile, which should provide a safe space for celebrating team success and progress. The focus should be on actionable steps for improvement rather than placing blame. The person leading the retrospective must manage the conversation and emotions of team members to avoid counterproductivity and team destruction. Moreover, the text highlights the significance of identifying the causes of disruptions and delays in the sprint cycles to ensure successful subsequent sprints and ways of working.
Analysing Disruptions in Agile Sprint Cycles
The importance of identifying the causes of disruptions and delays in agile sprint cycles to ensure successful subsequent sprints is discussed. The focus is analysing disruptions using analytics, reflecting on the retrospective process, and taking action to minimise interruptions. Ernest acknowledges that certain times of the year may cause more interruptions but suggests addressing direct problems within their solutions. He suggests constant analysis to forecast potential delays, and Ernest mentions keeping an eye on the stand-up meetings to be aware of any delays and interruptions.
Importance of Developer Skills and Efficient Processes
Ernest notes that in the business environment, technical skills alone are not enough for developers to be effective; they must also engage with the business and understand its requirements. Analyst developers are essential as they can listen, understand and communicate effectively with the team. Efficient and streamlined processes are crucial in an agile environment for incident management, change management, deployment, and peer reviews, contributing to the team's overall efficiency. Regular retrospectives are essential for evaluating and optimising processes, and peer reviews are important for maintaining the quality of the solution. Team support promotes a positive working environment where team members can thrive.
Figure 8 Data Management Lessons
Importance of Team Support and Agile Development
Team support is vital for promoting a positive working environment in Agile development. This approach emphasises team over individual failures and encourages all members to work together to reach a common goal. Scrum Masters should be part of the team, involved in all discussions, and available to support wherever needed. Additionally, it is important to recognise that lengthy business requirements templates are often not read or understood effectively.
Importance of Minimal Viable Product and Agility in Software Development
The key to efficient development is to focus on minimally viable products, which should be shippable and instil confidence in the business. Agility is crucial, even if it means substituting stories within a sprint to accommodate changing business availability. Understanding the synergy effect between different solutions is important, as solutions developed years ago can now add value to new solutions. To improve productivity, developers should be involved directly in the analysis process and demonstrations.
Role of Developers and Business Analysts
Combining the roles of developers and business analysts can lead to faster processes and improved productivity. Direct involvement in the analysis process and demonstrations enables access to customer feedback, resulting in quicker delivery to business without requiring additional players. Developers take pride in creating something from scratch; immediate positive feedback from businesses can boost their morale and motivation. Additionally, leveraging solutions developed years ago can add value to new solutions, further enhancing productivity.
Data Modelling in Agile Development
The use of business rules in data modelling within an agile development environment is emphasised in a discussion. The importance of peer reviews and design sessions in confirming business rules and ensuring clear requirements is highlighted. Data modelling is crucial in identifying and applying business rules in the data model. The design session involves initial design by the developer, followed by refinement and validation of business rules with input from the team and data modelling background. Challenges with interacting with business analysts and the need for efficient communication and formalisation of rules within the development process are discussed. Ernest also notes the shift towards a single-touch approach with business analysts, data modellers, and developers in the same role.
If you would like to join the discussion, please visit our community platform, the Data Professional Expedition.
Additionally, if you would like to be a guest speaker on a future webinar, kindly contact Debbie (social@modelwaresystems.com)
Don’t forget to join our exciting LinkedIn and Meetup data communities not to miss out!