Sohil Changan

As AI enters the process there is a shift of overall responsibilities for humans in the role. As AI is integrated in processes the human intervention shift from daily activities or steps to monitoring process and reviewing decisions generated by AI model. Lets take Incident management process as an example. When AI is embedded in the process for incident management the regular service desk tasks such as ticket allocation, applying work arounds for problems appeared in past will be automated and AI will do this stuff from the historical data and reducing human efforts from the process, such Auto Ticket Triaging, Auto ticket resolution and allocation will reduce human efforts in the process. So The roles for Human in the process will be changed as follows: Role Before AI After AI Service Desk Agents Handle all queries manually Focus on complex, emotional, or edge-case issues Operations Lead Monitor SLAs and coach agents Oversee AI performance, handle escalations, and refine AI training data Process Analysts Analyze ticket trends manually Collaborate with data scientists to improve AI models and workflows KM's Maintain static FAQs Curate dynamic, AI-consumable knowledge bases QA Team Review agent interactions Audit AI responses and ensure compliance/accuracy The Following will be the new roles and skillset required for the role: 1. HIL Specialist - This will trigger when resolution for tickets will have a less confidence and could result in affecting CSAT scores. 2. AI Solution Integrator - Creates a AI model and design prompts for input of the data. design workflows for running the process in a optimal way 3. Compliance Lead - Monitors and ensures all the AI compliance are properly followed. 4. Change Management Lead - Transition existing process to AI enabled processes by following change management principals with aligning with all stakeholders. We can redesign the processes with human AI collaboration as follows: Feedback Loops - To retrain models Process Segmentation - bifurcation of incidents to be handled by AI such as P3,P4 tasks and use HIL for P1,P2's. Transparent hand off's - Use Chatbots at various level such as customer facing and agent facing transparently. They must be aware that AI is been used. AI Performance KPI's - Use KPI's such as accuracy, deflection rate, drift, CSAT so that AI performance can be monitored over a period of time.

AI systems can degrade time based and it can impact workflows without anyone noticing it. Taking an example of customer support process. AI model is implemented for classifying and routing incoming tickets. Below mentioned are the early signs of AI degradation: 1. Drop in accuracy and Confidence scores - The model gives you low accuracy scores to predictions. 2. Increase in Manual efforts for routing and overrides - Agents are frequently correcting AI decisions, No of tickets escalated gets increased. 3. Change in input - Change caused due to new product lines, language shifts. 4. CSAT decline - Negative feedback increases, your customer satisfaction KPI downgrades. 5. Lead Time increases - Model response time increases which leads to decline in customer satisfaction. WIP or ques increases and throughput rate declines for incidents solving. 6. SLA Bridges starts increasing - As tickets are routed incorrectly, there lead time for solving increases and FTR decreases. Following are the ways in which these can be monitored are: Data Drift detection - It gives alerts when your input distribution is deviating from training baseline. Model performance monitoring - It measures confidence drop by x pre defined percentage that is set as baseline for stability in the system. HIL (Human In Loop) feedback - It tracks override rate, agent correction, It gives alerts if HIL override rate exceeds x percentage set as baseline. Business KPI's - KPI's such as CSAT , NPS and other customer feedbacks can keep the system in monitoring. Systemic Testing - Testing can be done periodically for known inputs to check the response, type of a poison test.

For a AI infused processes there is a need to shift from traditional audit approach to a more dynamic audit approach which can models, prompts, flows. Since AI uses these approach for its BAU working so audits should be more focused on data integrity, accountability, ethical compliance. Below mentioned are the approaches that can be used while building AI audit framework: 1. Prompts and Flow Designs Questions to ask - Are they documented and have a version control mechanism in place? How is the ambiguity handled? Check Items - Prompt repository, Flow Diagrams for execution. 2. Model Governance & Lifecycle Monitoring Questions to ask - What type of AI model is used? Who trained the model and what type of data is used? Is there any documents and updation policy? Check Items - Model validation, Testing, Drift detection, data revision mechanism, Schedules. 3. Data Integrity & Bias Questions to ask - Is data input clean? Are there any detection system for detecting Bias? Check Items - Bias Audits, Data steps to be followed, data lineages. 4. Compliance & Ethical Alignment Questions to ask - Does AI comply with regulatory guidelines set up a board? Are guidelines embedded in the design and the system Check Items - Regulatory mapping, gap analysis,. 5. Human In Loop Questions to ask - Are there any human decision points integrated? is there any recovery mechanism if AI fails? Check Items - HITL workflows and paths to be checked. Feedback Loops. Below mentioned are the best practices for ensuring fairness and transparency: 1. Documentation of AI processes should be mandate. 2. Continuous monitoring for drift, bias, performance for the AI model. 3. Proper aligned communication strategy with stakeholders and business objectives. 4. AI audits at a set frequency.

By executing Swiss Cheese model in the process for Operational Excellence. OpeEx is a philosophy for delivering value to the customers and stakeholders by utilizing principals of Lean, Six Sigma, TOC to optimize the processes and improving quality, productivity and reducing cost. Swiss Cheese model identifies how multiple layers can prevent failures. Considering process for Order Fulfillment. Defense layers are: 1. SOP's - Defines How to perform a Task 2. Training - Ensure employee understand SOP's. 3. Quality Checks - Inspection at each stage. 4. Automation - Bar Code scanning for mistake proofing's. 5. Customer Feedback - Captures issue's for continuous improvement. Holes in the Cheese are: 1. SOP's may not be revised. 2. Training not effective. 3. Human error in inspection. 4. Automation may fail. 5. Feedback may not be analyzed properly. This understanding helps in guiding improvements for driving business excellence as follows: 1. Executing FMEA's for proactive risk assessment. 2. Kaizen - driving continuous improvement projects for closing holes such as improving SOP's, training effectiveness and upgrading automations. 3. Improve process reliability by using Lean Six Sigma projects by reducing errors and variations in the process. 4. Performance management - developing KPI's to measure the effectiveness of processes and improve them. 3.

Change Management is the important factor while undergoing a DMAIC project. As the change management is key accelerator for driving improvements, its plays a vital role for ongoing sustenance. Following is the way by which change management impacts DMAIC phases and how a MBB can play a role in efficient management in change. 1. Define - As I lead the project for reducing INC backlogs ,In this phase problem is defined and the foundation stone is set for the project. So its important to set the governance of the project. Set clear communication strategy with stakeholders. Identification of members of CCB(Change Control Board). MBB - MBB helps to facilitate the stakeholder engagement, Clearly set the project charter and define each and every role and their responsibilities for project execution. 2. Measure - In this phase its important to collect accurate data for measuring the current phase and current performance level. I gathered the data for overall factors for backlog and validated the data with the team. MBB - MBB guides in data handling and takes data driven decisions while making statistical inferences. 2. Analyze - In this phase RCA can change processes standards and can impact product or output. So here role of CCB is very crucial to authorize changes while studying there impact on each and every parameter. In the project I ensured overall participation from Solution, Developers, Testers and Ops Team so that everyone is on a same page. MBB - MBB helps in facilitating decisions and conducting RCA workshops and brainstorming sessions. 3. Improve - This is the phase where change is implemented and the resistance from the stakeholders are more likely in this phase so the role of a proper change management system or a CCB(Change control Board) is very crucial. I ensured pilot runs are effectively conducted for implementing solutions for automations and Gen AI. MBB - MBB drives the pilot implementation plan. Coaches stakeholders on effective implementation plan. 4. Control - This phase ensures ongoing monitoring which is again a important aspect of change management. Here a whole monitoring was set with the CFT for monitoring backlogs. MBB - MBB drives the governance for the monitoring and makes Control plans, Select control charts, measures after improvement capability study. MBB in this act as a change leader while focusing on strategic alignment, Coaching mentoring, stakeholder engagement and effective communication.

The Reasons for which the improvements slips away in control phase are as follows: 1. Resistance To Change - This is the common problem in every organization, once the change is reinforced, people tend to resist the change and come back to old ways of working after improvement. 2. Insufficient Monitoring - Sometimes lack of continuous monitoring is there, in which we may miss some data points that can effect process stability and process capability. 3. Lack Of Accountability - RACI should be defined properly per improvement as it slips away when not done properly. 4. Lack Of Training - If employee's are not trained for the new processes and there is lack of documentation it slips away after sometime and do not sustain. These tools and techniques can be used in control phase for sustenance: 1. Control Charts - This can be used to monitor the process and identify any outliers. 2. Standard Process - Proper SOP's need to be developed and taken signoff before implementation. 3. Training and Awareness - Proper training to be imparted for better understanding of Lean Six Sigma and its implementation. 4. KPI's - Performance measurement indicator's to be set for new process or after modification to ensure the target is met regularly.

Following are the problems that are surfaced when AI solution solves a wrong problem. 1. Misleading results - When AI solutions are not aligned with the real or exact problems, They will give results bit will not solve the actual problem,. 2. Inefficient use of resources - Wastage of efforts invested in building, designing and architecting the solution. As the solution is inefficient in solving the real issue all these efforts are a waste which results efforts invested in terms of Man Months, which can directly impact cost. 3. Loss of Trust - As Solutions addressed wrong issue's stakeholders start to loose confidence on AI solutions that can start losing trust and affect sponsorship for budget allocation and project execution. 4. Consequential Harm - If AI solutions solves wrong issues in critical areas like healthcare, Legal & compliance it can lead to serious consequences that can harm the reputation of the organization and will result in negative impact on brand. Following are the some of the ways in MBB's can help in identifying correct or real problem statements for deploying AI solutions. 1. In Depth Analysis - MBB's go exactly deep for finding out the actual problem with relevant data analysis for backing the problem statement. 2. Stakeholder engagement - MBB acts as facilitator for driving engagement 3. Data driven insights - MBB uses data to drive statistical inferences for validating results and decisions taken during execution of the AI solutions. 4. Benefit Analysis - MBB helps in visualizing the benefits in terms of value, cost and efforts invested.

AI governance in a organization should be linked to organizational strategy with all the pillars such as continuous improvement , Customer value, Performance measurement, Leadership Involvement. It should be a enabler for sustainability and operational Excellence. These are the following ways AI governance should be in organization: 1. Strategic Alignment - AI governance should support operational excellence by enabling AI initiatives. It should drive continuous improvements and process optimization to enhance efficiency and customer value through AI integrated solutions. 2. AI governance Pillars - AI driven organizations should have these pillars such as: 2.1 Responsible AI - AI ethics board setup is required to maintain the fairness, transparency and accountability. 2.2 Performance - AI KPI's to be integrated in Scorecards for monitoring 2.3 Risk Management - AI assessments should be conducted regularly for mitigating any risk towards AI enablement and to plan the measures for the risk. 3. Culture Building - AI awareness trainings to be mandate for all the employee's to maintain AI literacy level. These levels can be designed by AI governance Board. 4. Governance Board - A CCB or a AI governance board to set up for proper governing the overall AI development in the organization.

As organizations would be more inclined towards AI enabled decision making or a AI driven organizations. Traditional Metrics measuring Quality. Cost and efficiency will be out dated and there will be metrics measuring quality and effectiveness of solutions rather than Time parameters. There will be more focus on predictability of problems or defects as organizations wants more insights on predictive analysis and forecasting accuracy. Here are some examples for Metrics that will be outdated and metrics that will be in place to be monitored: 1. AHT(Average Handling Time) - So when organizations would be AI enabled the measure will not be on the Time taken to resolve a problem as Bots and AI chatbots will be there to resolve it. The measure would be what is the effectiveness of the solution. This can be measured through CES(Customer Effort Score), it will measure How customers are satisfied with the solution and ease of implementation. AI Assisted Resolution rate, it measures how many problems are resolved by AI solutions. 2. Forecast Accuracy(MAPE) - It measures accuracy of forecast with real world. But it does not show volatility or drift. As in drift in data inputs with time. This can be replaced by Drift detection rate, it will measure and trustworthiness and stability of AI forecast.

The ways in which AI Team and MBB can create a better solutions are: 1. There should be clear roles and responsibilities divided between AI Team and MBB's so that they can work in a synergy and can contribute more. This can often prevent from conflicts between two and overall help in utilizing skills from both of the teams or individuals. Following are the Role and Responsibilities of a MBB and AI Teams are: MBB Focus should be on improving and analyzing process for optimization and value generation. Customer value should be on priority. To align AI solutions with organization's goals. Focus on structured problem solving. Value generation for business in terms of financial success as well as customer outlook. AI Team Focus mainly on solution architecture. Overall technicality of the solution. Idea generation for problems in brainstorming sessions. Implementing AI solutions. 2. Proper Communication between both the sides by means of workshops, brainstorming sessions, so that they can have an idea of each others zone and the gap can be bridged between two. This will create a synergy between two and help the organization to be more efficient. 3. Focus on each others strength, As MBB's are more skilled in mapping the process, defining problem statements or analyzing process and identifying hidden problems and bottlenecks from the process. Using data for creating meaningful inferences and validating benefits for value realization and on other hand AI team is skilled for generating idea for automations and creating AI solution architecture for problems. They are skilled in implementation of solutions and providing technical support wherever required. So both the skill sets are important for organization to carry a sustained and a profitable business.