Leaderboard

I have not been trained or certified as an MBB but I can apply what I have learned in this course. Here's an example of an AI solution technically is working as it should but has become a part of the problem. Consider a business who has a customer service center and their customers are experiencing long wait times. In an effort to decrease the long wait times, they create a Chat Bot. After implementing this AI solution they certainly can see the the call wait times has significantly decreased because the Chat Bot can "answer" them quickly. So, technically, this AI solution is a success. Wait times have drastically decreased. But the company begins to hear from their customers how angry and frustrated they are, even more so than when they had to deal with long wait times. The business failed to understand that what they should have been really trying to solve was increasing customer satisfaction, not merely the symptom of addressing long call wait times. The Chat Bot caused greater unsatisfaction because customers now have to make repeated calls (even though they don't have to wait) because the many "simple" calls are often precursors to more complex issues and the Bot could not handle these, thus forcing customers to start over with an agent, which leads to more frustration. Also, agents may now have to deal with more calls from customers because the Chat Bot did not properly diagnose the underlying problem. This situation wasn't created by the Chat Bot, but by those who didn't have the foresight to really understand how they should have created the Chat Bot. At the end of the day, technology or technical solutions, such as AI, will not be blamed for these problems that arise. Those who created the AI solution will be. You don't want to be that person. Back to the original thought of creating an AI solution. The business thought it was to merely solve lowering long wait call times. But the real root of their issue was customer frustration and dissatisfaction. Their "AI solution" was focused on the wrong thing and it even caused a deeper problem for them How could this have been prevented? Using the FRT process and documentation which captures the Desired Effects (DE), the Undesired Effects (UDE), and the Negative Injections (NI) of any AI project and solution. FRTs can help to envision the ideal future state of an AI solution but also proactively identify negative consequences BEFORE a dime gets spent on creating the solution. The FRT would have captured the root cause by addressing and thinking through the UDEs and also creating NIs to create answers for these UDEs. Utilizing the FRT process and documentation, along with creating a very thorough and thoughtful BRD, would have greatly increased a proper AI solution that result not only in lowering call wait times, but mor importantly, raising customer satisfaction.

Some interesting answers to the question. There are 2 winners for this question - Mark Wexelberg and Sargun Diwan.

In 2022, Klarna launched a full-speed AI deployment automating most of its processes using AI solution and realized cost savings equivalent to 700 FTE. One of the processes they automated was their Customer Service Support. After a while, customer complaints and dissatisfaction ballooned. Customers claimed that AI responses were too generic and unhelpful when dealing with real-life problems. While AI solution like chatbots can handle simple and repetitive queries, emotions or complex issues were not addressed. Klarna realized that while AI solutions promise speed and cost savings, it can compromise service quality and customer satisfaction. Klarna decided to rehire employees to address poor service quality and customer complaints. This is a testament that AI solution isn’t about replacing humans, but rather, enhancing the human workforce with smarter tools and better support system. As an MBB, following were my recommendation: 1. Use VOC to identify critical customer requirements (CCR) where complex issues and customers needing to talk to human to solve their concerns will surface. 2. AI solution aims to enhance customer experience leveraging on personalized interaction for higher engagement. This was not apparent in case of Klarna. It is recommended to take advantage on Deep Learning capabilities of AI solution. Such model can identify complex patterns, making it suitable in image recognition, voice recognition, and natural language processing. 3. Lastly, while drawing the to-be process map, HILT (human-in-the loop) principle is recommended. In cases of complex customer concern, AI can escalate the concern to its human counterpart to further assess the given concern and provide necessary resolution.

Example - Think of a situation in Human Resource Management domain where AI model is built to analyze and reduce employee turnover. In the absence of a MBB, automation/transformation team defines the problem as “predict which employees are likely to leave”. The model accurately predicted attrition rate basis the available data but failed to point out the root cause of the problem and reduce attrition. Only later by involving a Master Black Belt and post thorough root cause analysis it was tabled that the primary cause of attrition was lack of career growth opportunities within the organization which led to high attrition. The primary root cause was not a part of the HR data that the AI model was built on, hence incorrect problem statement will not result in the desired output. MBBs contribution to problem definition stage – 1. MBBs will ensure that the project goal aligns with the Business Case 2. CTQ drill down – Business Case will then be linked to operational objective and post baselining the current performance RCA will be conducted 3. RCA – Using techniques such as 5-WHY, Fishbone analysis , Affinity etc. root cause of the problem can be arrived at 4. Hypothesis Testing – Testing the hypothesis to identify the true trend before full scale deployment of the model is as imperative as the pre-work like clear problem definition Conclusion – To ensure successful deployment of AI model which gives desirable and effective output the pre-work done involving clearly defining the problem statement, it’s link to the strategic and operational objective, CTQ drill down, RCA etc. is imperative for MBBs to ensure the project is directed in the right direction.

AI solution solving the wrong problem is due to the failure of managing the scope correctly. It delivers results that is irrelevant or misleading, wasting time, resources and poor decision-making. Hence, defining issues accurately is essential. For instance, a bank that utilizes AI to predict which customers shut their accounts. The AI spots those customers with accuracy, but the fix doesn't cut down on customer loss. Because the real issue was not about identifying who would leave the bank, it was getting to the bottom of why they were going and how to keep them around? When determining the problem, ask who will be departing instead of asking why they are departing and how we can keep them. AI answers might be spot-on, but they don't help the company achieve its true aim. The main cause gets missed, so any steps taken based on what the AI predicts don't deal with the actual problem. MBBs can play a crucial role by: 1. Asking the right question and probing, Are we dealing with the actual problem? 2. Using root cause techniques such as 5 Whys and Fishbone diagrams to examine more before rushing into AI modeling 3. Setting clear success criteria and focusing on results like keeping customers instead of just how well forecasts work MBBs ensure that AI solutions are both technically correct and practical by helping teams frame problems correctly.

MBBs have deep expertise in process optimization and structured problem-solving, and their role in structured problem-framing approach is paramount, let’s understand this especially in AI solution implementations in high-touch environments like Contact Centers. The problems of Mis-framing leading to ineffective Solutions Contact Center Chatbot Deployment Scenario: A contact center faces long customer waiting times. To quickly reduce Average Speed of Answer (ASA), leadership launches an AI chatbot to handle frequently asked inquiries, aiming to ease pressure on live agents and thereby reducing ASA. Surface Level Problem statement: The project team stated the problem as “We have long wait times because our agents are overwhelmed. Let’s implement a chatbot to handle FAQ’s and reduce wait times by 50%.” While investing 100,000’s of dollars to develop an AI chatbot, train it on FAQ’s, and deploying it as a FPOC for all customer inquiries. The chatbot in itself was technically proficient, using NLP and ML algorithms to interpret customer requests. What was missed: The team did not perform a thorough root cause analysis. Key problems included understaffing staffing during peak hours (only 60% of required agents), inadequate training programs that left agents unprepared for complex product inquiries, fragmented knowledge management systems that forced agents to search multiple databases, and high employee churn (45% annually) from workplace stress and limited career advancement opportunities. The Effects of Mis-Framing on AI Performance Following the chatbot deployment, AI gave generic responses to complex customer issues, causing greater frustration among those needing detailed technical support. Instead of reducing call volume, the chatbot generated additional calls from customers seeking clarification on the AI’s responses or requested immediate escalation to human agents. Findings based on BSI Analysis: The pre-implementation baseline, calculated with the Bottleneck Severity Index formula (BSI = Volume × Cycle Time × (1 - First Time Right%) × Severity), showed: • Volume: 1,200 calls per day • Cycle Time: 8.5 minutes average handle time • First Time Right: 65% • Severity: 3.2 (scale of 1-5) • Baseline BSI: 11,424 Post-chatbot implementation revealed: • Volume: 1,350 calls per day (increased due to chatbot escalations) • Cycle Time: 11.2 minutes (longer due to frustrated customers) • First Time Right: 58% (decreased due to inadequate agent preparation) • Severity: 3.8 (higher customer frustration) • New BSI: 20,365 (78% increase) The AI solution made matters worse: with customer complaints increased, call deflection remained below 15%, and net promoter score (NPS) declined further, and the organization having to face increased operational costs due to higher call volumes and longer resolution times. In addition to the above consequences, wastage of resources and loss of stakeholder trusts add to the negative impact of mis-framing on AI effectiveness. Suggested Practical strategies for MBBs to improve problem framing in AI projects a. Engaging in structure problem statement development using LSS thinking and tools o Use SIPOC and VOC to clarify process boundaries and understand demand drivers o Defining CTQ’s and linking them to customer pain points rather than convenience metrics like ASA. b. Apply BSI for comprehensive bottleneck assessment o Train the project teams in evaluating each BSI component Component Key MBB Questions Volume Is the call volume avoidable or failure demand (e.g., repeat issues, unclear policies)? Cycle Time Are agents slowed down due to poor tools or unclear procedures? First Time Right % What’s the root cause of low FTR? Training, systems, or information gaps? Severity Are we prioritizing automation for high-impact or low-impact queries? o Trend Analysis: Ongoing BSI monitoring to spot patterns and predict bottlenecks before they become critical. This enables teams to address root causes proactively instead of reacting to symptoms. o Use Pareto analysis of BSI to identify Top drivers and guide the AI strategy accordingly. c. Facilitating structured problem definition workshops and fostering stakeholder engagement o Run problem framing workshops that bring together diverse perspectives and stakeholders (operations, IT, HR, training and customer experience.) o Use tools like affinity diagrams and root cause analysis techniques to identify underlying issues that may not be apparent to any single stakeholder group and before confirming the need for AI. o Translating insights into well-structured problem statements (what is wrong, where, when, to what extent and impact on CTQ.) o Making use of the RACI matrix to ensure comprehensive problem understanding. • Inform: Keep executive leadership aware of project progress and findings • Consult: Gather input from frontline agents, customers, and IT teams • Responsible: Include customer service managers, training coordinators along with operations teams and customer experience specialists in problem definition sessions • Accountable: Work closely with the project sponsor on the project approvals. d. Deploy Control Measures Before Automating o Test hypotheses through small-scale pilots that test technical functionality and business impact of the proposed solution before scaling AI. o The pilots need to monitor impact on Leading Indicators (FTR, Escalation Rate, Post-Chat Survey Scores) to validate alignment of proposed solution with identified root causes. Hence the mis-framing of problems in AI initiatives may lead to technically accurate but operationally ineffective solutions, wherein MBBs are mandated with the task of diagnosis with discipline. Using BSI as a key metric identifies real process friction points and thereby guiding the organization to ask the right questions before investing in AI, and ensuring the final solution addresses the true constraints, improve customer experience, and deliver sustainable business value.

Problem statement framing and Project outcome is the basic to do for any problem solving. An experience I could share with one of my project which is of Customer Satisfaction Improvement. Project team while working with the required stakeholders had collected the data required to improve the customer satisfaction which had all relevant information from the historical data. The problem statement framed here was what response from agent could lead to DSAT and the team worked on creating a prediction model which determined which statement is apt for CSAT and which could lead to DSAT. It helped the team with sentence framing while responding to customer but the DSAT kept increasing. Since the project was to predict the outcome of sentence formation it did not determined if the overall satisfaction which had different factors involved apart from only communication. MBB while working on the project could help in the problem statement identification by determining the causes of the problem, create factors which are highly contributing to DSAT and then frame an objective statement which should be recommended for problem solving of the project to improve customer satisfaction as a whole.

In Medical coding, AI solutions are usually applied to automate the coding process, reduce errors or enhance revenue cycle management. If problem is not clearly defined, even technically proficient AI may not deliver value, because it may focus on symptoms instead of root causes. Master Black Belts (MBBs), with their knowledge of Lean Six Sigma methodologies, process enhancement, and data-driven problem-solving, are ideally positioned to ensure that the correct problem is identified. Example: AI in Medical Coding Addressing the Incorrect Issue Scenario: A hospital aims to decrease claim denials within its medical coding process, blaming the problem on coder mistakes in assigning ICD-10 codes. An AI system is created to automate code assignments using historical patient data, achieving a 95% accuracy rate in code prediction. However, claim denials continue to be high, and coders express frustration with the AI overriding their expertise in complicated cases. Why It Fails: The issue was misidentified as "coder errors in ICD-10 assignment" when the actual cause is inadequate clinical documentation (for instance, vague or incomplete physician notes). The AI, trained on existing records, accurately assigns codes based on the data available but cannot rectify documentation deficiencies, which lead to denials. The issue here is misinterpreted as "coder errors" instead of "poor clinical documentation". Hence, the claim denials are not reduced by AI effectively. When AI tackles a wrong issue, the resources are not able to provide any impact and there is dissatisfaction among stakeholders. This can be prevented by Master Black Belts by implementing DMAIC, conducting root cause analysis, engaging key stakeholders, validating assumptions, ensuring data correctly, and integrating change management. If the problem is reframed to focus on the quality of documentation, MBBs confirm AI solutions can give meaningful outcomes, such as reduction in denials and improved efficiency in the revenue cycle.