Q 784. Business Excellence thrives on feedback loops — PDCA, DMAIC, A3 — but most AI agents don’t “learn” from process improvements unless explicitly redesigned or retrained.
Imagine an AI-enabled process where regular improvements are made by human teams.
How would you ensure that the AI solution evolves with the process — without becoming obsolete or misaligned? What role can MBBs play in making AI part of the continuous improvement loop?

 

The best answer will be selected on the basis of: 

• Practicality of integrating AI into improvement cycles  
• Clarity in the feedback mechanism for AI adaptation  
• Strategic role of MBBs in maintaining alignment

 

Note for website visitors -

• This platform hosts two weekly questions, one on Monday and the other on Thursday. 
• All previous questions can be found here: https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/. 
• To participate in the current question, please visit the forum homepage at https://www.benchmarksixsigma.com/forum/.
• The question will be open until Monday or Thursday at 5 PM Indian Standard Time, depending on the launch day. 
• Responses will not be visible until they are reviewed, and only non-plagiarised answers with less than 5-10% plagiarism will be considered for winner selection. 
• If you are unsure about plagiarism, please check your answer using a plagiarism checker tool such as https://smallseotools.com/plagiarism-checker/ before submitting. 
• All correct answers shall be published, and the top-rated answer will be displayed first. The author will receive an honourable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

• Some people seem to be using AI platforms to find forum answers. This is a risky approach as AI responses are error-prone because our questions are application-oriented (they are never straightforward). Have a look at this funny example - https://www.benchmarksixsigma.com/forum/topic/39458-using-ai-to-respond-to-forum-questions/

• We also use an AI content detector at https://quillbot.com/ai-content-detector. Only answers with less than 45-50% AI-generated content will be considered for winner selection.

To ensure that AI solutions keeps evolving progressively without misalignment, AI should be embeeded within the existing improvement cycles as a participant in the loop and not just as a seperate tool. This is very relevant in Nigeria as it is a resource conscious settings where very lean adoptions of Ai solutions must deliver a measurable value without it requiring any constant expensive retrainings.

1. The practical integration of AI into impovement cycles: To reduce thw cost and complexity of AI soltions, Organizations in Nigeria can use low-code AI platforms that allow team leads make frequent updates to logics or just retrain model themselves without an heavy dependency on data scientists.
2. Clearing feedback mechanism for AI adaption is crucial in Niagerian industires like manufacturing or financial institutions where policy shifts can drastically change baseline condition.

AI systems are not fight and forget like the Nigeria road(haha) after a rainy season. they always require regular check ups and maintenance. 

When we talk about Continuous improvement, as a concept, we refer to a constant WIP mode of innovation, enhancement and incremental progress. Take all possible learnings and loop it back an input to further refine a product or process.

VOC, VOB, error types, new data or new pattern or behavior of a certain process or a machine that can be studied, performance monitoring. Getting RCAs. Feeding it back into the system and closing the loop of a continuous self-learning improvement process with the help of Artificial Intelligence.

 

1. Natural language programing Reinforcement Learning :
   • AI agents can learn by interacting with end users and learning best practices from online forums and receiving feedback (rewards or penalties).
   • Over a period of time, AI models or AI agents can improve their decision-making based on outcomes.
   • Example: AI in customer service optimizing responses based on customer satisfaction scores. Also, in an AI agent environment if a customer rates a low score or not resolved on a survey. AI can ask customer if he or she wants to get redirected to additional support
2. Utilize online libraries:
   • System based or conventional training methods have a focused content and is periodically reviewed once or twice a year. Unlike traditional models trained once on a fixed dataset, online learning allows models to update continuously as new data arrives.
   • Can prove to be extremely useful in dynamic environments like fraud detection system that can improve itself whenever a new fraud pattern emerges or email classification or query categorization.
3. Optimized Human-in-the-Loop (HITL):
   • AI backend can incorporate VOC of output or a human feedback to refine and improve performance. On a continuous basis which a key component of continuous improvement
   • For example, customer service agents correcting AI-generated email drafts helps the model learn better phrasing, grammar, formatting, and tone.
4. Use concepts like A/B Testing and Feedback Loops:
   • A tried and tested AI system can test different strategies (e.g., email templates) and learn which ones perform better.
   • Manual or online VOC and Feedback loops help the system adapt to changing customer behavior or business goals.

---

e.g. In a Banking Email Customer Service Context:

• AI can learn from:
  • VOC (NPS scores, complaints and RCA)
  • Agent corrections to AI-suggested replies, check of all queries are answered in a multi query email.
  • Frequency and Escalation patterns (e.g., which types of queries lead to dissatisfaction)
  • Compliance checks (to avoid regulatory violations)

 

Though there will be challenges to guarantee a continuous improvement on AI based models, if we study it enough it can be overcome.

 

Challenges like

Propagation of systematic Bias. For e.g. an AI model might be more favorable to a certain type of machine or high-performance shift timing, or certain region in terms of Sales etc.

Distribution or pattern shift. Or drifting of parameters,

Real world the situation changes dynamically so AI will have to be trained to Adapt. Failing which it will follow a fixed pattern and might not necessarily be effective.

 

In manufacturing or Healthcare sector or if we speak from a Six sigma perspective AI can

 

1. Conduct SPC if we feed it in initial stage. Analyze process deviations
2. If we find some points or processes out of control, we will implement solutions to get the process in control, AI agent can learn from such corrective actions
3. It can also Suggest process changes to reduce defects depending upon previous corrective actions taken by us or information available online.
4. Would be better poised to predict process output or future failures or improvement opportunities

Most AI models, after deployment, are static, and without continuous improvement, they are vulnerable to becoming outdated. AI can be trained for constant improvement with a feedback loop intentionally embedded.

The system must be designed for adaptability to ensure AI solutions evolve with continuous improvement efforts.

Steps for Continuous Improvements

• Ingestion on a regular cadence, Updated Process Data, and Human feedback
• Automating feedback Ingestion: Feeding monitoring data and data pipelines back to the model
• Retraining: Automating the retraining of the model based on KPI
• Framing the Problem: AI should address the problem within its proper context.
• Governance: Aligning AI outputs with current process goals and standards.
• Decision Makers: Keep humans in the process of validating, optimizing, and overriding for better learning.
• Auditing: Versioning for clear visibility on when, how, and why AI models are updated with business alignment.

 

AI model improvement is not a “one-size-fits-all” nor a “one-time” solution, as with every other model in the world; things keep changing based on new data available, new processes, new ideas, and new and improved technologies. Therefore, it should be a living component of the process.

If medical coders or auditors recognize any persistent issues, or if Business Excellence teams alter workflows (for instance, by modifying DRG validation checks or POA flagging protocols), those insights are frequently confined to process documentation, unless they are explicitly reintegrated into AI retraining or rule modifications.

How to Ensure AI Evolves with the Process in Medical Coding - Practicality of Integrating AI into Improvement Cycles
This is achievable only if AI is regarded as an ongoing participant in the process instead of a  fixed tool.

In Medical Coding, regular coding audits, patterns of errors, and feedback loops from inquiries can be systematically gathered.

Feedback Mechanism utilize organized, documented, and classified AI error logs, accompanied by established retraining timelines and configuration modifications.
MBB Role Act as stewards for AI process alignment, oversee AI feedback cycles, spearhead AI-impact PDSA/DMAIC initiatives, and promote governance for enhancements in AI.

AI can be trained to learn from continuous improvement initiatives, and when combined with Lean Six Sigma framework, it can bring huge transformational impact. Continuous Improvement gives AI context and AI gives continuous improvement speed and scale. Together, they build agile and intelligent operations.

AI models equipped with ML can analyze vast volumes of process data from multiple iterations of improvement projects to identify failure modes in value streams. It can also identify hidden patterns in defects, waste or quality.

 

By embedding DMAIC Logic into AI workflows, AI models can detect outliers and abnormal trends in real-time, it can suggest next-best action or root causes, simulate and evaluate impact of corrective actions, create documentations, trigger alerts or notifications, etc.

 

Moreover, by feeding results of continues improvement feedback into the model, it can continuously refines its predictions and smarter prioritization over the time.

 

With this approach, organizations can achieve operational Excellence at scale. They can automate improvement prioritization and decision-making which will allow teams to move from reactive to predictive. Cycle time for improvement can be reduced. With intuitive AI agents, non- six sigma people can also engage in problem-solving, making continuous improvement part of daily operations.

To ensure an AI-enabled process stays aligned with continuous improvements driven by human teams, the AI must be redesigned into the organizations existing feedback loops and its like how we treat any other critical process tool.

Incorporate AI feedback into the PDCA/DMAIC cycle

 

Performance should be monitored as part of the Control phase in DMAIC or the Check/Act phase in PDCA.

Need to clearly define clear metrics for the AI, such as accuracy, false positives/negatives, or ROI impact.

Build a routine cadence daily /monthly, quarterly to review these metrics alongside process KPIs.

Close the loop with structured human feedback

 

Design touchpoints where front-line process teams and SMEs can flag AI misclassifications, missed opportunities, or changing business rules.

 

Use simple feedback channels — e.g., error tagging dashboards, feedback forms, or auto-logging when exceptions are handled manually.

 

Feed this data to the AI team for periodic retraining or rule adjustments — this makes the human-in-the-loop model part of standard work.

 

In order to ensure the AI system evolves with the process, I would design the AI system with a built-in feedback mechanism that’s linked to the human lead process improvements. Every time a team completes a PDCA, DMAIC, or A3 cycle, the key insights and process adjustments should be captured and fed back into the AI system. It would be a standard process to document every process improvement initiative in a specific template, generate relevant training data and feed this to the AI.

An MBBs role here would be to bridge the gap between the process excellence teams and the technical teams managing the AI. The MBB will also be responsible to create the templates needed to capture the insights and updated process steps (If applicable) which can be feed back to the AI through the established feedback mechanism. The MBB would also need work with the technical team to set up a process to effectively use this template for training data generation. Additionally creating and managing a governance plan should also be the MBBs responsibility.

To make sure an AI solution grows with ongoing process enhancements, we need to think of it as a living part of the improvement system—not just a one-time setup. This starts by putting AI into the same feedback loops that power Business Excellence, like PDCA, DMAIC, and A3. The main objective is to create a continuous learning system from which the insight collected from the process modifications is leveraged to refine the AI system.

Ways to adopt AI to advance improvement
1) AI integration at the operational level:
a) Integrate AI in prevailing improvement cycles like PDCA, DMAIC and A3 for implementation
b) Adopt MLOps tools to retrain AI after process modifications
c) Align process review periods with AI update implementations
2) Clear Feedback System:
a) Record improvement data like KPIs and root causes in organized ways
b) Enable checks to spot model drift or performance gaps
c) Start model updates when real-world results don't match up
3) MBBs key role:
a) MBBs to act as mediators for process owners and data science teams
b) Define goals for both process and AI success
c) Make sure AI models shows the latest process standards and improvements

AI can evolve with continuous improvement if it's designed to learn continuously. This requires automated feedback mechanisms, structured data from improvement initiatives, and strategic leadership from MBBs to ensure alignment and sustainability.

Answer: MBB’s plays crucial role in AI-enabled process where regular improvements are made by human teams.

Communication and clarity are really important for effective collaboration between human and AI, where AI systems can clearly summarize their decision-maker processes while human can effectively communicate their vision and feedback, resulting to create a trust and helps for effective collaboration.

In this process, AI model learnt from human guidance and behavioral pattern which allowing them to refine algorithms and improve their performance, while humans develop new skills and perspectives through their work with AI. This whole process will create virtuous cycle. Refer below flow chart:

 

 

The main role of six sigma experts (Master black belt) is to identify the areas that needs attention and accordingly reduce the process variability through applying different approaches i.e. DMAIC (Define, Measure, Analyse, Improve, Control) and DMADV (Define, Measure, Analyse, Design, Verify), PDCA (Plan Do check act) and A3 technique – which a structured problem-solving and continuous improvement approach MBB plays an important role in context of continuous improvement journey:

· Strategic alignment ensure translation of business issues into data driven opportunities and also Identification of critical pain areas and guide AI tools like Failure mode effect analysis and SIPOC (suppliers, inputs, process, outputs, customers).

· Working together with data analytics engineer to define different factors and responses and data cleansing approach.

·During the process of integration and validation of AI models, various statistical tools like DoE, SPC, hypothesis testing and MLR ensure the accuracy and reliability of the model's assumptions resulting robust process.

· Effective collaboration of AI model in the Six Sigma (DMAIC) provides organizations to improve accuracy, and sustainability in continuous process improvement at initial phase when critical business process is under development phase.

·AI can monitored the process continuously and provide real time data and feedback and as a result improvement are sustained, thus any deviation from the written procedure are immediately addressed.   

 

Clarity in feedback mechanism, improving the performance and reliability of AI systems.

1)     Clear objective of AI model can improve the accuracy and enhance the user experience and overall performance.

2)     User friendly interface can help the user to provide easy and concise feedback.

3)     Guide AI tool to analyze the feedback form in such a way that it is easy to identify the common issues and areas which needs attention.

 MBB’s also helps to align six sigma projects with organization goals which ensuring improvement projects are directly connect with strategic goals of organization.

A six-sigma expert can also prioritize urgency of the project. Additionally, also define the KPIs for continuous tracking the progress of cost saving and improvement projects.  

MBBs are creating a culture, where awareness should be provided to the people about the continuous improvement within the organization which leads to the adoption of best practices which finally enhance the creativity among team member.

 

May I respectfully suggest these questions be answered from an "AI Solution Architect" persona rather than a MBB 

 

Here are the AI/ML techniques I would employ into the Agent, the implementation steps, evaluation strategy and ethical considerations 

 

AI|ML Techniques:

Reinforcement Learning: The Agent can learn actions through trial and error, guided by human feedback as reward or penalties

Active Learning:  This involves selecting the most informative data points for human feedback, which will minimize the amount of labeled data needed while also maximizing learning efficiency.

Human-in-the-loop (HITL): This integrates human feedback directly into the learning process, allowing for real-time adjustments and improvements.  The Agent can capture all the human feedbacks and add it to the KB data, keeping it up to date and relevant.    

Transfer Learning: This technique adapts to changes without having to retrain from LLM scratch. It leverages the pre-trained models but by also fine-tuning them with new data and feedback.

 

An interface, like a web browser, where people can provide their updates and the Agent will append the updated data into the KB file or database.  The Agent will always pull from that updated KB, thus not becoming obsolete and misaligned.   

 

 

   

 

 

• Define the Standard operating procedures for AI models or solution for undergoing review regularly. SOP should focus on data and process alignments along with regular human oversight (ex : Audit)
• Developing automated monitoring systems to assess the data or concept drifts or any performance impacts against the baseline.
• Implement a process to regularly collect feedback from the customer to access the AI performance.

 

The Role of MBBs (Master Black Belts):

• They should act as a process champion for spearheading the continuous improvement activities for AI solution
• Supporting in converting insights into business requirements and leading change management for AI solution

Let us answer this, with the help of an example. 

 

Cricket Ball Manufacturing process

We will consider a Cricket Ball manufacturing process as a hypothetical example
 

Background:

•  Now, let us say we have 2 types of Cricket Ball manufacturers XYou Sports Inc and YMe Sports Inc.
• XYou Sports Inc produces ball in Country 1 where the ball will turn (Spin) more a
• YMe Sports Inc produces ball in Country 2 where there is more swing. 
• Now the national team of Country 1 plays with the national team of Country 2 in one of its Cities…
• Therefore Country 2 feels uncomfortable playing to a different brand of Cricket ball (made by X) as it has got more spin. 
• So, to counter that, Country 2’s Cricket board looks for XYou Sports Inc manufacturer to supply that brand to its national team.

 

The steps mentioned above, depict the AS-IS process view

 

AI-Enabled Process where human improvements were made

As XYou Sports Inc manufacturer found out that, multiple countries may need multiple type of Cricket Balls, it decided to leverage AI in getting useful insights and ultimately getting things done quickly.

As a first step, it ensured that AI process (and hence solution) was setup that addressed that the ball will be more conducive to Spin (turn) when implemented in Country 1. The AI was fed with data, and it was more like a supervised learning for the AI. The humans in the loop, were doing manual verification of key differentiating parameters/factors such as weight of the ball (in Ounces), size of the ball, bounce, seam, stitching, materials used … 

 

AI solution evolves with the Process

Once XYou Sports Inc received the request from Country 2 as well, the team realized that they can still explore on AI technology and leverage it for bettering this process.  It started to identify what kind of key differentiating factors can be included as part of the AI.  The AI solution agent now has to look for an additional information(parameters) – how the prevailing weather in Country 2 impacts the cricket ball, how does the pitch/soil (in Country 2) supports this type of Cricket Ball (manufactured by XYou Sports Inc).  

 

How did the MBB help here for XYou Sports Inc?

The MBB envisaged a plan. As this process is an ongoing journey and may have uncertainties and require constant feedback from the stakeholders (eg, National teams, Cricket Boards of respective countries..), he felt using Scrum as an agile framework might help the team to navigate through the uncertainties that might prop up.. He suggested the team to use this and also requested them to put a Kanban board for radiating information on a routine basis to all stakeholders

 

He had put a high-level plan that stated as

 

Planning Type	Objective	Remarks
Vision Planning	Developing a scalable tech excellence support Agent that supports the Cricket ball manufacturing process of XYou Sports Inc
Release Planning	Release Phase 1:  Defining the AS-IS process to AI (Basic Knowledge Integration of the Cricket Ball manufacturing process)   Release Phase 2:  Re-imagining the defined process in phase 1 by considering all key parameters that is required to make the ball suitable in multiple countries (Country1, Country 2)
Sprint Planning	Release 1 Sprint 1 – Creating the KB agent and AI agent for defining few basic parameters (such as weight and shape) Sprint 2 -  Defining the KB agent and AI agent for remaining parameters such as materials, bounce, seam, stitching   Release 2 Sprint 3 – Defining the KB and AI agent for additional parameters such as pitch soil, weather conditions of a country  (which can influence the ball behaviour)	Duration: 2 weeks Note: Sprint 1 will have a minimum viable product (setting up the AI process and adding very limited functionality)     Release 1 had 2 Sprints and Release 2 had 2 Sprints   Sprint 2 had the feedback incorporated from the stakeholders that came from Sprint 1. Similarly for Sprint 3, 4 the team got feedback from the previous Sprints and incorporated the feedbacks which were relevant to them   Subsequent releases had subsequent Sprints based on the emerging needs for the manufacturer.

 

Integrating AI into improvement Cycles and AI process got adapted with proper feedback

As we see from the above table, the team leveraged Scrum as a framework to build an iterative and incremental development of their AI based Cricket ball manufacturing process.   What was a cumbersome exercise in getting huge amount of data across multiple parameters and few parameters which were changing based on the Country and its weather conditions (worst if there are multiple weather phases – summer, winter, autumn..in a country) , it became much more simpler when done with AI. Every Sprint produced some incremental values keeping the stakeholders happy.

 

The AI system improved itself over a time period as it gathered more data (in the usage of how the cricket balls behaved in those countries across weather seasons) and had reinforced learning to improve itself

 

With every Sprint, there was a Sprint Review that happened where the stakeholders were presented with the finished work (in the Sprint).  Whatever feedback was given was taken into consideration and those were implemented.

 

Strategic Role of MBBs in maintaining alignment

The MBB was able to devise a strategy for AI solutioning of this Cricket Ball manufacturing process. He was able to setup a vision, release plan and then help the team to come up with Sprint goals for each of the Sprint making the team adjust to the uncertainties

 

Conclusion:

We saw here how AI enabled process with human efforts can navigate through complex scenarios/situations in an incremental manner addressing emerging needs with quick feedback cycles. The most important thing is quick release(deployment) of the value that you want your stakeholders to get and have continuous exploration of the market needs and adapt your system accordingly.

 

Most of the AI agents need continuous redesign to adapt the new optimized process changes & improvements. however futuristic approach should be that AI learns and adapts the changes themselves and evolves as human drive changes rather than a still component

 

1.      Ensuring AI evolves with process

Integrate AI into continuous improvement advise chain- that is treat AI as process agent and assess its performance under new conditions

Monitor prediction determine model confidence level

 

2.      Create Model Governance

• Schedule validation to calibrate and assess whether the AI is performing basis latest process improvements.
• Change Management: Whenever a process change or new development made, review an impact assessment on performance.

 

3. Creating “Human-in-the-Loop” (HITL) Feedback mechanism

• AI should be designed to receive feedback from users correcting AI deliverable .
• Human can review, compare and validate the AI output delivered and establish whether it was at required benchmark

 

3.      Continuous Learning basis backed up data

·        The new data can assist AI for the required change.  you may use this data for to retrain AI

• Ensure all AI inputs, and outcomes are logged in structured & defined formats.

What Role Can MBBs Play?

MBBs play as a bridge for process improvement and AI adoption:

1. CTQ and output Alignment

• MBB reviews whether delivered AI solution meets customer requirement and critical to quality
• collaborate with different functions from process owners to development team and assess the performance basis requirement

 

2. Performance Measurement

MBBs can help define AI metrics to assess how well the AI is adapting to process improvements and so can assess its contribution to the overall improvement goals.

 

3 MBB helps in developing the Data Governance: MBB can ensure the quality of data, its ethical use and privacy as the AI continuously learns from evolving process change

In a bigger context, AI can be trained to improve step by step over the time as how humans improve based on regular practice and feedbacks.

 

There are some better ways of training AI towards continuous improvement:

 

Online Learning: Regular trainings will have a fixed data set and the analysis will always provides same results irrespective of multiple trials. But in the online learning, AI models assist in getting the data set update with new data’s as and when required which improves the results and the solutions and help in adapting to different conditions. It also shows the predictions will vary based on data availability.

 

Trial & Error Learning: This is kind of a Reinforcement learning where the agents will chance their actions against suitable strategies to show the improvement in performance over the time to get rewarded.

 

Refresh the prompts: Regularly re-train the AI systems with new or modified prompts, newer data sets or prompt to react based on the interactions, will make AI system to get multiple iterations and align with different scenario’s to provide better results.

 

For example, the search engine will provide better and relevant results based on how to prompt and what we select.

 

Hence, Continuous improvement is the key or backbone of how AI systems developed. However, the AI models will not improve themselves autonomously, but the developers of the models play a vital role in providing how it should enhance and perform better.

Yes, absolutely correct that AI and strategic thinking collide

Here goes how AI can be plaited into continuous improvement loops like PDCA or DMAIC framework.

AI Learning via continuous Feedbacks

AI models, especially those integrated with machine learning, can be designed to monitor process performance, Detect inefficiencies or anomalies in real-time. Analyse outcomes and compare expected vs actual results to identify areas for refinement. Adjust dynamically by Use reinforcement learning or online training to adapt based on feedback loops.

 

MI models traditionally relied on static data, but continuous learning enables them to continuously update and refine their understanding. The process of continuous learning involves initial model training, updating the model with new data, and evaluating its performance.

 

DMAIC Frame & AI Frame

 

Define – (AI – Study customer Data to help and define problems)

Measure – (AI - Use logs, digital data to track performance)

Analyse – (AI -  Predictive model)

Improve – ( AI – simulation such as Monte Carlo simulation)

Control – (AI – Monitor Metrics & prompt for any deviation against standard)

Examples

·  In manufacturing, AI learns from production data to reduce 8 kind of waste and improve quality — form of real-time Six Sigma.

·  In Service Sector, AI RPA analyse customer complaint patterns and self-adjust responses or escalate intelligently.

·  In IT, AI-driven observability platforms continuously learn from incidents to pre-empt outages.

Insightful answers to the question. The best answer is from Swapnil Madhav Chaukar. Well done!

 

Answer from R Rajesh is also a recommended read.

 

My 2 cents - The AI solution needs to be made a process document and like we revise process documents (e.g. - SIPOC, process maps, SOPs etc) after every process change, we need to revise the AI solution after every change!!