Q834

Many AI initiatives are technically sound — models perform well, agents work as designed, and pilots look promising.
Yet, months later, organizations struggle to point to real, sustained value from these solutions.

Think of a specific AI-enabled process in your domain that failed to meet expectations — or could potentially do so.
What do you believe is the primary reason value was not realized: poor problem selection, weak integration into the process, lack of adoption, governance gaps, or something else?
What would you do differently to avoid this outcome?

⚠️ Any answer that is generic or does not connect with a specific, relevant process will not be approved.

🏆 The best answer will be selected on the basis of:

• Relevance and clarity of the chosen scenario

• Depth of insight into why value was lost

• Practicality of the corrective actions suggested

Note for website visitors—

• This platform hosts two weekly questions, one on Monday and the other on Thursday. 

• All previous questions can be found here: https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/. 

• To participate in the current question, please visit the forum homepage at https://www.benchmarksixsigma.com/forum/.

• The question will be open until Tuesday or Friday at 9:00 AM Indian Standard Time, depending on the launch day. 

• Responses will not be visible until they are reviewed, and only non-plagiarised answers with less than 5-10% plagiarism will be considered for winner selection. 

• If you are unsure about plagiarism, please verify your answer using a plagiarism checker tool such as https://smallseotools.com/plagiarism-checker/ before submitting. 

• All correct answers shall be published, and the top-rated answer will be displayed first. The author will receive an honourable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

• Some people seem to be using AI platforms to find forum answers. This is a risky approach, as AI responses are error-prone because our questions are application-oriented (they are never straightforward). Please take a moment to review this amusing example—https://www.benchmarksixsigma.com/forum/topic/39458-using-ai-to-respond-to-forum-questions/

• We also use an AI content detector at https://app.originality.ai/. Only answers with less than 45-50% AI-generated content will be considered for winner selection.

Not every but significant  AI solutions fail. In many cases it’s the rush to prove technologically updated with the industry instead of focusing on the business requirement or need.

I would like to emphasize on the banking sector, where I have experienced notable issues such as targeting the problem statement, low quality data input and a resistance in adoption to change. Due to conservative nature of work many AI initiatives hit a roadblock as they prompt regulatory and customer/ stakeholder issues which result in impacting the yield coming out of the AI solution to the business problem statement

Most cases there’s a heavy push from the top management without a convincing use case which cause issues for the project team. The project team gets into a loop of approvals, data cleaning, cross selling, user access, etc.

Following are few reasons why AI solutions fail wrt banking sector

1.      Data quality, access, and conservative approach

When a retrospective session is held it traces back to unadministered, poor quality data based upon which the models are developed which result in instability and biasedness.

2.      Lack of E2E workflows integration

In many cases a successful proof of concept is the extent to which an AI solution project reaches smoothly. However, the risk & control, routes and process engineering around the decision is neglected. The result is a low confident prototype that is overshadowed by manual interventions.

3.      Inefficient change management

Business stakeholders, analyst, relationship manager are not upbeat with the technology. There is also a resistance due to fear of job loss. Because the concept is not widely shared and lack of training the models remain underutilized. A local champion, bonus/incentive and the confidence to even fail is missing .

4.      Regulatory, Compliance and Risk

There are unexplainable documentation and no monitoring, in such case audits fail and cause risk. In the banking sector delayed decision making , approvals, frequent roll backs are critical for banking department such as AML, frauds, credit & collection , etc

5.      Undervalued operational cost

Banks need to be cautious over frequent trainings, new features, data cleansing. There needs to be a standard and discipline that needs to be followed over the time as training get stale, models deteriorate, cost increase resulting in lesser business benefit, so the projects die a slow death

Now the question arises, how the banks can avoid these issues. Following are some of the pointers:

1.      Attention towards sustainable value generation

Develop every use case with specific problem statement , quantifiable metrices and impacts. Operational cost on governance, discipline and training  need to be compared with the cost of manual intervention , monitoring, fraud loss and other inefficient metrices.

2.      Early investment in governance, MPOps and Data cleansing

To keep the model consistent in production, data foundation and high risk use cases need to be robustly build. Frequent monitoring and drift detection keep the model up to date.

3.      Design efficient Change strategy

Collaborate with the stakeholders, operations, relationship managers and other staff to explain the usage of tools in daily operations. Use AI not to replace humans but to engage AI for low impact decisions , exception highlighting, prioritization .

 

The above, translate directly into value‑stream‑level design, measurable benefit tracking, and strong change management around AI‑enabled processes in banking environments.

Domain :- Aerospace Subcontract Precision Machining shop

The particular AI task that has failed: The use of AI in the predictive management of tool life in our 5-axis milling cells (introduced mid-2024)

The plan sounded awesome: sensors on spindles + data from spindle vibration and/or forces loaded into an AI that could predict when the inserts were going to fail and/or chatter, and then exchange the tools at just-in-time cycles, as opposed to at predetermined intervals. AI was trained with our tool life monitoring raw data.

We did pilot run on 2 machines, it was grand success. Then, on two machines, we were able to cut tool use by 18%, unplanned stops by 60%. Surface finish / roughness problems were down. We rolled it out to the all high machine hour rate 28 machines.

Six months later. The value had disappeared.

Tool costs crept back up, unplanned stops re-emerged (different machines), and we actually rejecting  a greater number of parts out of aggressive feeds the AI was optimizing for.

Main reason for the failure was the inability to integrate the process + the failure to embrace the value by the people who matter most, the machinists and machine setters.

The AI system was technically good (accuracy level around 88%), but:

•            It produced alert messages on a screen that nobody monitored during the course of the shift — the operators were too busy watching the machine cut physically to control feeds.

•            As soon as it called “change insert in 18 minutes,” the setter usually didn't comply with it because “the tool still looks fine” or “I don't want to stop mid-pocket and get a witness mark.” "Tribal knowledge" always overturn "AI" every time.

•            No feedback loop — if the operators chose to override the AI, the reason for the override was not recorded, so the system never learned from the experts.

•            "We never altered the bonus payments or daily objectives – to stop and alter tools 'early' meant he'd fail to reach his daily number of pieces, and you guessed it right what he would do."

“The AI was dealing with a theoretical problem (problem of tool life) whereas the actual problem for the team was, ‘hit the numbers on the board today’ and every day.”

What I Would Do Different Next Time And What We’re Currently Doing With the v2 Launch

1.           Include the shop floor team from day one. ‘Machinists and setters are in the room when we define success. Their key performance indicator (safe, on-time parts with no drama) becomes the artificial intelligence system's top goal not just tool cost.’

2.           Engage in their workflow, not in parallel. Alerts are now giant flashing lights & voice notification on the machine HMI and tool change notice is a one-button acknowledge. No need for a separate dashboard.

3.           Make adoption painless and rewarding

o            The daily target now includes “AI-guided tool changes completed”.

o            The “Override Button”: This will force a 5 - second voice note or reason into the data, and this data will go straight back for model retraining. Operator need not have to move away from the machine to feed this data.

o            Bonus pool is monthly and dependent on yields of entire cells rather than individual piece count.

4. Start small, proving value in their terms, and then scale We scaled back to 4 machines, operated for 3 months with new rules, then proved to operators that they can go home earlier on Fridays with higher-quality products. Only then we will scale to all machines.

5.           Kill it quick, if it doesn't stick. Have a tough gate review, if adoption <80% after 60 days or yield isn't improving. We will pulling the plug and call it a day, no more wasting of efforts needed.

Bottom line from the tool crib department, The issue was not the AI. It was us, The once who built the clever tool and stuck it in a vacuum and wanted people to dance around it.

“Real value happens when the AI is incorporated into what people are already doing and make there life easy, not adding too many extra things to there already existing to do list.”

"We are finally seeing the positive results of the v2 cells, specifically the reduction of tool cost by 14% and unplanned stops by 45%, and most importantly, the operators now are able to defend the system against management’s inquiries regarding the cost."

"Lesson learned," Harris said. “If the individuals operating the machine do not love it, no model accuracy in the world can save you”.

🏆 Best Answer: Adil Khan
Clear, concrete failed use case with strong insight into why value was lost (adoption, incentives, workflow fit) and what changed to recover value. Highly practical.

✅ Approved: Sandeep Saha
Strong domain relevance and sound reasoning on data, governance, and change management.

Gap: Lacks a single, specific AI process failure with measurable outcomes; remains largely conceptual.