Skip to content
View in the app

A better way to browse. Learn more.

Benchmark Six Sigma Forum

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

Who Is Responsible When an AI Recommendation Is Followed — or Ignored?

Featured Replies

Q844

In many processes, AI systems suggest actions, priorities, or decisions — but humans still choose whether to follow them. When outcomes turn out well or badly, responsibility can become unclear: Was it the AI’s recommendation, or the human decision around it? Think of a specific process in your domain where AI provides recommendations but does not act autonomously.

How should responsibility be defined when:

  • The AI’s recommendation is followed, and things go wrong?

  • The AI’s recommendation is ignored, and things go wrong?

What kind of clarity or structure would help avoid confusion, blame, or risk-avoidance behavior?


⚠️ Any answer that is generic or does not connect with a specific process will not be approved.

🏆 The best answer will be selected on the basis of:

  • Relevance of the chosen process

  • Thoughtfulness in defining responsibility boundaries

  • Practicality of the proposed accountability structure

Note for website visitors

Solved by Tabrez Shaikh

The Process: AI-Powered IT Support Triage and Resolution

In modern IT Service Management, AI models (like those integrated into Jira or ServiceNow) analyze incoming tickets to assign a Priority Level (P1–P4) and suggest a Knowledge Base (KB) article for resolution. The Service Desk Analyst must then decide whether to accept the AI classification or override it.

1. Responsibility When the AI is Followed and Fails

If the AI classifies a "Server Down" alert as a P3 (Low Priority) and suggests a routine reboot, and the analyst follows this resulting in a 4hr outage of a critical business application the Service Desk Analyst is 100% responsible.

The Logic: AI lacks "situational awareness." It might not know that today is "Payroll Friday" or that a specific server is the backbone of the company current marketing campaign.

The Failure: Following the AI blindly is an analytical failure. The analysts job is to apply the "Business Context" that the AI doesn't have. If they don't they are simply a data-entry clerk not a support professional.

2. Responsibility When the AI is Ignored and Fails

If the AI flags a series of failed login attempts as a P1 (High-Risk Security Incident) and suggests immediate account lockout but the analyst ignores it because "the user is a VIP" and they don't want to cause an inconvenience, and that account is then used for a data breach the Analyst is responsible for "Gross Negligence."

The Logic: The AI provided an Informed Warning. By ignoring a high confidence security recommendation without a documented technical justification the agent actively bypassed a safety control.

The models or AI are built via human intervention but have intelligent brain and decision making powers.

  • The AI’s recommendation is followed, and things go wrong?

  • The AI’s recommendation is ignored, and things go wrong?

In my opinion for both he above scenarios, there has to be a logic behindz AI means artifical human and should have the same functions as of RACI Matrix- Responsibility., Accountability, Consulted and Informed.

Removing Accontability, i think Ai can be held responsible, can be consulted or informed for further course of action.

AI must not be help accountable as it is not considered a resource with high influence or power. This would further mean that if AI is made accountable, then the day is not far that AI will control humans too.

I am working in a Scientific Research Publishing company. Our reputation and the Brand value is entirely built on the accuracy with which we are producing the scientific articles, this accuracy then results in acquiring the trust of the authors and as well as the audience from the research community who are reading and consuming our content

Today's question is very interesting and for that I would like to explain a scenario we experienced in our organisation.

Scenario: currently we have deployed an AI based solution to check the content correctness and completeness as well as to check the Plagarism and any kind of text fabrication which is been done by the author

These checks are performed using the AI based automation and AI calls that particular manuscript pass or fail. And after that we send it for the actual even best beer review system where they analyse the scientific content, equations and chemical compositions. They perform those relevant steps before publishing the manuscript but the Plagiarism and the content typesetting and corrections are checked by using AI

Recently we were using AI, which is supporting the editors in their work faster and efficient. But in few cases, when we obseved that AI failed to work. The AI failed to capture the plagiarized content and as per the process the manuscript was published. After that few authors or researchers they came back to us mentioning the article they are referring is published by us is having copied data, the research is not unique and it looks like it is the fabricated one.

Our changed view towards the AI usage : Here in this case we can clearly say that AI fail to perform the required task and in a way we can say it has lied to us. AI mistakenly accepted those articles which we shouldn't have a accepted if checkes manually.

Though AI has given incorrect output that doesn't mean AI could be held hundred percent responsible, because we are in scientific publishing domain and we are the get keepers here who are responsible to analyse the data and the content and we are responsible to publish the right content everytime.

Hence, in our case here AI is just an enabler which was helping us and our editor to perform their task effectively. This doesn't mean that whatever answers in produce using AI is 100% accurate. Being the scientific content publisher, we cannot put the entire blame on the AI system.

We can consider AI as our helper or junior editor assistant which is there to help us.

AI if that commits mistake then we should take the responsibility of that mistake, because it's always the journal manager who is there to sign and accept the article at the end of the review. The final review is always lying with the human touch.

To conclude this, we can say, AI can give us evidences but the humans can provide the judgements.

While considering the AI based solution, we can delegate the task to AI but we can not delegate the responsibility to AI.

Domain : Manufacturing : Oils and Gases

 

Context :

In Air separation unit the process is maximum hazardous and very sensitive, always on trigger of process failure(Even in stable condition) even due to small errors and internal noise or external noise, this leads to high risk of safety to the Employee, Environment, Surrounding and assets this is safety concerns due to critical characteristic of products and process, sensitive operations of high pressure and temperatures and other parameters. also the On supply to On site customer is so critical it’s mandatory to be on top and vigilant in managing the process and plant consistently.

 

Intent :

The intent was to build stable Artificial Intelligence Predictive and control Model  to ensure high ‘’Safety and customer service and performance’’ even from start of the process and then maintaining the End to End stable process parameters which leads to better temperature and flow distribution and pressure ratios to attain the desired cryogenic product out put.

HOW and what considerations are made to build the below AI Model :

AI has emerged has assistant, guide and consultant to review the present process conditions been operated based on real online data and analyse that in real time in few seconds and make the suitable decision to the bring the process bias, to reflect the process output the intended output for highly safe and to ensure high Safety and customer service and performance

To build the trust on AI Model by the operator, process operation team we have involved then in the design, considered all technical details, design conditions of the End to End process Lets say from high capacity Air compressors and high capacity Turbines and communication flow to End to End stake holders and taken all suggestions that would call a need for inclusion in the AI Model as Operators and Process Owners being the face of the process and close to reality they know the process very well.

Major methodology Executed.

  • Risk analysis, Brain Storming,

  • Suitable best considerations were made, solutions were identified & Evaluation.

  • The solution was built, tested.

  • Simulations were done with involvement of Shop floor operators and Process Owners

  • Ensured trust building and empowerment to Shop floor operators and Process Owners by involving them in End to End development and till Go-Live Commissioning.

  • Control Measures taken to sustain the implementation and developed Trust

While on implementation we had the question ‘’How should responsibility be defined’’ to ensure the recommendations from AI predictive model is followed and ignored, what necessary actions to be taken and by whom, who is responsible, accountable, Concerned and Informed.

 

AI has emerged has assistant, guide and consultant to review the present process conditions been operated based on real online data and analyse that in real time in few seconds and make the suitable decision to the bring the process bias, to reflect the process output the intended output.

  • But the question remains is if ‘’The AI’s recommendation is followed, and things go wrong?’’ How should the plan of action to be taken & who should be responsible for it, What could be best RACI ?

&

  • The AI’s recommendation is ignored, and things go wrong?

As AI Model doesn’t provide a definitive Fix, it’s not a solution or application which is not affected by any External or Internal Factor and it’s not one time installed or invested and forgotten.

It’s to be tracked and treated as element of evolution and a process of Continuous improvement and evolution really holds good for this to assure system should not fall behind when the AI solution keeps merging with new horizons.

Below Structured approach was deployed :

Tools and System :

Below tools and systems were implemented to find the true structure approach so that team is not lost with lots of information, change and new updates and would help to follow the simplified necessary steps, eliminate Non value addition, avoid confusions, blame, or risk-avoidance behaviour or perform the activities with risk.

System & Standards :

  • RACI Matrix :

RACI Matrix was defined and developed to ensure the structured flow to take the actions, by whom, who is responsible, accountable, Concerned and Informed.

This was evaluated with cross functional heads and team members, communicated end to end on RACI so that all are aware and aligned.

  • SOP : Standard Operating Procedure : SOP revisions with new versions :

    • Updated Step by Step SOP are created for each process lines and considered new operating conditions with AI predictive Model mentioning, below important detailed needs are incorporated.

    • When to take actions when things can go wrong with AI Proposal and AI Proposals are Ignored

    • When to Ignore the AI recommendation ? When to override and take the control in manual ?

    • How to treat, What actions to be taken when the Alarm given by AI predictive system ?

    • What to actions to take When the sudden and unexpected internal or External Noise appeared in the system ensure ?

 

  • Standards :

    • Standards were created

    • Standard Documents creation and approvals as per approval hierarchy like creator, reviewer and validation, approver

    • Review and revisions for new versions with updates so that change is tracked.

Tools & Systems :

  • ONE DOC :

Application/Software to manage the Standards was implemented, with ‘’Definitive Identification Number’’ this has ensured the live availability of right standards documents in the Library to the Process Owners 

 

  • Alert system & Action Log Books & Reporting :

Implemented specific action Log book via Online Share point action log book, in which Operators and Process Owners mention the alarm, deviation from AI system and what actions were taken, remarks for the difficulties faced, also the need of further evolution in the AI program to Nullify the error or Noise,, or to mention the Idea if any ?

 

  • Share point (Online) :

Online Share point action log book, in which Operators and Process Owners mention the alarm, deviation from AI system and what actions were taken, remarks for the difficulties faced, also the need of further evolution in the AI program to Nullify the error or Noise,, or to mention the Idea if any ?

  • MOC : Management of Change

    Management of Change is ensured and given a clarity and facilitated the Live updated Document at any point of time, this has helped to avoid confusions, blame, or risk-avoidance behaviour.

Conclusion :

AI Model provides solutions which is probably close the stable process as needed by the standards but there are high chances also that AI suggestions needs to be thought about and questioned to avoid the failures due to possible wrong suggestions provided by AI by not taking Noise in to considerations or additional necessary prompt in it’s background verification and suggestions.

So End to End thought out Structured process is to be implemented as mentioned above to ensure no confusions, mistake, blame game, to held accountable and responsible, to keep the risk behaviour in control by the AI system also by human.

Domain: Solar Cell & Module Manufacturing Sector

( Capacity: 1.3 GW Solar Cell & 1.6 GW Solar Module Production and Supply per annum).

When it comes to Responsibility of Decision making, it fundamentally becomes a governance problem. In the domain of manufacturing , Process & Quality control or Solar Engineering, AI primarily works as a support for Decision making , not a decision maker. So the responsibility of Success or Failure both lies around the decision authority, not the support system.

Lets start with combination 1: If AI recommendation was followed, but implementation failed.

In this case the responsibility of the failure depends on how the governance followed. An MBB must find out, whether AI was validated for this case or not? Were the input data validated? Is the Risk level sufficient for the automated recommendation usage? Were the human review systems utilized properly?

If all of these governance are followed properly then responsibility definitely belongs to the Process/System ownership, not the operator who runs the process. This is because the operator followed the approved decision framework and it indicates that there is a limitation in the model or risk assessment was not done properly.

In Solar Module Manufacturing, AI can help to conduct DOE or simulations to improve yield of Lamination process. Assume after the analysis AI recommends to increase the temperature to reduce the Bubbles and improve the yield. But after the implementation, the bubbles increases and yield decreases.

Now let us check the vastness of the failure: Before the inspection of the lamination process, at peak capacity utilization, there are minimum 120 Solar modules waiting in process and post process. If the governance are followed , due to human review systems for each implementation only 7 or 14 modules will be rejected. So there will be failure but it does not affect the business in large scale.

But if the governance was not followed properly then the responsibility shifts towards the human decision maker or the individual operator. If we take the example as above, lets assume that during the hypothesis tests AI indicated low confidence level and the operater ignored that and ran the process without any pilot check. There will be minimum 120 modules rejected. Some manufacturer have 8 or 12 laminators like that. The situation would be catastrophic.

Combination 2: If AI recommendation was ignored, but implementation failed.

If the deviation was justified, documented and the followings are the causes for the deviation:

  1. SME identified contextual factors.

  2. safety or compliance are not met.

  3. Input data is not reliable.

then the responsibility of the failure (or we can say proven wrong) remains with the decision authority.

Example: Suppose at Final Testing of modules AI recommends acceptance of marginal module IV curve results, but the quality manager does not allow it after a structured authority discussion, cause it became high risk factor in the last Pre shipment inspection. So he ordered to do frequent calibration of the Sun Simulator and it decreased the capacity utilization. But later it was found that the customer allowed it and there was no need to do so much calibration in the sun-simulator.

But again the catastrophe didn't happen. There were less capacity utilization but the lot didn't fail.

But what happens if the AI recommended to do the calibration whenever the value of the reference module fails the Nelson rules, but the operator ignores that blindly and it leads to rejection of the lot at PSI stage. This is the true example of AI recommendation was ignored without following the governance. Here the responsibility of the failure solely belongs to the operator and result is also catastrophic.

To avoid this confusion, the decision making authority should be structured properly with proper decision responsibility matrix:

a. The AI team should only be responsible for Model development and recommendation.

b. The responsibility of validation and limits of the model lies with the process governance team.

c. Ultimate Operational decisions comes under the authority of the process owner. (In our cases of example: Lamination process owner, Quality manager. Not the lamination operator or the Sun Simulator Operator)

d. Management is the owner of Risk level determination before implementation of any autonomous or AI driven changes.

A fundamental principle need to be followed without any deviation: AI always recommends- But the process owners are the people who decides. AI should never be a authority, always a highly analytical junior expert.

  • Solution

Who Should Be Responsible when AI Recommends?

Let see this through a BPO Case Study in Customer Complaints Escalation.


The Business Process Outsourcing (BPO) sector is one of the areas in which efficiency and accuracy are the most important factors that explain the reason why AI systems are implemented into the operational workflows to provide advice to human agents. Customer complaint escalation is one of the crucial and representative processes. In this case, AI models can process incoming complaints - through sentiment analysis and detecting keywords and historical data - to prescribe the cases that are to be flagged and sent up the hierarchy to senior specialists or the management as urgent concerns. The recommendation is sent to the agent who makes the final click: "Escalate" or "Do Not Escalate." The issue of accountability is complicated and controversial when results - an over-satisfied customer or a harmful churn - are achieved. Not only is clarity philosophical but it is also operational and a legal requirement.

Relevance of the Selected Process.

A high stakes BPO process is customer complaint escalation. Poorly handled grievance may result in a loss of revenue, damage of reputation, and regulatory backlash. Within this domain, AI recommendation engines are conditioned with thousands of previous tickets and can tell how severe the target is. Yet they act upon lines, rather than knowledge. They may also fail to notice a low-key threat issued by a high-value client or over-tag a complaint which is angry but mainstream. The process is a perfect example of the paradigm of AI recommends and human decides, which is why it would be better to refer to it when discussing accountability. The initiative under consideration is the implementation of the name Escalate-AI, a platform in use in one of the client service departments of a BPO to minimize the time spent on resolving the issue and avoiding the further escalation of small ones.

Setting Responsibility Boundaries: Two-scenario Framework.

Scenario A: The Recommendation of AI is adhered to and it goes wrong.

The flag of a complaint that is observed by Escalate-AI as High Priority is followed by an agent. The case is hastened to an overworked manager who is distracted and not given any real crisis which is brewing in another department and the initially flagged customer ends up being not satisfied with procedural and robotic response. Who is responsible?

AI/Developer Responsibility: The accuracy and clarity of its recommendation is the responsibility of the AI. In case the system was trained on biased information (e.g., giving excessive preference to some trigger words in particular demographics) or the logic of the system is a black box, the creators of the system and the organization, which deployed it, will be responsible to provide a flawed tool. Their design results in its very recommendation.

Human Agent/Operator Responsibility: The human agent is left with the responsibility of adopting situated judgment. They possess the background the AI lacks: history of the customer, his tone of voice, and subtlety of the ticket story. They should consider the AI their professional companion, and not their superior. Accepting a recommendation and mindlessly implementing it without applying their human expertise is a lapse of their duty of care.

Conclusion: The responsibility is collective, with the culpability biased. This responsibility to abrogate final judgment is placed on the human agent and the chain of his or her supervisors. The organization is liable to any system flaws in the AI that deceived a reasonable agent.

Scenario B: The AI Recommendation is Disregarded, and Things Do Go Wrong.

Escalate-IA suggests a complaint priority which is Standard Priority. However, the agent notices real distress and tries to amplify it by hand, but is prevented through a workflow rule which demands further reasons to go against the AI. The agent is frustrated but does not give up. The case goes out of hand in social media.

AI/Developer Responsibility: In this case, the possible failure of the AI is in the false assurance. Even a Standard Priority tag can form a hazardous cognitive anchor and induce the agent to second-guess his own sound intuition. Moreover, in the event that the system design is pro-actively creating friction, or punitive actions of exceeding its recommendation, then the process, itself, is guilty.

Human Agent/Operator Responsibility: The responsibility of the agent was to be the voice of the customer. Though they first of all discovered the risk they finally accepted a system they felt was rigid. It is their responsibility not to use the channels they have (e.g., report to a superior) when they assumed that the AI was erroneous.

Process / Owner Responsibility: The third party that emerges in the scenario with criticality is that of process design ownership. They are heavily responsible to the managers that introduced a strict, override-squelching workflow. Their culture of human wisdom as something inferior and systemic risk-aversion superseding it was formed.

Conclusion: The burden of responsibility moves a lot to process owners and managers. Their responsibility is to have a decision-support system that does not promote good human judgement.

Adaptability of the Proposed Accountability Framework: The Three-layer Model.

In order to avoid cross-blame and finger pointing, BPOs are advised to embrace a simple and practical accountability model with regard to the activities like the Escalate-AI.

1.      Responsibility Level (Clear Ownership).

-          AI Providers/Data Scientists: These are in charge of the accuracy and the recall and clarity of the recommendation. They must be capable of providing confident ratings as well as write plain-English justification of each recommendation (e.g., recommend due to phrases X, Y, and such-like combinations in the history which brought about 80 percent of the time).

-          Process Owners / Operations Managers: Accountable as regards to decision making structure. They must devise processes that encourage deliberate overrides, must leave comments too short to counter AI (so you think, not penal), and must get audit deliverables regularly to optimize AI and human training.

-          Duty to Informed, Contextual Judgment: Humans - They are the final decision-makers and must be ready to act as an AI tool and not an assistant or an oracle.

2.      The Digital Paper Trail: The Transparency Layer:

The recommendation of the AI with justification, the action of the agent with justification of the agent (especially of an override)  should be recorded in each ticket. This does not need to be blamed but learn. It offers a decision journal that is auditable.

3.      The Continuity of Calibration (Refinement Level):
They should look at successes and failures of decisions through a cross-functional council (Ops, QA, AI Team, Agents) on a weekly basis. Was the AI wrong? Was this activity by the agent prudent or careless? This loop of feedback will constantly enhance the training programs and AI model itself.

Benefits gained though the initiative:

Using this systematic approach to the Escalate-AI project, our BPO process of customer escalation shifted the disorganized blame into a systematized learning. Once considered a failure measure, override rates turned out to be an important source of knowledge. They disclosed edge cases not found by the AI, which are being improved in the model. Being granted explicit override rules and cognizant that they would be responsible of their decision made agents interact with the AI more considerably. It was not a zero-fault system, but a clear-lined accountability system as a result. The time taken to resolve went down and what was even more important was the rate of fatal complaint mishandling also went down sharply since the hybrid human-AI system had become robust and its points of failure was known and owned.


Finally, even in the BPO sphere and other areas, accountability has to be designed, rather than determined retrospectively once AI makes a suggestion and a human actually makes a choice. It is a collective yet non-uniform burden, in which the clarity of role, openness of process, and a culture of learning systems are forthcoming as opposed to the issue of finding one guilty figure.

  • Author


🏆 Best Answer

Taby Sheikh
This is the most complete and mature response among the human-written submissions. Strong domain anchoring (BPO escalation), clear split between “followed vs ignored” scenarios, and a practical accountability framework (decision rights + audit trail + calibration loop). The response also correctly highlights that accountability must be designed upfront, not debated after failure — which is the core of this question.

Approved


Approved

Bharath CN
Very strong real-world depth in a high-risk manufacturing environment (air separation). The response stands out for process realism: operator involvement, risk analysis, SOP updates, MOC, action logs, and clear override logic (manual takeover during noise / instability). Excellent governance thinking.

Suman Acharjee
Strong manufacturing framing with governance logic done properly. Clear distinction between process owner vs operator accountability, and a good use of “validation + confidence + pilot scale” thinking. Also gives realistic consequences (small batch vs catastrophic scale-up).

Aditya Bhavsar 45
Clear, concise, and correctly framed: AI as assistant/junior editor, humans retain judgement and responsibility. Not as structured as the best answers, but the principle is correct and well communicated.

Vijay Yivaturi

Very strong real-world case with high operational credibility. The answer is anchored in a real process (tax payment processing), clearly shows both failure modes (followed vs ignored), and most importantly explains why the failure occurred (override allowed without revalidation). The impact is quantified and believable. Excellent end-to-end accountability framing across developer + end user + leadership.

iambpawan

Strong ITSM example with crisp, practical logic. The response is clear about what the AI cannot know (business context), and correctly assigns accountability to the analyst when they blindly follow or ignore high-confidence recommendations. The “gross negligence” framing for security override is strong and realistic.

🟡 Conditionally Approved

(Good intent, but does not meet the “specific process” requirement and does not answer both scenarios properly.)
Dhruva Kapur
The answer is largely conceptual and generic. It discusses RACI and AI accountability philosophically, but does not connect to a specific process or provide an actionable structure for preventing blame/confusion.


Not Evaluated (AI Content)

Kush Singh
Preethi Bijesh
Jinad_Padiyath _tPv5
Himanshu_Lohani_WpY8
Ankit Kulkarni
vijay gonsalves


Create an account or sign in to comment

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.