Q 817.

In real-life operations, data is rarely perfect — records are missing, inputs are inconsistent, and information arrives late.

Yet AI systems are often expected to respond confidently, even when they’re working with uncertain or partial data.

 

Think of a process in your domain where AI depends on multiple data sources or user inputs.

How should the AI behave when some data is missing or unreliable — should it estimate, ask for clarification, delay the decision, or escalate?

What safeguards or logic would you build to ensure it still makes responsible and useful decisions?

 

⚠️ Note: Any answer that is generic or does not connect with a specific, relevant process will not be approved.

 

🏆 The best answer will be selected on the basis of:

 

• Relevance and realism of the chosen process scenario

• Clarity in defining how AI manages uncertainty

• Practicality and balance between accuracy and responsiveness

 

Note for website visitors -

• This platform hosts two weekly questions, one on Monday and the other on Thursday. 
• All previous questions can be found here: https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/. 
• To participate in the current question, please visit the forum homepage at https://www.benchmarksixsigma.com/forum/.
• The question will be open until Tuesday or Friday at 9:30 AM Indian Standard Time, depending on the launch day. 
• Responses will not be visible until they are reviewed, and only non-plagiarised answers with less than 5-10% plagiarism will be considered for winner selection. 
• If you are unsure about plagiarism, please check your answer using a plagiarism checker tool such as https://smallseotools.com/plagiarism-checker/ before submitting. 
• All correct answers shall be published, and the top-rated answer will be displayed first. The author will receive an honourable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

• Some people seem to be using AI platforms to find forum answers. This is a risky approach as AI responses are error-prone because our questions are application-oriented (they are never straightforward). Have a look at this funny example - https://www.benchmarksixsigma.com/forum/topic/39458-using-ai-to-respond-to-forum-questions/

• We also use an AI content detector at https://quillbot.com/ai-content-detector. Only answers with less than 45-50% AI-generated content will be considered for winner selection.

AI Powered Investment Portfolio Management

Wealth management firms across the globe use different AI-based portfolio advisory system that integrates and synchronizes with multiple data sources  like live market feeds, macroeconomic indicators, client risk profiles, social sentiment data, and geopolitical signals — to dynamically rebalance client portfolios.

 

100% correct data is a myth:

·      Market sentiment shift predictions swings either which ways after unexpected or expected RBI announcements.

·      Social sentiment analysis returns incomplete / biased result.

·      Inflation updates lags behind real-world trends.

The true challenge is that an AI model must decide whether to rebalance client portfolios immediately, estimate the impact of missing variables, or delay recommendations for more information to emerge — this could be resulting in risking either opportunity loss or investor exposure.

 

How does Uncertain / Missing Market Data is handled

1.    Using statistics estimate using market-correlated Data:

When a data feed like currency index is not available or missing, an AI substitutes estimates using other correlated variables like commodity prices or VIX - Volatility Index via Bayesian method.

Each substitution is has a confidence score which is reviewed manually and has a traceable metric, ensuring transparency for auditors and compliance teams.

2.    Relying upon confidence score based rules:

If model confidence >95%, AI proceeds is programmed to continue with automated minor portfolio rebalancing (e.g., shifting 5% from equities to government security bond or gold ETFs).

Between 75–95%, AI flags it for financial analyst review.

Below 75%, the system delays rebalancing and triggers a “market alert” escalation to the risk desk.

 

3.    Flag for Human Review:

In ambiguous cases like conflicting social vs. technical data, the AI model requests validation from the portfolio manager / financial analyst.

 

4.    Delay and Contextual Re-assessment:

The AI temporarily freezes automated actions while running scenario simulations using historical analogs like similar price behavior during past inflationary cycles.

This prevents knee-jerk decisions that could amplify volatility during flash crashes or liquidity shocks.

 

Safeguards for Responsible AI Decision-Making:

 

1.    Dynamic confidence scoring: All model outputs include uncertainty flags visible to traders.

 

2.    Human-in-the-loop compliance: No high-value trades occur without validation when data completeness <90%.

 

3.    Continuous learning loop: After data validation or market closure, AI adjusts weights to improve resilience to incomplete data next time.

 

Impact and Market Responsiveness

In a market data that is fragmented or volatile, the responsible AI system behaves prudently but adaptively considering:

It only acts decisively when data confidence is solid i.e. confidence >95%.

It requests human review when uncertainty is high i.e. confidence 75% - 95%.

It self-adjusts model weights to learn from new market patterns if confidence is <75%.

 

Conclusion

In the financial sector, an AI managing portfolios under market turbulence must balance speed with accountability. The right combination of probabilistic estimation, confidence score based escalation, and human verification prevents reckless trading saving investors from potential unprecedented loss while preserving agility — ensuring resilient, ethical, and market-aware AI decision-making in an unpredictable economy.

 

 

Domain: Massive Multiplayer Online Role-Playing Game (MMORPG) Development / Live-Ops and Gameplay Quality

How Should AI Handle Uncertain or Incomplete Data?

In online game operations, data is never perfect.
Crash logs may go missing or never be reported by players. Logs from iOS and Android are sometimes unclear version details (10, 12, 21, etc.) may be missing, and regional network reports (EU / US servers) often conflict or the issue appears only on one server.

Yet, AI systems are still expected to monitor gameplay, detect lag and protect the player experience in real time.

At the same time, AI also supports longer-term gameplay balancing — tracking champion usage, win rates and game modes (like Dungeons or PvP Arenas) that players avoid.
But this data is often incomplete or takes months to mature.

For example, a “Legendary” champion showing only 1% global usage might indicate a deeper design issue — but AI cannot instantly know whether it’s due to poor skill synergy, long skill cooldowns, weak damage multipliers or simply player preference.

This is where AI must clearly separate immediate technical reactions from long-term design evaluation.

---

The Process: Live Event and Champion Monitoring

During a live PvP event, AI detects lag or long matchmaking queues when players are ready and waiting to fight — especially during limited-time windows (e.g., 14:00–16:00).
Meanwhile, over several months, AI observes that a few Legendary champions maintain <1% global pick rate.

Two very different type of actions one demands instant technical action, the other requires patient data analysis & validated by human.

---

How AI Should Handle It

1. Act Immediately on Minor Technical Issues even with broken / partial data

When lag or performance drops are confirmed, AI can safely apply reversible fixes such as:

• Dynamically reducing graphics load during lag (e.g. from Ultra/High to Low).
• Adjusting frame-rate limits (e.g. from 60 FPS down to 30 FPS).
• Shifting players to alternate regional servers to reduce matchmaking delay during time-bound PvP events.
• Performing server load balancing or rerouting traffic between servers within the region or cloud regions when waiting time crosses a threshold.
• Censoring offensive words in chat automatically (F***, S****d) when AI is uncertain but suspects potentially inappropriate language.
• Restarting a regional server when it stops responding.
• Re-syncing user progress when save-data upload fails.

These actions are real-time, safe and reversible they restore gameplay smoothly without waiting for human input.

---

2. Observe, Don’t React on Long-Term Trends

Champions balancing (Buffs/Nerfs) must never be adjusted automatically by AI.
AI should continue gathering trend data across events, servers (EU/US servers) and player tiers (New / Mig Game / Eng Game) before flagging persistent underuse or overpowering for manual review.
Example insight:

“Legendary champions <1% usage globally over 3 Months.”

Automatically nerfing or buffing based on partial data risks major backlash from players.
Some players (“Whales”) may spend thousands of euros monthly and invest heavily in a Legendary champion and get them through RNG shard systems, so sudden AI-driven uncontrolled changes to champions could cause frustration and loss of trust.

---

3. Escalate for Human Decision-Making

When trends are identified, AI should summarize findings and escalate to design or QA teams for testing and controlled rebalancing.
If a champion becomes overpowered or bugged (e.g., one-shot kill any boss) after champion re-balancing, AI must flag and communicate it immediately in-game so players are informed that the issue will be fixed soon avoiding players wasted spending ( Real Money ) on bugged content. Escalate it to developer as AI can monitor such data in live server. It can check all type of reactions like particular champions skill interactions (Buffs / DE Buffs)

Cheating and Bot Mitigation (Pattern-Based)

For repeatable abnormal patterns — such as impossible damage numbers or automated click intervals — AI can act instantly to:

• Flag suspicious accounts.
• Freeze rewards or invalidate match results pending audit.

This ensures fairness without affecting normal players.

AI can also recommend merging low-population servers to reduce matchmaking delays in future live PvP events.

 

Complete and right data can produce wonder using AI system, however in real work getting the right and complete data is a challenge and hence it is impacting all the processes. 
Let us take insurance process as an example and let us see how uncertain or incomplete data impact the entire process and how AI can help in handling the same.
1.Data Validation and Uncertainties detection
a. Identify the missing data or inconsistent data
b. Flag any anomalies of data
c. Assign confidence score to inputs 
2.Probabilistic or Bayesian Modelling 
When there is a missing data Ai should not consider that as Zero rather AI should use Monte-carlo simulation or Probabilistic or Bayesian Modelling to estimate likely value

3.Imputation Strategies: If there is any gap in the data then
a. Apply Statistical calculations ( Mean, median, model based calculations)
b. Always track the source of the data to track the imputed values
4.Human -in loop of decision making
For high impact or ambiguous cases 
a. Route those cases to have human validation by checking and validating the evidences
b. Present uncertainty score or confidence score to help human make proper decisions
5.Transparent Output- AI output should carry 
a. Confidence Score like ( X% chance this claim is a “Valid” claim)
b. Also should provide the reasons if it is giving Low Confidence score to enable Human Loop Decision 
6. Continuous Feedback loop
a. Whenever new data set comes model should be able to adjust the same
b. Use feedback from underwriters or claim adjusters to improve imputation and uncertainty handling
c. Build data pipelines to automatically update uncertainty metrics as data quality improves

In publishing, especially within editorial processes and content recommendation engines, AI takes inputs from author details, reader behaviour metrics, tagging information, citation databases, and current trends. But in practice, these data sources are often not synchronized or lack the latest updates. A manuscript might have missing keywords, reader feedback might be delayed, or citation data could be outdated. Therefore, the question is, how should AI respond when faced with these discrepancies?

 

First, confidence without clarity is dangerous. The AI needs to signal uncertainty upfront when recommending articles or flagging content for review, but lacks key metadata. That will mean tagging the output as “based on partial data” and prompting the user to share any additional or missing information.

 

Second, the estimation should be acceptable, but only when the stakes are low and the assumptions are clearly communicated. For example, publishing a medical research article on COVID-19 carries far more risk than approving a mathematics book’s manuscript. In higher-stakes decisions, the AI should delay or escalate. It is better to flag the issue to a human editor than to perform a high-risk step incorrectly. Escalation should follow clear, predefined rules. For instance, if essential details like conflict-of-interest statements or author credentials are missing, the system must stop and inform the appropriate team right away.

 

Lastly, to promote responsible behaviour, the AI must have built-in safeguards. These include

1) confidence level thresholds

2) fallback protocols

3) audit trails

 

When data is missing and the AI system's confidence level drops below the set threshold, it should either pause and ask for additional information or the missing information. In such cases, it might rely on historical patterns or standard templates as a backup, but only if it clearly flags those assumptions. For every decision made under uncertain conditions, maintaining a detailed record of the context in the form of an audit trail is essential so that human reviewers can understand the reasoning and intervene whenever needed.

🏆 Winner – Shashi Prakash for his Investment Portfolio AI — a flawless design of confidence-based, human-supervised decision-making under missing market data.
🥈 Runner-up – Adil Khan for his Gaming Live-Ops case — great distinction between reversible AI fixes and human-led balance updates.
🥉 Special Mention – Indrani Ghosh Dastidar for a structured uncertainty-handling framework in Insurance.
✅ Also approved – Manik Sood for responsible escalation logic in Publishing.