Leaderboard

Process Context My team also manages the central master data management for 50+ plants today, and this will grow to 70+ plants by 2027. The entire fleet data management is handled by two people in my team. Every time we commission a new plant or acquire a new plant, we need to align it’s material master with our fleet database, to avoid duplication, planning errors, and wrong spares being introduced into SAP. In a typical post commissioning and acquisition, we review minimum of 5000+ incoming material records against an existing 345000 item fleet master. Practically for my team, each item review takes at least 6 minutes without AI. The AI-enabled Process To solve this, I built a Python + AI solution using a MiniLM semantic model, combined with rule based checks. The program setup classifies each incoming item into three categories, Auto, high confidence match to directly map & upload in SAP. Review, ambiguous match, reviewed by the master data team. Reject, no valid match, program generates a new master data creation template for my team, to directly load into SAP. You can clearly see, AI does not create master data blindly in this case, it recommends, and the team decides. When We Trust The AI I have defined clear rules after testing the model for almost 10 days with millions of lines, semantic similarity is high & critical identifiers (model number, size, rating). It checks if descriptions and attributes are complete and consistent. One more rule I have setup is to keep standard, low-risk categories, and excluding verified MRP items, and these items directly flow straight into Auto category & are uploaded without manual touch. When We Override The AI Team deliberately does the review when similarity scores are close across multiple candidates, technical digits conflict even if text similarity is high. Then we also look at if item is maintenance critical or safety critical. We jump to the poor descriptions as well. In all such cases, team’s priority is correctness, not the speed. Safeguards That Keep The Balance We have built simple controls to avoid blind trust or even excessive overrides, Strict thresholds for Auto classification, mandatory team’s review for all Review cases, spot audits of Auto mappings, tracking & analysis of override patterns to improve program, and we have clear ownership, AI suggests, Team decides. Impact In Real Numbers Now with this program, my team completes 5000 item migration in 10 days in total instead of 2 months. I have a clear breakdown of 10 days, Data setup + AI pre-load + first analysis is done in 0.5 day SAP mapping for Auto category takes 1 day Manual review is done for Review category in 7 days New MD setup for Reject category is done in 1.5 days This has really improved my team’s output and bandwidth, and also reduced the onboarding risk for new plants, and best part is, it is allowing two people to scale this work for our growing fleet. Bottom Line I trust AI where signals are strong & mistakes are low impact, I override it where ambiguity or risk is high. As you can see, we are improving the overall process, idea isn’t to remove people from the process, it’s to make sure people spend time only where judgement actually matters.

Case Study: Smartphone Discount Optimization - Amazon Great Indian Festival 2023 vs. 2025 Executive Summary: During Amazon's Great Indian Festival (AGIF), smartphones represent the highest-revenue category, contributing approximately 35-40% of total electronics sales. This case study examines how AI recommendations guided discount strategies, where they succeeded, where human override was critical, and the resulting business impact. Amazon faces a multi-dimensional optimization problem during AGIF smartphone sales such as (1). Brand partnerships - Negotiated discount caps with OEMs (Samsung, Apple, OnePlus, Xiaomi), (2). Margin protection - Maintaining profitability despite aggressive discounts, (3). Inventory management - Balancing stock across fulfillment centers, (4). Competitive pressure - Matching/beating Flipkart's Big Billion Days pricing, (5). Customer acquisition - Converting first-time smartphone buyers, (6). Same-Day Delivery promise - Ensuring SDD fulfillment at scale. AI decisions and suggestions were trusted for Hourly Price adjustments based on Real-time data processing, inventory positioning based on Pattern detection at scale, and customer segmentation by studying hidden behavior discovery. Human decisions (Override AI) are implemented while deciding the Brand-specific strategy – considering the Relationship & negotiation factors, Margin protection through multi-stakeholder trade-offs, Discount timing psychology is override when studying the behavioral economics, statistical interpretation during sample size & context awareness, and Competitive response - Real-time market intelligence Quantified Impact Summary Value Created by Trusting AI: AI-Driven Decision Revenue Impact Margin Impact Dynamic hourly pricing +$89M +$7.1M Inventory optimization +$42M +$3.8M Personalized discounts +$67M +$5.4M SDD routing efficiency +$12M +$4.2M Total AI Trust Value +$210M +$20.5M Value Created by Human Override Human Override Decision Revenue Impact Margin Impact Apple strategic discount +$107M +$3.2M Xiaomi margin protection -$38M +$4.25M Flagship discount timing +$61M +$2.8M Total Override Value +$130M +$10.25M Combine Optimization – Total Incremental Values (2023 à 2025) Incremental Values Revenue in millions Margin in millions AI-Trusted Decisions $210 $20.5 Human Override Decisions $130 $10.25 Total - Combined Impact $340 $30.75 Attribution Analysis: Revenue Growth Attributed to Framework: 95% of USD 356 million total growth Margin Improvement: 0.9 percentage points (8.2% to 9.1%) AI Contribution: 62% of total value created Human Override Contribution: 38% of total value created Key Learnings: When to Trust vs. Override Trust AI When: 1. Processing millions of transactions in real-time 2. Detecting patterns across pin codes 3. Optimizing within defined constraints 4. Speed of decision exceeds human capacity 5. Historical patterns are strong predictors 6. Objective function is clearly defined Override AI When: 1. Strategic relationships are at stake 2. Competitive dynamics are fluid/real-time 3. Behavioral psychology factors apply 4. Statistical conclusions contradict business materiality 5. Multi-stakeholder trade-offs exist 6. Novel situations outside training data 7. Qualitative factors can't be quantified MBB Reliability Framework: 1. AI systems can be used for data collections by studying transaction logs, clickstream, inventory. 2. For pattern detection AI models can be used for customer segments and price elasticity. 3.It is the joint responsibility of an AI and MBB for statistical validations (T-tests, regression, significance testing). 4. At the stage of Contextual Interpretation MBB holds the major responsibility. 5. During Strategic Integration, both MBB and leadership holds a major responsibility (market share vs. margin trade-off). 6.MBB's post validating AI outputs against business reality can finalize the Reliability Assurance Conclusion: In smartphone discount optimization during AGIF 2025, AI recommendations delivered $210M in incremental value through pattern detection and real-time optimization. However, human override decisions contributed an additional $130M by incorporating strategic relationships, competitive intelligence, behavioral psychology, and contextual interpretation of statistical results. The optimal approach is neither blind trust nor reflexive override, but a structured framework that assigns decisions to the entity—human or AI—best equipped to handle them.

It is very critical and important to understand that Al doesn't think the way a human does. Al is like an automatic machine which is programmed, and its objective is to check thousands of files or activities faster and then provide the output the way it is programmed making it easier and convenient and enhancing the capability and efficiency quickly. Al is basically cross checking its own memory and functions in a way that are based on pre-defined rules and the way it is trained and programmed. This activity adds value, however if the rules are mis-configured or updated incorrectly it can have a severe damage on the business and company's reputation hence Al should be used to gain efficiency, but it should not replace human judgement. Real Scenario in my office - I would want to highlight one scenario specific for a chat process, Due to state regulatory compliance specific for one of the states in US ex. XYZ state, The medical information of the patient was not supposed to be provided due to HIPPA compliance. The BOT was programmed in such a way that it should route those chats to a service representative and in return the service rep. will assist by providing the information only if a signed authorization letter was available from the patient to disclose the medical records. A new update was received and the BOT was supposed to be programmed accordingly for a different state ex. ABC state, however the same rule was applied and programmed to process XYZ erroneously due to which the BOT started providing medical information of the patient which was incorrect and non-compliant further prone to fines and penalties along with a possible litigation when the patient finds that their medical information is provided without their consent. As part of the routine auditing work, QA picked up some random samples and was able to see the variance and highlighted it post which the concerned team was informed about the issue and the BOT was reprogrammed and overwritten as earlier Al recommendations and outputs need to be trusted where there are low risk scenarios and rules are clear; outcomes are predictable and no ambiguities and this should match with historical patterns. We need to be very mindful in maintaining the right balance between trusting the Al outcomes and have a understanding on when to intervene and this would require good judgement and strong risk governance. In the above scenario, the high-risk actions should require human review, and overrides should be monitored continuously to help improve the system and the output provided to be accurate. Al needs to operate as a help in making decision support and not a decision authority Cues, safeguards or thumb rules needed to maintain the right balance 1) The level of trust needs to be aligned with tasks that are simple and can easily rely on automation 2) As mentioned above on the scenario pertaining to state regulatory compliance. The tasks should be routed for a human review considering the severity and impact of these errors 3) Regular spot checks, targeted audits, sample checks need to be done to check if the rules are operating correctly as programmed As a thumb rule, it should be followed that the simple, repetitive tasks with clearly defined rules and lesser error impact can be automated using Al. However, errors pertaining to high impact and business reputational harm must remain under human control. This way Al remains an important support mechanism and tool without compromising compliance, trust or responsibility.

Domain : Manufacturing-Oils and Gases Context : In Air separation unit High Capacity compressor and High Capacity turbine are very sensitive equipment which operates at very high pressure and high speed, temperatures and expansion ratios, here the key is how this high precision process to be operated stable absorbing all the noises external and with in process noise and deliver a consistent parameters output values so that requirement of process is met, unless these this condition is met the process control always unpredictable and with quality variations. Intent : The intent was to build a Artificial Intelligence Predictive and control Model even from start of the process and then maintaining the End to End stable process parameters which leads to better temperature and flow distribution and pressure ratios to attain the desired cryogenic product out put. It was clear decision and direction of business to deploy a BB Project to achieve the first level of consistent Sigma value of grater than or Equal to 3.0 and later to review on improving further. HOW and what considerations are made to build the below AI Model : AI has emerged has assistant, guide and consultant to review the present process conditions been operated based on real online data and analyse that in real time in few seconds and make the suitable decision to the bring the process bias, to reflect the process output the intended output. To build the trust on AI Model by the operator, process operation team we have involved then in the design, considered all technical details, design conditions of the Air compressor and Turbine and communication flow to End to End stake holders and taken all suggestions that would call a need for inclusion in the AI Model as Operators and Process Owners being the face of the process and close to reality they know the process very well. When Should People ''Trust'' an AI’s Recommendation : I will give a real example : The operators and shop floor people are the owners of the process, they believe only when the AI model delivers a result which is as expected by them for their process objectives and centre line management KPI given for each parameters. First Confidence with Simulation : So considering this point we have engaged the shop-floor team and operators in design and development to assure that how its build to take them in trust first, with their suggestions and then the same Operators and process owners were participated in simulation of the AI model, They run by themselves seen the output and given the feedback for corrections and fine-tune, many times they accepted the results also. the expected results in the same. ''First Trust'' after 2nd-Simulation & Dry Run : The logics and model were corrected based on End to End review and asked the Operators and process owners re-perform the simulation once again in presence of technical experts, The results to operators and process owners found to be favourable. ''Evident proof of trust'' on AI Model during Commissioning and Go-Live & Post Go-Live : The plant, Air compressor and Multiple turbines were started and taken in line with AI Model predictive & Control system, Found to be favourable with expected desirable result and stable process parameters with in control limits, observed the simulation nodes and possible deviation would lead to failure and the bais, taken control to verify the observed deviation optimized further for the consistent stability of parameters. Positive Business outcome : For Air Compressor : Achieved the Process capability of 1.25 and with a sigma Value of 3.75 For Turbine : Achieved Process capability of 1.42 and with a sigma Value of 4.26. Customer satisfaction due to improved product quality, due to consistency in Online supply to customer. Savings in Power consumption Savings idle running of equipment and deterioration. Realization of product soon after few hours of Factory start up. Measures taken to sustain the developed Trust : Regular Audits & AI performance review Documentation until complete sustenance System alarm tracking/decision lists of shifts MOC, Management of Change method implementation Clear accountability (list of monitors, approver, in charges, especially on Logical changes) Review for bias, fairness, and Legal & regulatory compliance Maintain deviation logbooks for each parameter, failures against the target values AI Model performance review, correction, maintenance and development : Review the performance of the AI system on continuous basis with team and adapt the necessary change to be brought in to the model new dynamics. Take all the learnings from the feedback given by the alarms of system and people. Training and communication Ensure each and every change in the model is communicated to the end to end team members and provide full necessary training on the actions to take. Example, if AI Model is to be over rided by Manual control or visa versa from manual control to AI Model, Ensure to provide hands on experience. 2nd part of the Question : When Should the operator and process Owners should Override AI Model. AI Model doesn’t provide a Permanent Fix, it’s not a solution or application which is not affected by any External or Internal Factor, so considering this we have designed ‘’alarm’’ system which AI predicts the deviation which is out of it’s control and informs the user to take necessary action either online with AI or by taking AI in OFF Mode and override manually to control the process until the External or Internal factors are nullified and then take back the AI Model in line once the noise factor really disappeared. One of the example of External Noise factor for which AI Model has to be taken off and Manual Human Override has to happen. AI Model can’t accommodate the dynamic process variation of downstream or upstream caused due to sudden Pressure fluctuation by control valve malfunction, power frequency variation. This surge in Power frequency or Power factor leads sudden dip in Compressor pressure, RPM and Turbine RPM and expansion ratio. During the above situation if the process is not taken in control sure it leads to destable the whole plant , plant will shutdown different out of control settings, so It’s necessary to take the process out of AI Predictive control and operate manually to bring the process under control and then after manual stabilisation give back the control to AI Model. Conclusion : AI has emerged has assistant, guide and consultant to review the present process conditions been operated based on real online data and analyse that in real time in few seconds and make the suitable decision to the bring the process bias, to reflect the process output the intended output. But AI Model doesn’t provide a Permanent Fix, it’s not a solution or application which is not affected by any External or Internal Factor and it’s not one time installed or invested and forgotten, it’s to be tracked and treated as element of evolution and a process of Continuous improvement and evolution really holds good for this to assure system should not fall behind when the AI solution keeps merging with new horizons. As additional details : How above AI enabled Model was developed and the major steps followed as below : Risk & Bias : The whole process was studied for Severity of failure Air compressor and Multiple Turbines, occurrence and detection, especially on detection failure at each step of the process and instruments while incorporating the logics for predictive model, bias limits were identified, precision modelling were used to arrive at the accuracy of the flow meters and control valves, power and power factor tuning for ramp up and ramp down. Model definition : AI predictive Model was developed by considering historical manual operations data, Risk thresholds, keeping continuous sustainable focus with Model metrics, logical validation, establishing stability and calibration of all instruments and analysers. Simulation & Dry Run : The developed model was tested on dry run simulation for Air compressor and Multiple Turbines to see the model is a best fit and need of correction and fine tuning for desired output, simulation run was executed to identify all the predictive proposal based on assumption for the defined out put of each process parameters, power settings, pressure out outs, temperature, speed of turbines, inlet and out let pressures, temperatures, Valves, pressure relief valves, sensors with the reduced bias. Process conditions of Air compressor and multistage turbines and instrumentation output was measured and noted, temperature and pressure requirements at each process/instrument outputs, logics were corrected, replaced, fine tuned for desired output. Commissioning and Go-Live & Intended sustenance overtime : The plant was taken in line with AI Model predictive & Control system, observed the simulation nodes and possible deviation would lead to failure and the bais, taken control to verify the observed deviation optimized further for the consistent stability of parameters of Air compressor and Multiple Turbines and achieved the Process capability of 1.25 and with a sigma Value of 3.75 and for turbine parameters achieved Process capability of 1.42 and with a sigma Value of 4.26.

Narratives + Berkie AI MarketPlace ( Internal AI) Berkie pulls data where possible and drafts/reshapes sections (Property Overview, Market Overview, Borrower & Sponsor overview, Maps & Aerials, Strengths & Weakness) Analyst will have the final say on what to keep, change or delete before anything goes to Mortgage Bankers and Underwriters Here AI is clearly recommending content and structure, not deciding. The risk is when: Some analysts might over trust (pass AI text through minimal checks) Others might ignore the AI ( rewrite everything) and lose the time savings we are aiming for. Below is how we defined when to accept V/S when to override and the practical rules of thumb to keep the balance. AI assisted Narrative drafting For clarity, here is exactly what AI is recommending: Data interpretation and summarisation: From property details, sales/loan data, crime reports, comps, market stats, sponsor info AI proposes : A property overview, Market Overview, Sponsor description, and Initial strengths and weaknesses wording( based on the inputs shared by analysts) Language polishing and standardisation: AI rewrites analyst's bullet points into Narrative form, enforces US english, clarity, tone, and structure. Section organisation & emphasis: AI may choose what to lead with, how much weight to give to crime v/s amenities, etc , based on the prompt. The AI recommendation is: "Given the provided inputs and prompts, this is the Narrative you can send to Mortgage Banker" The Human decision should be: " Is this narrative accurate, appropriate, and aligned with banker expectations for this specific deal?" Decide when to accept and when to override We trained analysts to make the accept/override decisions in three simple phases. Facts & input checks: " Is everything that looks factual, actually true ?" Default Stance: NEVER blindly accept AI's factual statments. Verify first. Accept AI's wording when, every fact and statement is directly traceable to Berkie data, Analyst provided notes, Attached reports(crime, market study, rentroll, etc). AI is merely re-phrasing things you already confirmed.(e.g summarising CRIME report which analyst already read). Override or heavily edit when: 1. AI infers anything beyond the input e.g a) "The sponsor is highly experienced".. when the input only says "2 prior deals" b)"The area is considered safe".. based on limited or ambiguous crime info. c)"The market is booming".. without strong support in the market data. Numbers are rounded/changed or additional figures appear which are not in reports AI uses vague, unanchored claims like strong demographics, high demand etc without references. Rule of thumb: If we cannot pin point the exact sources, we do not accept as-is. Either adjust to match what we can back up with source or remove it. Emphasis & Framing: "Is the story this drafts tells is right for this deal?" Here is the AI is recommending about how the deal is positioned, not just the facts. We accept AI's framing when a) The draft matches analysts judgement of the deals story. For a strong deal, it should lead with genuine strengths and mentioned balanced risks. For a challenging deal, it should acknowledge core risks clearly but professionally. b) The ordering feels right when high impact positives are clearly highlighted and Major risks are not buried at the bottom or softened to the point of misleading. Override and reshapen when a)The AI is too optimistic relative to the risk profile(e.g crime or sponsor concerns are downplayed) b)Important banker specific angles are missing, example MB has explicity asked to stress sponsor's track record or focus on a particular market risk.c) You read the narrative and feel, "If I were the banker, this would give me the wrong impression." Rule of thumb: Ask "If the banker only reads the page once, would they walk away with the right high level stroy"? If the answer is anything other than Yes, then we must override AI's framing in that section. Language, Style and efficiency: "Is the AI doing better than I do manually?" Here AI is the strongest and we want to avoid overriding out of habit. Accept AI's language when a) We have already validated the facts and framing (1&2) b)The text uses US english, Is grammatically correct and clear, avoids redundancy and overly salesy language. c) Our edits are purely minor preference tweaks(e.g swapping a word, reordering a clause) Overrider or partially edit when a) The tone does not match Berkadia style ex: Too casual, Too promotional or too negative b) There are repeated phrases or clunky sentences that could confuse a banker c) The same sentence can be flagged by error matrix for language/format. Rule of thumb: Once facts and framing are correct, bias toward accepting the AI's language unless it clearly violates style guidelines or would confuse the reader. Dont rewrite just because we would phrase it differently. Safeguards to prevent over trust To stop analysts from rubbing stamping AI output we must: Mandatorily review checkpoints in the tool. Before marking a draft as "ready for MB review", analysts should check boxes likes a)" I have verified the factual accuracy of AI generated section" b)I have ensured that key risks are neither omitted nor downplayed. This keeps responsbility clearily with analyst. System warnings for high risk sections. We flagged certain sections as " always require extra check" a)Sponsor description (risk of overclaiming experience or financial strength) b)Crime/Safety commentary c)Any inference heavy market commentary. In the UI, These sections are a) Highlighted and b) Have a small message " High jugde area - verify carefully, do not rely on AI alone" Random sampling and feedback. Weekly once a random request of each analyst is reviewed to see a) Where analysts accepted AI text verbatim b)Whether those sentences were indeed fully supported by sources. If patterns of over trust is detected during audits, we will use them in training and tweaking the prompts if necessary to be more conservative. This way, In the Narratives + Berkie Market Place ( Inhouse AI of Berkadia) Analysts know exactly when to trust AI(polishing, structuring, supported summaries) They know exactly when to override(judgement heavy areas, sponsor risk, crime, deal story) And we keep pratical balance between quality, speed, and human responsbility, instead of drifting towards blind trust or total disregard of the AI.