Sargun Diwan

As a part of the Business and process Excellence team’s across different organizations I have been involved in numerous projects involving process optimization and implementation of RPA’s.
Having managed omni-channel contact centers, I have closely worked on implementations of ACD’s (automatic call distribution) and Autonomous IVR's for telephony set-ups. And have been fortunate to be a part of Live Chat implementation projects involving Chatbots in recent years.
With the expansive capabilities of AI, what we had set out to implement was an interactive, self-learning, autonomous “Dynamic Packaging Module-DPM” for an OTA’s Holidays business.
The idea was to integrate the DPM and Chatbot on the site, which would be able to handle Customer Inquiries related to the packages involving mostly customizations and any related questions to T&C’s, payments, offers, amendments and cancellations, upgrades, visa, insurance etc. immediately.
Given, the fact that 60% of the inquiries were for “Customized Packages” it was important for our teams to be able to handle such customers by offering complete details “tailored” to their requirement in the First Interaction itself thereby improving conversions and customer experience, hence the decision to go ahead with the project.
So, in addition, to building a Backend Product loading and managing tool involving the integrations of supplier API’s, Direct / Extranet contracts, GDS and Airline feeds, offline packages, ground transportation and ancillary services. We needed a Front end, which allowed both our teams and the End customers to be able to “Customize packages” as per their needs on real-time basis, while making use of the AI assistant. And finally, all interactions, requests, quotations to flow into the CRM and ERP tool for overall booking management, human agent and AI model training and retraining, Quality Audits using AI for all interaction summaries.
The Front-end or the “DPM” was built using AI/ML algorithms, using on-premises LLM’s, which were trained on some 100,000+ itineraries for countries across the world. The AI QA tool had been trained on Sales scripts, product knowledge, soft skills and Quality monitoring sheets to score conversations. Both these tools had their respective KB’s.
Let’s look at the project implementation in terms of restructuring of processes, new roles and responsibilities both for Human agents and the AI, handoff protocols laid down, Human-AI Collaboration framework and efforts, Phase-wise implementation plan, Training requirements for the team and Revised KPI’s.
AI-Integrated Sales Process for Holiday Package OTA
Revised Process Flow
 
-          Initial Customer Contact
• The AI chatbot initiates the first interaction within 15 seconds of a customer visiting a Landing or Package page, offering assistance to “Customize a package” or offer the “off-the-shelf products”.
• The AI qualifies the lead by gathering destination preference, departure city, travel dates, trip duration, no. of persons traveling, travel intent – couple, family, honeymoon etc. and budget, if any.
• By using set parameters, the AI assesses the complexity of the inquiry.
• Basis the complexity scores the system directs the next steps for the handling of the query.
 
-          Queries Handled by AI (70% of initial contacts)
• Inquiries regarding package availability and pricing
• Standard booking changes (dates, passenger details)
• FAQ’s bout policies (cancellation, baggage, visa requirements)
• Basic information on destinations and weather
• Payment and confirmation support details
• Post booking status updates and reminders
 
-          Human Agent Involvement (30% of initial contacts)
• Multi-city itineraries that require customizations
• Group bookings for 10 or more people
• Special requests incl. accessibility, special medical needs, meals etc
• Managing refunds that involve exceptional circumstances or exceed standard refund policies
• Catering to High-value bookings above $X,XXX per person
• Handling customer escalations or requests where customers explicitly ask for human assistance
 
Evolution of Human Agents: From being Order Takers to Travel Consultants
The agents are now responsible for offering strategic travel advice, instead of just processing standard transactions. While focusing on customer motivations, recommending upgrades, and creating WOW experiences, the agents now act as Travel Curators.
 
-          New Core Responsibilities of Human Agents
• Curating complex itineraries involving multi-city routings
• Developing long-lasting relationships with HNI and repeat customers
• Promoting and upselling premium experiences and services
• Handling sensitive customer service-related issues
• Ensuring the quality of AI recommendations by regularly Auditing conversations
• Training the AI system through feedback and updating information in AI KB for retraining
 
AI – Human Handoff Protocols
• Full conversation history and customer profile to be provided by AI to the agent with a ‘Warm transfer process’ to be followed.
• The system marks the ‘Priority’ to define the urgency of the inquiry and provides the previous interaction sentiment.
• AI provides recommendation to the agent on the suggested action items based on customer type.
• AI to ensure customers don’t have to repeat themselves through smooth context transfer to the human agent.
• Handoff scenarios include – Complex itinerary requests, emotional or sensitive situations, HNI customers, technical issues or complications.
 
Collaboration Framework b/w AI-Human
AI support by Human Agents
-        AI training and feedback
• Agents flagging incorrect AI responses through live monitoring and post-chat audits.
• Updating new product knowledge directly into AI KB.
• Creating templates for recurring complex scenarios.
-         Quality Assurance and Risk management
• Reviewing AI conversation transcripts on regular basis for accuracy.
• Providing feedback for adjusting AI tone and persona.
• Identifying and recommending cultural nuances for regional and international customers, which AI might miss.
• Ensuring AI responses comply with industry regulations, company policies and meets customer privacy standards.
-         Data enhancement
• Tagging successful booking and upselling conversations for AI learning.
• Provide inputs on travel seasons (peak, off-season, shoulder) and booking patterns for AI to incorporate.
• Highlighting objection handling patterns to assist with AI’s response improvement.
• Providing SWOT analysis against competitors for improving AI pricing algorithms.
• Regularly calibrating to improve AI response accuracy.
 
Real-Time AI Support for Agents
-          Dynamic Sales Assistance
• Recommending agents with relevant packages during live conversations.
• Alerting agents of any ongoing flash sales or inventory updates mid-conversation.
• Offering real-time access to competitor pricing during conversations.
• Identifying and recommending potential upsell opportunities based on customer profile and sentiment analysis.
 • Offering alternatives in case the customer rejects the initial recommendation.
• Providing real-time destination specific expertise when agents handle requests for unfamiliar destinations.
-          Quality and Administrative Task Automation
• 100% QA of customer conversations.
• Monitoring conversation sentiments and raising flags when necessary.
• Identifying knowledge gaps and recommending relevant training’s.
• Summarizing of customer conversations and updating CRM records.
 • Scheduling follow-up emails, callback reminders based on the conversation and commitments made by the agents.
-          Booking flow optimization
• Automating mailers to customers with booking and document checklist.
• Pre-filling of booking forms with customer data received during query and quotation stage.
 • Validating Customer’s travel documents for their visa, immigration and validity requirements, ticketing and booking confirmations etc. and alerting agents for any identified issues.
• Generating the payment and cancellation schedules basis booking and travel dates
• Populating instalment and discount offers recommendations.
-          Performance Optimization
• Tracking conversion rates by agents and recommending relevant trainings for improvement.
• Sharing insights into best practices and successful sales patterns.
• Recommending staffing for aligning Work force according to peak intervals.
• Tracking C-SAT Scores depending on the type of interaction and customer feedback.
 
Gradual Phase-Wise Implementation Strategy
• Phase 1: AI manages only basic FAQs and availability checks.
• Phase 2: Introducing booking processing and payment handling.
• Phase 3: Adding multi-city, complex package bundling and recommendation algorithms.
• Phase 4: Achieving the complete integration with personalization features.
 
Training Requirements for Agents
• Tech skills for navigating through the AI systems.
• Advanced consultative selling techniques training.
• Training on cultural sensitivity for regional and international markets – New markets.
• Advanced CX training for crisis management and de-escalation techniques.
 
Introducing a Dual layer of KPI’s
-          AI KPI’s
• Response time – Less than 5 seconds for 95%.
• Deflection rate – 75% or more of routine inquiries handled by AI.
• Accuracy rate – 95% correct information provided for each interaction.
• Conversion% - Glide path targets for the first 06 months starting from 05% to 10%.
• Availability – 99% uptime during business hours.
• Handoff efficiency – 95% of AI to human transfers happen without customer repeating themselves or getting frustrated.
• C-SAT scores - Above 4.25 out of 5.
• Average order value – Increase in AOV’s by 15% MoM.
 
-          Agents KPI’s
• Complex booking conversions% - Glide path targets for the first 06 months starting from 15% to 30%.  
• Up sales % - 30% of the bookings converted accept upgrades.
• Customer Lifetime Value / Repeat customers - 30% increase in customer retention for agent handled customers.
• Handoff efficiency – 90% of the AI transferred interactions to be resolved within first agent interaction.
• Escalation resolution – 95% of the complaints to be resolved within 48-72 hours
• AI training and feedback contribution – Achieving weekly and monthly contributions targets towards AI improvement and training.
• C-SAT scores - Above 4.5 out of 5
 
Following an integrated approach, we were able to transition from a transactional to a consultative sales process. While AI handled the maximum volume and routine tasks, this allowed the human agents to focus on building relationships and solving complex problems.
As a result, we were able to see faster response times, improved CX, and increased revenue per booking.

As a BB and having been a part of the DMAIC projects I agree that the success of the project hinges significantly on effective Change management. Change management is a structured approach to transition individuals, teams and organizations from the current state to a desired future state and is not merely an addon but a critical driver of adoption and sustainability.
Let’s look at how change management is appropriate across the five phases of a DMAIC project and the pivotal role an MBB plays in ensuring success. I would like to draw relevant examples from a project that I was a part of for an Online Travel Company working to improve its revenue for its Holiday Packages inside sales teams.
Change Management across the five phases of DMAIC
A.      Define Phase
In the define phase, we identify the problem statement, define project goals, project scope, stakeholders and customer (CTQ) requirements.
Relevance of Change management in Define Phase:
·         Stakeholder alignment – Stakeholders need to understand why the project matters and how it benefits them, hence an early buy-in is crucial. As change management ensures transparency about the “why” the project matters, hence it reduces resistance by addressing concerns upfront.
·         Communication Plan – Having a structured communication strategy is equally important to conveying the project intent, business impact and benefits to those involved. It involved including the sales leads and agents to explain how improving conversion rates will boost the Holidays Package revenue and, in turn, their incentives.
·         Sponsor engagement – Having the project sponsor’s commitment to champion the change ensures the project isn’t deprioritized during operational challenges.
Define Phase actionables:
Post kick-off of the project to improve Holiday packages sales conversions, workshops were conducted not just with the higher management, but also with team members and team leads. By incorporating the VOC (in this case the sales team members), we co-created a project charter. The agents felt heard and became partners in the project rather than showing resistance and becoming subjects to the project. They helped define the problem statement “Our current process doesn’t equip us to handle complex queries effectively, leading to lost opportunities”, which is far more empowering than “Agent performance is poor”.
B.      Measure Phase
We capture the baseline performance, validate data sources and measure current state performance (e.g. Current Conversion% of the process).
Relevance of Change management in Measure Phase:
Change management ensures that those involved accept and support the data collection process by invoking-
·         Trust in data – Sales teams often distrust data especially if performance is linked to incentives. Change management enables address concerns regarding data misuse or “hidden agendas”.
·         Clarity for Metrics – Communicating what is being measured, and why it’s being measured and most importantly clarifying the goal is to find flaws and not to blame members. Involving team members in metric development also leads to increased ownership.
·         Acceptance for baseline findings – Presenting data findings openly helps to avoid surprises later. This also builds psychological safety and readiness for the upcoming project and improvements.
Measure Phase actionables:
The sales team were explained that the tracking would involve reviewing important business metrics including AHT, number of follow-ups for converted and non-converted queries and system-generated quotations shared per query etc. We positioned it as a “Gemba” in order to shadow the agents and understand their daily challenges. We also had the team leads share a transparent funnel view of the query lifecycle, this breakdown from queries :: quotes :: bookings, led the team to understand the bottlenecks better, thereby reducing the blame game.  And we also had the team members design query trackers to ensure they were practical and captured the right information. This approach led to minimizing fear and more accurate baseline data.
C.      Analyze Phase
Identify, validate, and prioritize root causes of the problem from the collected data using tools like Fishbone diagrams, pareto charts and regression analysis.
Relevance of Change management in Analyze Phase:
This phase challenges long-held assumptions and beliefs. Change management fosters collaboration and consensus amongst stakeholders, by –
·         Engaging diverse perspectives – Like the frontline agents know the customer objections firsthand, similarly the conducting cross-functional workshops with sales, marketing and customer service teams may reveal as to why holiday package queries convert low (e.g. pricing issue, poor follow-up, script inefficiencies, unclear promotions) ensuring the identified issues resonate with those who will implement the solutions, building a collective problem-solving mindset.
·         Managing fear of blame – Teams may fear punitive action and hence can create defensiveness in root cause identification. Change management emphasizes process over people.
·          Consensus creation – By having identified root causes validated by members ensures they feel solution are built with them and not imposed on them.
Analyze Phase actionables:
The data analysis revealed that the conversions were highest when the agents instead of following a rigid script, acted upon and used the product repository to quickly create customized itineraries. This contradicted the established belief that script adherence was the key to improved conversions.  Using visuals and clear language including showing the agents Pareto charts and scatter plots, they were able to see the undeniable correlation in the data and realized the need for a new approach with focus on dynamic bundling options and not static scripts.
 
 
D.      Improve Phase
Develop, test and implement solutions to address the root causes, pilot changes and validate improvements. This involves brainstorming, prioritizing solutions, piloting changes and creating an implementation plan.
Relevance of Change management in Improve Phase:
Change management is important to ensure these changes are embraced through training, communication and support.
·         Co-creation of solutions – Involving teams in designing the new process, will ensure they identify pitfalls and will make them more invested in solutions.
·         Piloting and iteration – Involving frontline leaders and “change champions” in pilots to create success stories and generating pull from others.
·         Training and Support – Providing trainings to address both skills and mindset is a must, while making support readily available during transition. Also, acknowledging that there is a learning curve and performance may dip temporarily.
·         Celebrating Early wins – Publicly acknowledge the success from pilot teams to build momentum.
Improve Phase actionables:
The pilot for a new holiday bundling tool along with a CRM Dashboard that populated Key knowledge base articles was initiated. The pilot team provided feedback, the tool and dashboard were refined. After 4 weeks their conversions improved by 30% for complex queries. These agents shared their experience in team forums leading to peer-driver adoption.
E.      Control Phase
Sustaining the improvements and the gains, by standardizing the new process, creating monitoring plans (using control charts), updating SOP’s and have a response plan incase the process goes out of control.
Relevance of Change management in Control Phase:
Change management focuses on embedding the changes into the organizations culture and systems, or else the team’s may revert to old habits.
·         Systematic integration – Ensuring new scripts and tools become part of standard practice and not temporary fixes by using regular coaching, feedback loops and regular huddles. Additionally, aligning the job descriptions, KPI’s and incentive structures with the new way of working.
·         Performance monitoring – Making use of dashboards and control charts instead of micromanaging teams.
·         R&R’s – Re-enforcing value of the change, and sustaining motivation by linking improvements to ongoing rewards or career development.
·         Knowledge transfer – Creating SOP’s and training modules to prevent knowledge loss.
·         Handoff to Process owner – Clearly transition the ownership of the new process from the project team to the process owner who is responsible for the ongoing process performance.
Control Phase actionables:
The holiday bundling tool and CRM dashboard were integrated into the formal process SOP’s. The team’s performance scorecard, which was reviewed weekly, now included metrics directly related to the new process. The incentive structure was modified to reward both high conversions and consistent usage of new tools.  The changes stuck as they had now become the new standard of work.
 
The Role of an MBB in Driving Adoption and Sustaining Results
A project’s success is measured by the tangible and sustained improvement it brings forth in business performance and not just by the statistical analysis.
A MBBs role extends beyond the technical expertise to leadership, engagement and sustainment. Below is how they can drive adoption and ensure lasting results –
·         Communicating the vision – Translate complex data and project goals into a compelling narrative that answers “what’s in it for me?” for every impacted group.
·         Stakeholder engagement – Identifying early, and proactively and continuously engaging with key stakeholders from project sponsor to front-line agent, ensuring alignment, trust and fostering a shared vision.
·         Managing resistance – Anticipating, understanding and addressing resistance not as a problem to be crushed, but as a valuable feedback about the real-world impact of the change.
·         Training and Development – Overseeing tailored trainings to ensure everyone is confident in the new processes. Coaching project leads, process owners and team members, empowering them to become change champions to carry the improvements forward.
·         Sustaining the gains – Institutionalizing the project changes in the organization’s systems, ensuring the project’s impact endures and guarantees a lasting return on investment.
 
 
 

AI Governance Framework for Business Excellence
AI integration is transforming how decision-making, and operations are performed in organizations. As AI automates more business functions, strong governance becomes essential for responsible deployment. An effective and well-structured governance framework builds trust, reduces risks, and aligns AI advancements with organizational goals, regulations, and stakeholder expectations while maintaining competitive agility.
A.       Proposed AI Governance Framework Elements
-          Ethical Guidelines
o    These are a set of clear, non-negotiable principles that guide all AI initiatives, translating company values into technical requirements.
o    Defining acceptable use cases and explicitly prohibiting any unethical or biased applications.
o    Using reference frameworks including – EU AI Act / NIST AI RMF etc to translate these principles into policies and decision logs to ensure how each AI solution meets the guidelines.
-          AI Governance Structure and Oversight committee
o    A council of senior executives with cross-functional representation responsible for strategic AI direction and policy approvals
o    The panel reviews AI projects not only for business objectives but also for ethical standards and societal impact
o     Conducts periodic audits and model validations including ad-hoc sessions for urgent issues
-          Data Management Guardrails
o    Its imperative to maintain an AI repository with the details of the AI models, training data sources and intended usage
o    Monitoring data quality, lineage and privacy controls to ensure compliance with the adopted guidelines frameworks and the existing data-governance policies
-          Risk assessment and mitigation
o    It covers categorizing potential risks into – Operational, reputational, legal and ethical headers with their respective mitigation strategies
o    A Tiered framework for risk assessment (Low, medium, high) allows for agility by matching the level of oversight to the potential impact of the AI projects, thereby, ensuring low-risk projects aren’t affected by unnecessary governance whereas the high-risk projects receive intense scrutiny
o    The protocol also covers the real-time tracking of AI performance metrics, bias emergence and unexpected outcomes with incident response procedures for addressing AI system issues
-          Stakeholder engagement and communication
o    It involves including the employees / end-users, customers and the external advisors in the loop during design and post deployment of AI projects, to ensure that development and deployment of AI are not done in silo
o    Comprehensive training for teams to understand AI capabilities, limitations and their role in governance
o    Publish the explanation of the AI models purpose, performance and disclosures to build trust with customers and partners
 
-          Performance and accountability mechanisms
o    Define AI performance metrics to measure accuracy, fairness, and business impact of AI systems
o    Recording of AI decision making processes, model changes and associated governance activities
B.       Stakeholders for AI Governance
 
Stakeholder
Role and Responsibilities
Chief Ethics Officer / Governance Lead
Manages the ethical application of AI and chairs the AI Governance Committee.
IT / Data Science Teams
Ensure models are technically robust, monitored, explainable, and secure.
Business Process Owners
To validate AI outputs against the business goals and customer outcomes.
Legal & Compliance
To ensure AI systems comply with regulations, data and privacy laws, and any ISO standards and AI frameworks, as applicable.
HR & Change Management
Conduct training, initiate communication, and change readiness for AI-impacted teams.
Internal Audit
Regularly review model performance, risk, and controls.
 
C.       Balancing Agility and Control
-          Real time monitoring and Alerts
o    Use of monitoring dashboards to track live model performance, flag issues and trigger alerts for intervention, thereby closing the gap between operations and governance.
-          Controlled Pilots and A/B testing
o    Iteratively test AI models in a secure environment before deployment to track issues during development itself.
-          Living document and Fact sheets
o    Document the assumptions, limitations, training and retraining cycles and model versions for transparency and control.
-          Continuous feedback loop
o    Use feedback from users and business scenarios into model retraining processes to support continuous improvement and ensure alignment with organizational objectives.
Subsequently, we can conclude that an effective AI Governance framework anchors the principles of  Transparency by laying down clear guidelines and documentation; Accountability by defining roles and responsibilities and putting in place the required controls and continuous improvement through real-time monitoring, feedback and evolution of the governance framework based on the best practices and stakeholder needs.
By adopting globally established standards and frameworks in AI governance, organizations can harness the transformative power of AI without  compromising ethical or operational integrity, while achieving its business excellence goals of quality, cost optimization and super customer satisfaction.
 

Proposed Business Excellence Metrics for the AI Era -
The use of AI in the core business processes is reshaping how value is created and delivered by organizations. Subsequently, the traditional KPI metrics we have used to measure performance in areas like quality, cost, and efficiency are becoming insufficient and redundant. Using these old metrics in an AI-driven environment can be misleading, causing organizations to optimize for the wrong behaviors and not reap ROI on their technology investments.
 
Let us begin by assessing the Traditional metrics and their shortcomings in an AI driven environment.
1.      Assessment of Traditional Metrics
Metric 1: First Call Resolution (FCR)
It has long been a primary KPI to monitor contact center efficiency and customer satisfaction, indicating a low effort experience for the customer and low cost for the business.
In an AI-Driven Environment:
Using AI-powered chatbots, IVRs, and self-service portals to manage simple, high-volume, transactional queries is an attempt to give the “Easy” Calls today to machines instead of humans. These were precisely the calls that used to be FCR wins for human agents. By filtering simple issues, AI is ensuring that the only calls reaching human agents are the complex, emotionally charged, or multi-faceted problems that the AI could not solve. And it turns out that these problems are more difficult to solve in one phone conversation. Following these developments, a high FCR rate might actually be a concern! It could potentially indicate that the AI is not being effectively used to screen issues, or human agents bring complex problems to a premature close just to attain an outdated target. While a lower FCR could signify that agents are appropriately handling the highly complex issues that require follow-up, research, and collaboration. Metric 2: Average Handle Time (AHT)
AHT measures the average duration of customer interaction. It has been a pivotal metric in gauging operational efficiency, used for staffing models and cost control. The goal has always been to reduce AHT.
In an AI-Driven Environment:
Since the calls that are able to reach human agents as mentioned above are likely to be important ones. We shouldn't be obsessed with how soon the agent can get the customer of the phone but rather with what quality and value is one giving. A complex issue, high-value customer retention or an upsell opportunity might require a longer AHT. Stressing agents to cut AHT on complex calls can have detrimental effect not only with regards to poor outcomes, customer churn, and repeat calls (which negatively impact other metrics). The AHT metric also disregards entirely the time customers may have already spent interacting with an AI chatbot, rendering the “AHT” only a partial — and potentially misleading — view of the overall customer journey effort. 2.      Proposed New Metrics
In order to track performance in an AI-driven setting, we need new metrics capturing proactive problem-solving, and the utility of human-AI interaction.
 
 
Proposed New Metric 1: Proactive Resolution Rate (PRR)
PRR is the ratio of potential customer issues that are identified and resolved proactively by the AI system before the customer initiates contact. PRR Logic o The AI tracks customer journey data, usage patterns, and system logs for anomalies that indicate there is a    problem in the process (e.g., missed payment, delayed delivery, odd user behavior in a software application).
o The AI then initiates an automated resolution using the SOP’s, FAQ’s and KB updates to assist the customer (e.g., retries the missed payment, informs the logistics partner, proactively sends a "how-to" guide, or sends a system alert to the user).
o PRR Calculation: (AI-initiated Proactive Resolutions / Total potential issues detected) x 100
·         This metric, most importantly, switches the mindset away from reactive service and illustrates the value of preventative excellence. It captures a measure of the organization's ability to avoid problems, which is a far stronger indicator of operational excellence and customer-centricity than how effectively it cleans up messes.
Proposed New Metric 2: Human-Assisted Value-Add (HAVA)
·         HAVA Score is a metric for evaluating the efficacy and efficiency of human agents involved in complex situations escalated by AI. The HAVA Score replaces the use of simplified metrics like AHT and FCR for these high-value encounters.
·         HAVA Logic: The HAVA Score is calculated after the interaction and based on a weighted calculation of the following:
Problem Resolution Success (40%): Was the customer's issue ultimately resolved? (Binary: Yes/No, or a scaled rating). Customer Sentiment Analysis (30%): AI parses the text or audio of the communication to measure customer sentiment levels (i.e., measuring if the customer's levels of frustration decreased, positive language increased, etc.) Customer Lifetime Value (CLV) Impact (20%): Did the interaction led to customer retention, a new purchase, or an upgrade, this can be done by mapping the service interaction to CRM data. Knowledge Base Contribution (10%): Did the agent record the solution for this unique problem, so it could be used for training the AI in the future? (thus helping avoid similar escalations). ·         HAVA provides a path away from basic efficiency and instead reflects the true value of the human agent in the world of AI. HAVA rewards agents to be thorough and empathetic problem-solvers. HAVA also promotes a learning cycle in which the agent is incentivized to make the AI smarter through KB updates, contributing to the improvement of the system over time.
3. Linkage to Business Excellence
These proposed metrics are directly aligned with the core principles of Business Excellence.
Business Excellence Principle
How Proposed Metrics Align
Customer Centricity
PRR is a measure of an organization’s ability to solve problems before the customer even knows about them, it is the most efficient form of customer-centricity and true commitment to an effortless experience.
The HAVA Score ensures that when customers do need to talk to a human, the focus is all about solving their complex needs and maintaining the relationship that impacts their perception of value and care.
Operational Excellence & Quality Improvement
PRR actively measures the quality of operational processes. A high PRR means that the underlying systems and processes that are driving the standard approach we work towards, are efficient, intelligent and self-healing, which is an essential component of modern operational excellence.
The HAVA Score assists and develops an environment for continuous improvement. Agents are rewarded for contributing to a knowledge base, ensuring human knowledge is captured, and then used to build up the overall human-ai capability to get smarter and smarter, and to be able to do more at scale over time.
Employee Engagement & Empowerment
HAVA, also enhances the human agent's role from "call handling" to "resolution expert or relationship builder." It enables and rewards them for spending time in solving complex issues whilst creating value - leading to higher job satisfaction and lower turnover. It recognizes and rewards the value of empathy, creativity and complex problem solving that are inherent to being human.
Value-Driven Leadership
With these metrics available to leaders, they can get a clearer and more informative view of their business performance. Instead of managing counterproductive metrics, they can focus on the real priorities: stopping customer issues before they occur, getting the most value for each human engagement, and designing a learning system that continuously improves with every transaction.
 
 

So by using structured root cause analysis and hypothesis testing techniques, we can validate the findings and propose targeted corrective strategies.
In addition, we need to start by analyzing different sets of data to understand trends, timelines, forecast accuracy etc.
Data sources may include:
Sales Data (12 months): Seeking trends to indicate a mismatch between high-demand periods and stock availability leading to Lost Sales impact.
Inventory Records: For stock-out frequency of high-volume SKUs across the above period.
Forecast Accuracy Reports: To check for variance from actual demand.
Delivery Timeline Logs: For Supplier delivery windows. On-time delivery performance >95%, suggesting minimal impact from logistics delays.
Key Insights identified :
Forecast Error Correlation: High forecast inaccuracies coincide with stockout events, especially for promotions and new product launches.
Replenishment Lag: Time between forecast input and stock arrival often spans 10–14 days, reducing responsiveness.
Demand Volatility Ignored: Forecasts do not factor in localized demand surges, social trends, or weather-related events.
Inventory Turnover Analysis: Low turnover rates in some categories suggest misallocation of inventory resources.
 
Analysis Techniques Used
1. Process Mapping (SIPOC Analysis)
2. Root Cause Analysis - 5 Whys Technique
Problem: Products are frequently out of stock
3. Fishbone Diagram (Cause & Effect Analysis)
4. Hypothesis Testing
Test 2 primary hypotheses using statistical analysis:
Hypothesis 1: Delivery delays cause stockouts. We found out that on-time delivery has remained stable at > 95%, so we can confidently disprove this hypothesis. It's noise, not the cause.
Hypothesis 2: Poor forecasting drives stockouts. By mapping “Forecast accuracy %” against “stockout incidents” we can find out the correlation between them. With a significant p-value we have statistical proof that this is the primary driver.
How to Prevent Chasing the Wrong Cause -
 
Define the Problem with Data - Begin with a problem statement from the Measure phase (e.g., "From Q2 to Q4, stockouts on A-list items increased by 40%, contributing to an estimated $1.2M in lost sales."). This is our ultimate benchmark. If an identified "cause" doesn't statistically impact this metric, it's not the right one!
Gemba walk – Speaking with the store managers, the inventory planners, and the logistics coordinators is important as they have qualitative insights that can help with data analysis.
Use a combination of Tools: Using a combination of tools like the Fishbone Diagram provides the structure, the Regression Analysis provides the statistical proof, and the 5 Whys provide the deep dive.