Vidhya Rathinavelu

Digging into What's Really Breaking a Content Moderation Process (Analyze Phase) To separate real causes from noise or symptoms, in the Analyse phase, a structured data driven approach is required. RCA techniques, stratification and validation methodologies would help identify the right root cause. Data Management: Ensure appropriate & accurate data is collected is the first time to identify actual root causes. 1. Process Mapping & Identification of data points: Create a detailed process map of the Workflow, identifying each step from input to output and data storage. For each step identify all the possible data points to be collected. Consolidate identified data points and proceed to collect data 2. Data collection & analysis: Collect the data and conduct multiple analysis like Trend analysis, Pareto, SPC & Cycle time analysis on the quantitative data. In addition, depending on the problem statement, we may need to collect & analyse qualitative data as well. 5 Why analysis, Fishbone analysis, FMEA and Cause & effect diagrams are some of the statistical tools that can be used to identify the reason for the breakages in the process. Separating Real Causes from Noise/Symptoms validation of the data for any hypothesized cause. If a cause is identified, there must be a quantifiable evidence that the cause directly impacts the problem. Is the problem consistently caused by the problem. If the cause only occurs sporadically, then the identified cause May be a contributing factor to another root cause. Conduct a PoC to see if elimination of the data backed up cause impacts the problem. Assess the potential impact that is expected by addressing the cause. On the basis of the impact, prioritise the cause with the highest potential impact. Preventing Mistakes (Chasing the Wrong Cause) Concluding the causes only with data validation to avoid chasing the wrong cause. Reduce/eliminate any assumptions with regard to identifying the cause Utilise multiple data points to triangulate the cause Involve diverse stakeholders during data analysis to uncover different perspectives and challenge assumptions Vidhya R

In Measure phase when we are tracking patient wait time, the easier metric to measure would be the time taken from registration to moving into the doctor's office. Instead, the key metric to be tracked must be the "Total time from patient arrival to seeing a doctor" or "time of arrival to admission" or "Time of arrival to discharge". This will help to identify the E2E process efficiency. To catch incorrect or bad data, some of the key measures that can be taken are: Clearly define what each metric measurement means. For ex: Patient wait time -- What is the starting point & what is the ending point. What are the stages and how do we ensure that data recording is consisitent Once the stages are identified, define the data collection at multiple points to ensure that there are no misses, people who collect data are calibrated. Before deployment of the data collection plan, do proof of concepts to identify ambiguities in definitions, issue with data collection methodologies/tools Once data is collected, data visualization techniques can help to identify outliers and patterns. In addition data collection errors also can be quickly identified. Cross referencing the data across multiple stages. For ex: In a linear process of arrival, registration, tests, etc, the arrival time cannot be later than the registration time. Cross referencing helps identify anomalies. Train everyone involved in data collection on the definitions, collection methods.

An unexpected application for AI in the insurance industry is Proactive risk assessment & intervention in policy updation Use Case: Apart from only processing applications & claims, AI can also be used to understand the exceptions/variations/real time data from market through news articles to understand the trends of fraudulent activity. This learning can then be used to compare the current underwriting policies and then propose changes Prompt and Flow Logic Orchestration 1. Triggers & Analysis: AI to monitor external information available on the web on weather, news, economical changes and info related to market. AI to also analyse internal customer information for customer profile, location, existing ailemnts & claims history 2. Risk indicator analysis: Identify anomalies that indicate a change in risks relating to nature, weather, market conditions, technological evolution that may impact a customer 3. Re-evaluation: AI to re-evaluate the customer proposal/claim on the basis of these external & internal information 4. Requirement generation: AI to generate a prompt to the underwriter to recommend additional reviews which DO NOT fall within the scope of current underwriting policies 5. Recommendations to policy: AI can suggest policy updates, risk mitigation ideas and proactive customer communication on identified risks. 6. Feedback: Use AI to collect feedback from underwriters Value & Efficiency that this will create: 1. Proactive risk management 2. Early detection of Emerging risk 3. Data - Driven risk assessment Vidhya R

AI Impact: Improving Efficiency on content moderation by increasing bot moderation Problem statement: Majority of the User Generate Content on social networking sites includes harmful, hateful and wrong information. Manual moderation is time consuming and also is prone to error due to the gray areas in policy impositions. The part of content moderation that I would want to automate using AI is to quickly & efficiently moderate the user generated content. Proposed Solution: Develop an AI that will review the user submitted content and the evidences available within the system and in open sources, then take a decision of if the content can be allowed or not AI Solution building approach that I would use is: Fine-tuned LLM Rationale to use the Fine Tuned LLM: Understanding of the context: Fine Tuned LLMs can better understand the language nittygritties and identify the forms of content in line with policy Scalability: Due to large data handling capacity, LLMs are best suited in my industry Adaptability: LLMs can easily adapt to the new/changes in policies Efficiency: Fine tuning of an existing model to leverage the current knowledge base is easier with LLM Vidhya R

Imagining multiple AI agents interacting for an insurance underwriting process: Multiple interacting Agents: Sales coordination Agent: AI gathers client information post closure of sale and guides the customer through the application process Verification Agent: Agent that analyses the submitted application as per the underwriting norms Financial underwriting agent: Agent that specifically analyses the income & credit related information shared by the application and assesses the risks Decision making & documentation agent: On the basis of information from Verification & financial underwriting gent, approval of the application is processed and the required information is moved into the backend for data storage Challenges: Communication/interaction challenges between agents Data inconsistencies in terms of formats & interpretations Waiting time for one agent to start post completion of previous agent's tasks Explanation of exceptions & decisions taken on exceptions may require human intervention Design of interaction: Define a consistent data format and schema which is common across all agents Create a messaging queue to enable harmonious interaction between agents Implement a workflow engine to handle task sequencing & running of parallel agents Data maintenance: Log each agent action and decisions taken Defining clear error handling procedures Vidhya R

Sticking to the process: One situation where we need AI to take a trade off is speed vs accuracy. AI must quickly decide whether the content can be moderated, meaning deciding where human validation is required. Logics/Factors to be considered: 1. Determine the severity of the content / Modification requested 2. Confidence level of the content based on parameters that are available 3. User reputation scoring. Determine the score of the user who proposed the change/content creator 4. Time constraints for human validation On the basis of above a quick scoring mechanism needs to be built which will be the decision making logic for the content & the content provider. Rules for AI to follow: On the basis of the above logics, thresholds have to be created for where AI can take a call directly and let the change/content go live. For cases where the threshold is passed, then it needs to be passed on for human validation.

A real scenario in this process is where we have a gray area for moderating features like city, mountains, monuments, historical places etc. These cannot be moderated by bot and are currently moved for Operator validation. If we can track these exceptions efficiently, AI can be trained to improve moderation accuracy & efficiency. I would track the below: Applicable Unique identifiers, Users reporting the issue, Details of the place, Count of reports, Resolution, Category of resolution, reason for resolution. Post capturing this information for a sample set of data then identify the reasons, resources and policy considered for taking the decision, clusterise it and feed it back to AI for learning

In a Google content moderation process, AI should pause and ask a human in the following situations: Edge Cases and Ambiguity Where the content's intent is unclear. When the content falls into gray areas of policy. If the user indicates sarcasm, satire, or nuanced language. Policy Violations with Context Where the context of the content is an exception, but the content violates a policy Appeals and Disputes For cases where we receive appeals on bot decision For disputes on the basis of bot categorisation on content or error classification during RCA New Trends and Novel Content Types When new forms of non permissible content or spam/abuse trends emerge that the AI has not been trained on. If there are new ways to circumvent policies. Sensitive Topics Content is for highly sensitive topics.

In the content moderation process that I am handling now, below are the rules that I would prefer AI to enforce and not prefer AI Rules I will trust AI to enforce: Content related enforcement: AI can scan the content that is getting added, modified or deleted and highlight content that should not be permitted into the platform. This may be hate speech, cruelty, inappropriate content, etc. Policy related enforcement: Content moderation is governed by a set of policies. In these policies there are non gray areas where the AI could clearly call it out as an enforcement or violation. These cases can be handled by AI with high speed at higher accuracy than humans. Rules I will not trust AI to enforce: Policy related enforcement: There are certain gray areas in the policy which will require human intelligence. These pieces of validation are required to take the right call within the boundaries of the policies. Hence this is a task that I wont entrust to AI. Vidhya R

Hi, In my line of work, handling highly sensitive customer complaints/escalations which involves handling complex & emotional customers is a task that feels too human to hand over to AI How I can reimagine the role of AI in this: Analyse & Categorise: AI can analyse the initial complaint, the tone on the call or email and categories the level of emotional distress and flag the case details along with a sentiment analysis for operators to be prepared to handle the customer appropriately Providing real time support & information: AI can extract info from back end DBs on the past history of the customer, the issues reported in the past and the policies that can help the operator to handle the issue. Sentiment Analysis & Emotional Cues: AI can perform real time sentiment analysis to detect subtle changes in tone and call out the emotional changes during calls, thereby helping operators to tailor their responses. Generating summaries for documentation: All of the customer handlings are recorded and operators spend time to document the interaction & record + save evidences of their responses/resolutin to the highly sensitive customer complaints. AI can generate these responses in parallel to the transaction, thereby eliminating the entire manual documentation process Future knowledge & best practice sharing Using these documentations, better Training need analysis can be done. Best practice sharing by utilising the way other operators handled a complex customer can be used for enhancing training material. Also, AI can learn and evolve better to handle future customers with better insights over time. Vidhya R

To create an effective Problem Definition Tree, a Black Belt should: Focus on specific and measurable outcomes. Prioritize actionable objectives. Maintain logical connections. Avoid overlapping or redundant objectives. Limit the number of levels. Use factual data and evidence. Iterate and refine as needed. When deciding at which level to stop and finalize project Y, a Black Belt should consider: Project goals and objectives. Stakeholder satisfaction. Time constraints. Budget considerations. Change in scope or organizational priorities. Clear identification of the problem statement and process improvements. Statistical significance. By considering these factors, a Black Belt can make an informed decision on whether to stop and finalize project Y at a particular level or continue with further improvements.

The Sparsity of Effects principle is a fundamental concept in Design of Experiments. It states that, in any system, a vast majority of output's variability can be attributed to a small number of input factors. In other words, most inputs have either minimal or no effect on the outcome and only a few critical inputs significantly impact the variation. This principle can be helpful to researchers in Design of Experiments because it can help them to identify the most important variables to include in their experiments. By focusing on the variables that are most likely to have a large impact on the outcome, researchers can reduce the number of variables they need to test, which can save time and money. The principle of Sparsity of effects is useful in the following ways: Researchers can allocate their time and resources to investigate the vital few input factors responsible for most of the output variability rather than studying all possible factors. Instead of considering a whole lot of data, using this principle complex problem statements can be simplified by identifying and focusing on the most important factors. Knowing which input factors have little to no impact on the system outputs enables decision-makers to prioritize optimization efforts on high-impact factors and consequently make better-informed decisions. Sparsity in principle refers to situations where most elements are irrelevant or have minimal contribution to the overall solution or outcome. In cooking, the concept of sparsity can be illustrated through the idea of using only essential ingredients to create a flavorful dish. Consider preparing a simple pasta dish with garlic, olive oil, and chili flakes. These few ingredients, when chosen carefully and combined well, can create a complex and delicious flavor profile. Adding more ingredients like additional vegetables, cheese, etc may not necessarily enhance the dish. Sometimes, overutilisation of resources can even detract from the harmonious balance of flavors. Just like how an excellent cook like me , understands that sometimes restraint in using ingredients, can produce the best results, sparse solutions focus on the most significant components to achieve an optimal outcome.

Levene's test is used to test if a specified number of samples, which have a normal distribution, have the same variance. This test is used to identify the homogeneity of the samples. During analyse phase, sometimes, we assume that the variances are equal for different sets of samples. Levene's test is used to validate this assumption. It is done before ANOVA. If Levene's test is failed, that is, if we are not able to validate that our assumption that variances are equal is not true, then ANOVA should NOT be conducted. Bartlett's test is also used to check if variances among multiple sample sets are equal. Levene's assumes that the data is normal, whereas for Bartlett's normality assumption is not a requirement. Levene's test is less sensitive to violations of normality whereas Bartlett's isnt. On basis of the data & the requirement, either of the test will be used.

A Design Scorecard is a tool used in the design and development process to measure, validate, and communicate the performance of a design against predefined criteria. It helps designers, stakeholders, and decision-makers to align on priorities, identify potential areas of improvement, and make effective decisions about the design process. It can be used to identify areas where the design needs to be improved, and to track progress as the design is refined. By using this tool, teams can ensure that their designs meet established goals and overall project requirements. Design Scorecard in DMADV: 1. Define: In the initial stage of defining project goals and customer requirements, the Design Scorecard can establish key performance indicators (KPIs) that will align with the objectives. These KPIs will serve as benchmarks to track progress throughout the project. 2. Measure: In Measure phase, it helps track critical measurements related to customer requirements and specifications, existing process capabilities and performance standards. 3. Analyze: During analysis, the Design Scorecard supports analysis of potential root causes, by validating collected data and providing insights into gaps in performance that require attention. 4. Design: In this phase, the Design Scorecard helps to prioritise features and functions based on their alignment with KPIs and overall project goals, thereby supporting optimal resource allocation. 5. Verify: It tracks verification progress by comparing newly designed products or processes against the internal & external specification. Thus, highlighting areas needing improvement or modification before final implementation. Used case for a DMAIC project: Design Scorecard can be used in service industry for projects taken up to address the customer escalations. It can be used in Define phase to establish the targets, both internal & client targets. Comparing where the team is against the targets and to understand the differences. This then can It can be used in the analyse phase to prioritse the root causes.

In real world, the data rarely follows a normal distribution. Data is affected by outliers and measurement errors, because of which IMR chart may not be effective in detecting changes, as these charts assume that the data is normal. In such cases, we can use charts for non normal data or we can transform the data, depending on the scenario and the data type. Data transformation techniques, such as the Box-Cox or Johnson transformation, can help stabilize variances and make the data more symmetric, allowing for more accurate interpretations and applications of control charts. Data can also be split into rational sub groups and then buliding control charts on each split data. Checking for process stability using transformed data helps to identify potential problems in a process. By transforming the data, we can highlight any trends or patterns that might not be visible in the raw data. This can help us to identify potential causes of variation. Then corrective actions can be identified to improve the stability of the process and prevent any problems from re-occurring. Example: A call center will monitor the waiting times in the queue. If the data is not normal due to variability, then the distribution will be a non normal distribution and IMR chart may not accurately detect the actual problem or the trends. If the wait time data can be transformed, then the analysis on the data will be more accurate and will help the call center to take effective decisions.

A Way of Working (WoW) is a set of principles and practices that defines how an organization works. It defines the way people collaborate, communicate, and make decisions.A well-defined WoW can help organizations improve productivity, collaboration, overall performance and create a culture of continuous improvement by providing a framework for identifying and solving problems. Here are some ways that WoW can help create a culture of continuous improvement: It provides a framework of how work is done. It helps to identify and eliminate waste and inefficiency. It encourages cross functional collaboration and open communication. It supports the development of new ideas and innovation. It provides opportunities for training & development It ensures transparency and sets accountability To effectively implement WoW, organizations should focus on clear communication, training and coaching, and a commitment to ongoing improvement.

A top-down diagram is a visual representation of a process, with the high-level steps/general steps at the top and the lower-level/more specific steps below. Top-down diagrams are mostly used in the software industry to show the architecture. They may also be used to show the steps in a manufacturing process or the flow of data in a database. To use a top-down diagram in a DMAIC project, we have to start by identifying the problem we are trying to solve. Then, brainstorm all of the possible causes of the problem. Once you have a list of potential causes, we can use anothe top-down diagram to drill down further Once the root cause is identified, action plans can be determined to fix it. Please see the attached diagram for the top down flow. A top down chart is used to: * Identify the overall goal of the project. * Break down the goal into smaller, more manageable tasks. * Track the progress of the project. * Identify any potential problems or risks.

A probability plot is the graphical method for validating the distribution of data. It is a line plot of the data points against the theoretical values, which are the values of a cumulative distribution function, of a probability distribution. A probability plot can be used to assess the shape, location, Spread & outliers. It can also be used to compare the distributions of two or more sets of data. To create a probability plot, the data points are first arranged in ascending order. Then the cumulative distribution function is calculated and data is plotted. A well-fitting probability plot will have a straight line. If the data points are scattered around the line, then the distribution is not well-represented Some of the insights that we can get from a curve that deviates from a straight line are that, data: is not normally distributed. has outliers. may be skewed. Does not facilitate use of standard statistical methods to analyze the data. requires transformation or use non-parametric statistical methods. Regards Vidhya R

Multi vari chart is a graphical representation from which we cannot draw statistical inferences. It is used to identify the multiple sources of common cause variations. We can check variations across piece to piece, variation on single piece and time to time using a multi vari chart Piece to Piece helps identify variations within a part of the product/process When we want to compare two different process/product, piece to piece can be used. Time to time variation is used to compare the variations between the different times of production Interpreting the Multi Vari chart: Means for each factor must be referred and the interaction of one factor with the other and the impact on the other can be understood by the studying the trend line that passes through the mean.

The IS - IS not tool is a powerful problem solving tool used for identifying root causes by using a set of questions to identify the gap area. This analysis required detailing out the when, where, who, how and the extent of impact for a problem/concerns. It is a matrix where the above information are listed down and then against each of the input for the when, where, how, who and the imapct, it is determined if the particular factor "IS" impacting or "IS NOT" impacting. Once this is identified, if there is an "IS" impacting, then the factors that impact the problem/issue are isolated. This tool also helps in identifying patterns. For ex: If you are looking at errors and then you are using this IS-IS not analysis against each category of errors, the cause that you may arrive at may result in a pattern. Thus, this tool also helps in identifying the significant few.