Leaderboard

All the answers are worth reading. Some points might appear repetitive but there are some unique points that will appear in different answers. Answers from Yuvaraj and Jess are a must read. Sumukha has provided the best answer to this question. Well Done! My 2 cents - "Audit by Design" is a good approach to solve for this problem.

Now a days with the introduction of the Artificial Intelligence in Business processes has made the need of the Audit for the AI infused process more and more relevant. The significance of Audit in the Business processes has always been there as the adherence to the standard operating process in normal audit process is checked and this is the way Organizations re able to maintain the standardization in the process. The problem came when we introduced the Generative AI in our existing Business process of Sales and Marketing where we had aimed to utilize the AI abilities to read through multiple previous executed RFQ and RFPs so that the previous data and pricing points can be utilized. When we had onboarded the Customer relationship management tool, Our Sales & marketing team had specific need to make the process for RFP submission faster and immediate need to include the Agentic AI capabilities of the tool to help them with identify past data points from any of the submitted RFPs and to help them analyze the failure point (of not receiving the order from client as winning bid). Also, Gen AI was supposed to help the design team in reading through the specification documents and help in identifying the Client requirements so that design team can also ascertain the dimensions and hence the materials requirement for the SCM team to arrive at the BOM quantity and price. The Audit team had this task of revisiting their Audit approach in which they had traditionally attempted the Audit processes of identification of the SOPs and then validating through random data points and requesting for the validating documentations. The Audit team had to first understand the AI working, how does a AI model is prepared, its testing procedure and then validation procedure before which the AI capabilities are ready to be used by different users and then the revisited audit process of the Sales and Marketing business process enabled with AI capabilities needed to be revisited. The Audit team was able to come up with the revised Audit process in which it had split its scope in four different categories: 1. Business process without AI 2. Business process with AI 3. AI process/suggestions/prompt 4. AI model and data source build ups 1. The First category was being audited with the help of manuals and SOPs. 2. The second category needed to test the reply suggested by AI for the prompts which were asked by the team members. e.g. The prompt entered by team members like 'Suggest me similar type of RFP submitted earlier with profit margin less than 5% in the total bid of 50 lakhs?' The Audit team needed to check the Reponses suggested by the AI tool with the database and cross check and verify if the AI reposes are in line with the database and providing the same reply which could have been provided otherwise by manual searches (may be time taken would have been a tad longer ~10x). 3. Audit team also used their own prompts to check on the responses and used to validate wrt the standard documents, organizations guidelines, SOPs, user manuals, Companies Policy etc. 4. The most important parameters of testing needed to check the AI model and data sources and data validation, biasness in the system and AI ethical practices. Audit had taken help of Chief Information officer to identify the right person from Information system who could validate the AI model as to how the model has been structured, has the model been in built with certain biases in terms of socio political Reponses to be filtered out as per the country of jurisdiction or is independent, whether the responses are biased or not and was providing scores (scoring methodology already defined) and then the team used to arrive at the individual rating for the four sub processes and then the final score of the Audit process. Based on the Audit score there were three categorize of satisfied, needs improvement and needs immediate attention and improvement. All the recommendations by the Audit team were explained in details so that the overall process adherence remains high across organizations including AI infused processes. The Audit team was confident that even though they have put in a place a process to Audit the AI infused process but still this will be a continuous improvement process as the AI model itself keeps on evolving through various developments day by day.

When auditing AI-integrated systems, transparency on how the model works, the security of the data, and whether the prompt logic system is working properly all need to be evaluated. AI systems should be evaluated on whether their decisions are traceable, if there are risks of bias, and if the system meets ethical standards. Examining the sources of the training data, monitoring model drift, and examining output consistency with given prompts constitute critical tasks. Employing accountability-enhancing tools such as explainability mechanisms, inline controls, and collaborative processes bolsters overall accountability. We will ensure proper and responsible use of the systems through regular alignment checks with the overarching business goals.

First of the the AI infused process should be audited for its customer centric solution as to if it is able to provide the solution to the customer requirements without human interventions. Second part of the audit should be the scenarios which are unpredictable and can lead to incomplete cycle or solution provision for which the AI has been created. The most challenging part is to draft or provide solution for all the possible scenarios and later on it can be identified during the Audit process.

The objective of any business is to create and sustain long-term value. While AI is beautiful and popular, transformation professionals should not be lured by fad. Choosing the right project to prioritize is vital in achieving such objective. AI solution applied in all other areas of business wherever possible yet not aligned with its strategic objective is detrimental to the organization’s growth and bottom line. It can flood the AI engineers’ workflow to extent of choking its way to success. Stressful and costly system would likely to emerge. More or less, due to long queue of proposed AI solution projects, the more important one can either be missed out or deprioritized. Therefore, it is critical to the management specially to transformation professionals to establish a due diligence framework in project prioritization such as business value analysis – regardless of whether its AI solution-based or not. Business priorities can be deciphered, understood, and better aligned through collaboration with the CFO or Finance Controller. This should be the first step when it comes to project selection and prioritization. Transformation professionals should serve as the bridge between top management, operations (marketing, sales, production, supply chain, etc.), and finance. As Lean Six Sigma or Transformation Professional focuses on financial benefits of every project, aligning first with Finance is paramount. Taking this step early on would yield on project results aligned with business priorities. This method would prevent stressful situation where competing priorities prevented, and lead to real financial benefit appreciated by top management and the business as a whole. While profitability, operating expense, and cash flow are essential financial metrics, throughput projects necessary to increase profitability should take on the priority seat. This is where AI can add strategic value to the business not by fad or chance, but by focused intent.

Excellence frameworks are all about building systems that are strong and last a long time. AI solutions, on the other hand, tend to become less stable as time goes on, especially those that use prompts, flowcharts, or hard-coded logic. Below are some signs to showcase how things are getting worse and some steps to mitigate: 1. If an AI solution stops operating or isn't as useful, it will need more aid from people or more escalations. - If people often ignore the AI and call a person for help instead, it means that the agent isn't handling new scenarios or edge cases as well as it used to. 2. Metrics for people that quit or grow angry, such as: - Less use - Sessions that weren't finished - People that are unhappy with the AI or give it bad reviews argue that it doesn't meet their demands anymore. 3. A lot of answers that are vague or "fallback" Agents who use fallback replies more often, such as "I'm sorry, I don't understand," may be showing: - Not being able to figure out what someone means - Drift in the base of knowledge - Making prompts too vague is a bad thing. 4. Output that is incorrect or not helpful If your knowledge base or LLM is out of date or likely to hallucinate, you can get answers that are - That's not true. - Not new and not very important - Not following the new rules, regulations, or procedures anymore 5. Things that alter over time If you ask the same question again and get a different or less helpful answer, it means: - Fast regression - Model changes that need to be setup - Drift in settings 6. Using technology to get into debt - If it's hard to upgrade, audit, or keep watch on the AI system because of prompt flows or logic sprawl, that's a good sign that it's not secure. How to Make Sure AI Deployments Last a Long Time 1. Make loops for feedback Getting input from users: Give them the option to vote up or down, give a thumbs up or down, or rate how happy they are. Please review and examine edge cases or failures using HITL (human-in-the-loop). 2. Check how well things are working and how regularly they are. - Rate of fallback - How fast it grows - How often things go well - How accurate the intent classification is - People don't talk to each other the same way now. - If you see them, take them as a sign that anything is weak or out of date. 3. Keep training or improving models. If you use LLMs, you should retrain them or improve their prompts on a regular basis by using: - New records of talks - Updated documents for terminology or processes - Users have fresh goals or needs. 4. Keep track of the many different types of prompts and reasons. - Usage of version control system - Changes made to documents - Regression tests and test cases to be added 5. Your design should not only be right, but it should also be strong. - Like NLU and embeddings for flexible, layered intent recognition - Use fallback flows to resolve difficulties. - Don't depend too much on rules that only function in specific instances. 6. Include managing the knowledge life cycle - Periodic or automatic update of Knowledge base - Put things you don't need anymore in a storage place. - Use metadata or freshness indicators to show how new the item is. 7. Plan how to run things and review them: - There will be frequent audits every three months. - Some quality characteristics for prompts are how clear they are and how well they work in different situations. - A team or organization that makes sure AI works the way it should Simply put, sustainable AI isn't just about having the right tools; it's also about keeping up with users, data, and systems that are always changing. People might not know things are becoming worse until they stop trusting you, so it's important to keep an eye on things, make changes when needed, and have good governance.

1. The obvious is that if humans are having to correct or being more involved in the solution than prier, it's a good indicator 2. The Agent starts to provide nonsensical answers, or generating irrelevant outputs 3. Models can output confidence levels. If they drop, it means the model will be less certain of it's answer or predictions 4. Seeing performance drop, latency issues 5. Requiring more compute resources such as CPU, RAM, I/O speeds 6. Data "corruption". Anything that happens to the data that changes from it's original state that the model was trained on. As they say. GIGO 7. How we define and use words today may change and mean different in the future but the model still thinks the "old" way. Or customer preferences change, making recommendations on old preferences will not be good These are just some examples of how an AI model can become outdated and not useful. To help the models remain robust, stable and sustainable, here are some ways. 1. Have a way and always monitor the performance of the model, looking latency, confidence scores, accuracy, etc.. 2. Know what can impact or change the data the model is using. This can happen very easily 3. Use statistical methods (e.g., PSI, A/B testing, A/B/n testing with challenger models) to detect shifts in the relationship between inputs and outputs. 4. Have automated pipelines in place to retrain the model on fresh, representative data. 5. Maintain clear documentation of data sources, transformations, and usage to understand the provenance of training and inference data. 6. Regularly review data for biases and ensure it aligns with evolving ethical guidelines. 7. Keeping Humans-in-the-Loop. Human intelligence layer. Thank goodness for this 8. Clear AI governance and responsibilities. - Who is responsible and has ownership. - Establish a review cadence. -Foster strong communication and collaboration between data science, MLOps, business teams, legal, and compliance - Thoroughly document model architecture, training data, deployment processes, and monitoring strategies to ensure continuity and enable future improvements.

AI solution is becoming outdated or fragile when there is a decline in the accuracy of response or a second ticket is created for the same issue due to lack of credible answer. Another indicator will be the increasing number of escalations by the solution. Ensuring a long term sustainability will be to continually review the AI logic and train the solution with more data. We could also get subject matter experts to review and audit ensuring the alignment with the business case and user needs.

Being a AI Solution Architect, it becomes crucial that design solution which will remain sustainable and clear roles and responsibilities are assigned. This approach can be defied with the help of below points: Indicators: Exceptions: Review audit logs to understand occurrence of exceptions and the output generated by AI solution. If exception numbers are high with unexpected outputs then it clearly suggest that it is not aligned with the business requirement. User Experience: Bad user experience and not using the AI tool will indicate it is not aligned with the requirements. Misalignment with policies, SOPs, business updates – If AI solution does not take care of frequent or periodic updates then it will not be able to function effectively. Preventive Actions: Governance: Define roles and responsibilities to ensure solution remain updated all the time. Define timelines. Feedback: Ensure timely feedback from the end users to understand ongoing issues. Reporting: Ensure periodic updates shared with the respective teams. Change Management: Establish change management framework and align particular AI solution with it. SOPs: Updating SOPs on periodic basis or as and when major changes are implemented.

We had built some AI Solution/agent in our manufacture and some of them were not suitable for our process and we had decided to remove it from our process such as AI Agent for prices Our business is related to steel, and the prices of steel variety daily So, AI solution was not perfect for this process There are signs/Indicators of AI Solution Degradation in Manufacturing 1) Decreased Accuracy A decline in accuracy or performance over time, such as increased error rates or not correct results/decision. 2) Increased Complaints Increase in complaints or feedback about the AI solution's performance, such as we had received complaints from sales department regarding prices. 3) Changes in Behavior 4) Knowledge Base Drift Changes in the underlying data or knowledge base that the AI solution relies on, such as steel world prices changes. To ensure Sustainability for Long-term 1) Continuous Monitoring Continuously monitor the AI solution's performance and accuracy, using metrics such as accuracy, precision, and recall. We should take actions immediately 2) Regular Updates Regularly update the AI solution's knowledge base to reflect changes in behavior. This is very important, AI Solution should be capable to update knowledge base to reflect actual data such as in our example steel price 3) Human Oversight Implement human oversight and review processes to detect and correct errors or inconsistencies in the AI solution's performance. 4) Flexibility and Adaptability Design AI solutions with flexibility and adaptability. 5) Collaboration and Feedback Encourage collaboration and feedback between all concerns such developers, users, sales and stakeholders to identify areas for improvement and ensure the AI solution meets user needs. 6) Implement Continuous Testing Implement continuous testing and validation processes to ensure the AI solution's performance and accuracy. By following these best practices and being aware of the signs of AI solution degradation, manufacturers can contribute to ensuring the long-term sustainability of AI deployments and maximize their benefits.