Leaderboard

For an audit, there will be an audit checklist. The audit checklist is key to an audit process. It does not matter whether its for a traditional process or an AI-infused process. Having said that lets quickly see the response for these questions : What new questions, checkpoints, or risk indicators should be included when auditing a process that includes AI components? 1. Do all the AI components serve the intended purpose 2. Do all AI components adhere to the Data governance & data privacy policies and procedures (Risk indicator) 3. Do the AI components adhere to the Safety/Security regulations (Risk indicator) 4. Do all AI components are easy to diagnose/dissect if there are any issues to fix 5. Do we have the right metrics to measure the performance of the AI components (checkpoint) 6. Do we have a benchmark/threshold value(against the traditional way of doing the process) defined for the AI components as how effectively deliver in terms of timeliness, providing quality output/outcome etc.. 7. Do the AI components as any ethical concerns (Risk indicator) 8. How much AI knowledge the auditor has, to audit an AI infused process – based on this only, the audit checklist can vary and the effectiveness of the audit itself will rely upon (Risk indicator) For instance, if I want to create an AI systems for an insurance company for creating an insurance policy , I may need to have several AI agents. May have an Agent for underwriting work, An Agent for Premium, An Agent for collecting user (customer) data and an agent for Policy creation. Now I need to check if all agents serve its intended purpose and also all agents should integrate and work in unison so as to get a policy created properly and in a timely manner. The AI agents also need to ensure that the regulatory rules/laws and compliance to GDPR laws of the region/country should be captured as of on the date of policy creation. They also should focus on any ethical concerns that might pop out as part of this process. These are potential area of risks. By ensuring the above audit checklist questions , checkpoints and risk indicators are addressed, we can ensure that this Insurance Policy Creation process can be smoothly done. How would you ensure transparency, fairness, and alignment with business goals? 1. With the help of a visual radiating tools like Kanban boards, display the state of each AI-infused process, to the stakeholders 2. As we deal with AI infused processes, the BRD plays an important role here. We should have the goals, objectives clearly articulated and communicated to all the stakeholders 3. Periodic feedback of the AI-infused processes will provide a overview of each such process and comparing with the BRD can point out whether the current state of the processes are in alignment with their business goals Conclusion: In my perspective for AI infused processes, the fundamental difference in auditing is the AI knowledge possessed by the auditor. IMHO, an ideal way to do an audit this type of AI-infused processes is to combine human with AI. So when an audit happens for AI-infused processes, the audit checklist can be fed to an AI Audit agent which can handle that and which can self-learn over a period of time (inspect and adapt) and then that can be overseen by a human auditor. This way the audit can be more effective while dealing with this AI-infused processes.

For a AI infused processes there is a need to shift from traditional audit approach to a more dynamic audit approach which can models, prompts, flows. Since AI uses these approach for its BAU working so audits should be more focused on data integrity, accountability, ethical compliance. Below mentioned are the approaches that can be used while building AI audit framework: 1. Prompts and Flow Designs Questions to ask - Are they documented and have a version control mechanism in place? How is the ambiguity handled? Check Items - Prompt repository, Flow Diagrams for execution. 2. Model Governance & Lifecycle Monitoring Questions to ask - What type of AI model is used? Who trained the model and what type of data is used? Is there any documents and updation policy? Check Items - Model validation, Testing, Drift detection, data revision mechanism, Schedules. 3. Data Integrity & Bias Questions to ask - Is data input clean? Are there any detection system for detecting Bias? Check Items - Bias Audits, Data steps to be followed, data lineages. 4. Compliance & Ethical Alignment Questions to ask - Does AI comply with regulatory guidelines set up a board? Are guidelines embedded in the design and the system Check Items - Regulatory mapping, gap analysis,. 5. Human In Loop Questions to ask - Are there any human decision points integrated? is there any recovery mechanism if AI fails? Check Items - HITL workflows and paths to be checked. Feedback Loops. Below mentioned are the best practices for ensuring fairness and transparency: 1. Documentation of AI processes should be mandate. 2. Continuous monitoring for drift, bias, performance for the AI model. 3. Proper aligned communication strategy with stakeholders and business objectives. 4. AI audits at a set frequency.

From what I have read on this topic, most companies are going to more interested in how we implement AI governance (understand regulatory frameworks), spot and manage AI risks, ensure ethical AI behavior, build explainability and transparency into our models, and the being able to track the AI compliance with precision. Regulatory Framework: No what policies that any AI solution must abide. This mean, in the USA, Federal, State, and of course, corporate policy. “Operationalizing” these laws means conducting privacy impact assessments, documenting AI lifecycle processes, ensuring human oversight, and keeping up to date as regulations evolve. Some ways in which we can develop and enforce these policies into our organizations would be: 1) building cross-functional AI governance committee, 2) define ethical principles and acceptable AI use cases, 3) establish an internal review process for approving model, 4) integrating checkpoints (e.g., bias testing, data review, etc..) into the development process or workflow, and 5) provide organizational-wide training on AI compliance. Risk management needs to always be in the forefront of any AI solution we create. Creating the solution is not the difficult part. Managing the risks will be and it takes a lot of due diligence, hard work and strict compliance. Some of the tools we can use are 1) regular audits and compliance checkpoints, 2) thorough documentation of how the model was trained and how it performed, 3) implementing some 3 party monitoring tools to detect drift and anomalies, 4) embedding risk reviews into the agile development workflows, 5) collaborating early with the legal and compliance teams, and 6) designing a sandbox environment for ethical experimentation. Ethical considerations need to be evaluated along with addressing potential ethical conflicts. Some of the key considerations are 1) avoiding harm to both the company and the customer, 2) promoting fairness, 3) preventing the misuse or overreach of the AI solution. Some of the documentation and reporting that can be produced to demonstrate AI compliance are 1) model cards (describing model intent, limitations, and metrics), 2) datasheets for datasets (providing detail such as origin, composition and bias review)

That’s true today; traditional auditing methods primarily focus on static inputs and outputs, as well as human decision points. However, when we discuss incorporating AI into business, we need to think beyond traditional methods. When we introduce AI into the mix, we need to consider its impact on business processes. We need to shift from static checklists to dynamic checklists, as the very nature of an AI-infused system is to evolve; so, our auditing should also evolve. The following criteria should be considered when auditing the AI process. 1- The quality of data is crucial, as it should be dynamic, and we continually ingest high-quality data. 2- The performance of the process is increasing or decreasing. 3- The process is handling the intent. 4- Provide improved prompt templates and regularly test them. 5- Continue testing new real-world cases, looking for new edge cases. 6- Keep on updating documentation and SOPs. 7- Revisit training data sources for accuracy and completeness. 8- Evaluate prompt engineering regularly for safety. 9- Keep feedback in the loop when AI outputs hallucinate. 10- Audit of change logs and tracking of model updates. 11- Does AI generate outputs following a logical path or in response to a prompt? 12- Is the system compliant with regulatory requirements, including HIPAA and GDPR, as well as ethical standards? 13- Does the system support business goals, e.g., customer satisfaction and revenue growth? Practical Implementation of Controls to Ensure Sustainability. 1- Firstly, testing should be comprehensive, using both scenarios and real-world examples. 2- Continue to monitor the quality of outputs, user feedback, and hallucinations. 3- Implement CI/CD for proper checkpoints and change management. 4- Always have up-to-date documentation, version control, and traceability of prompts. 5- Review of ethics and compliance regularly. 6- Include SMEs from cross-functional teams, e.g., data scientists, legal teams, and ethical champions. Due to the changing or dynamic nature of AI systems, it poses unique challenges and risks. 1- Decision outcomes are often unclear, and unintended consequences can lead to discriminatory practices. 2- AI systems are overly reliant on LLMs without a human in the loop. 3- Low-quality LLM prompts can result in malicious manipulation. 4- As it relies on the quality ingestion of data to LLM, performance will degrade if retraining is irregular. Conclusion. As mentioned earlier, an AI system audit should be dynamic and introspective, examining why AI did X instead of Y. If we want our AI system to evolve with new real-world challenges while being safe, ethically aligned, and aligned with business goals, we should keep humans in the loop, maintain transparency, and closely monitor the system from day one.

Traditional audit methods emphasize procedural adherence of the process (Inputs, Outputs, Compliance). At times, it also audits human accountability. However, AI systems introduce dynamic, opaque, and non-deterministic elements that require a broader and more adaptive audit framework. However, this technique will not be more appropriate as decisions taken by AI are based on inputs given to the program. Expanded Audit Criteria for AI-Infused Processes: Model Design & Development: this is to understand flawed or biased training data can lead to systemic errors or unfair outcomes. Are the AI models trained on high-quality data? Did the model documentation covers the assumptions and limitations? Is the purpose of the AI model aligned with business objectives? Decision Traceability: One thing that stood out — we’ve got to be able to explain how AI is making decisions, especially for anything high-stakes. Can we walk someone through the logic or input that led to a particular output? Are we keeping a log of changes to prompts or how the workflow is set up? That’ll be important later. Fairness, Bias & Ethics: AI can behave oddly if the training data’s off. We don’t want something that treats one group unfairly without us even realizing. Have we checked if the model is biased — like giving different outcomes for different user groups? Is anyone reviewing this regularly, or is it a one-time test and forget? Monitoring & Drift: AI isn’t static. Even if it works well today, it might not tomorrow. Are we keeping tabs on how it’s performing week over week or month over month? If things start drifting or it gives unexpected results, who jumps in and fixes it? Human Oversight: Automation’s great — but we still need people in control. Do we have any control points where someone can step in and override a weird decision? Are users confident about when to trust the AI and when to double-check? Some Observations in my Experience: Results that are unpredictable or inconsistent No one taking ownership when something goes wrong System changes happening but not being recorded Unclear governance, Details of responsible SPOC Full reliance on a tool, people response that they did not build and don’t understand fully Real-World Audit Tips I. Build on What We Already Do We dont need a brand-new process. Just add sections to current audit checklists by looking at bias, model changes, decision logs, etc. II. Assign a Go-To Person or Small Team Someone needs to own AI governance. Ideally, someone who understands the tech but can also speak business risk. III. Push the System — Hard Try edge cases. Throw in unexpected inputs. See how the model reacts. We learn a lot from how it handles the “weird stuff.” IV. Keep a Short, Clear Record for Each Model Doesn’t need to be fancy. Just include the model's purpose, last test date, any known issues, and how often it gets reviewed or updated. Make Sure It’s Transparent, Fair, and Aligned Transparent: Can someone outside the tech team follow what happened and why? Fair: Regular bias checks — not just one-and-done Aligned: Is the AI helping meet business goals or just running because “we have it”? At the end of the day, the check points is not just about the technology. We need to be able to establish: “This is what the system did, here’s why, and we are confident it was the right call — or we fixed it if it was not.”

Audit of AI infused process is important to ensure compliance and accountability, since it verify that AI systems function or operate as per the regulatory requirement, and uphold transparency and public trust. Followings are the key points that should considered during the audit: Ø Goal & Scope: It is important that goal and scope should be clearly defined. AI audit is very important to ensure effective examination of the system’s performance, compliance, and standards of ethics. Ø In-depth data and algorithm evaluations: It is important to confirm the accuracy and integrity of data and to rigorously evaluate algorithms in order to ensure the accurate functionality and uphold principles of fairness. Ø Implementation of continuous improvement: Action to be taken on audit findings and implement regular monitoring are critical step to maintain compliance and consistently improve the perfoirmance and reliability of AI System. Followings are the main area’s for an AI Audit: Ø Define the audit scope by determining which AI system will be evaluated and identifying the specific component or process to be examined. Establishment of clear objectives, evaluation criteria, and success metrics to ensure alignment with organizational goals and regulatory requirements. Ø Collection of relevant data related with the AI system i.e. input datasets, documentation related to algorithm, and output results. Make sure that the data is cleaned and validated to facilitate accurate and effective analysis. Ø In order to check whether the AI model function correctly and same are free from errors algorithms review play crucial role. Ø It is important to verify the AI systems in order to ensure that model comply with relevant regulations and standards, such as GDPR and CCPA. Below outlined checkpoints should considered while auditing AI component: Category Checklist Item Governance & Accountability Is there a designated owner for the AI system? Have the role and responsibilities been clearly defined and communicated ? Data Quality & Management Has the source of the training data been properly documented ? Has the data undergone through evaluation to ensure that it is free from biasness? Is the data thoroughly tracked from its original source or point of origin? Are data privacy and consent requirements are met with the policy? Has the model been validated using the data set that are both diverse and representative of use case study Explainability & Transparency Can non-technical stakeholders can understand the decision of AI model’s ? Are prompts and flows (for LLMs) are accurate? Security & Robustness Have appropriate controls been implemented to protect AI System from adversarial attacks? Compliance & Ethics Is there a formalized process in place to conduct for ethical review of AI use cases? Are audit logs and documentation retained for compliance? Business Alignment Are the goals of AI initiatives aligned with the objectives of business Are Key Performance Indicators (KPIs) clearly defined and actively monitored to assess the performance of AI system? In order to maintain transparency, fairness, and alignment with business goals, following approaches should considered: 1) Transparency: · Maintain audit trails for data, models, and decisions. · Use model documentation tools (e.g., Model Card). · Provide user-facing explanations for AI decisions. 2) Fairness: · Ensure that AI systems consistently uphold principles of fairness by avoiding discriminatory outcomes. · Conduct bias periodic audits regularly. · Engage diverse group of stakeholders throughout the model design and testing phase to ensure inclusivity, fairness, and broader perspective in decision-making · Implement feedback loops to catch and correct unfair outcomes. 3) Alignment with Business: · Define clear KPIs for AI performance that align with business goals. · Ensure CFT collaboration · Use AI governance frameworks

As a former Quality Management System auditor and QMR (Quality Management Representative), I would still rely on ISO standard to facilitate audit on AI-infused processes. ISO provides internationally and consensus-driven management system guideline from strategic down to operational level that span to multiple sectors, not just industry. Introducing ISO 42001:2023, this new standard focuses on building trustworthy, transparent, and risk-aware AI system supported by good governance, responsible leadership, and continual improvement. This is also known as Artificial Intelligence Management System (AIMS). Like any other ISO standard, AIMS is structured to facilitate thought-provoking questions such as but not limited to: “How to monitor performance and ensure AI behavior aligns with intended purpose?” “What are the requirements in conducting internal audit of AI Management System?” “When and how to address non-conformities and take corrective actions?” “What are the AI Lifecycle stages?” These guide questions were embedded in ISO 42001:2023’s clauses to steer AI solution experts and management alike in establishing a fit-for-purpose management system manual. In addition to ISO, organizations can also refer to NIST AI Risk Framework. NIST is a U.S. government agency that develops standards, guidelines, best practices, and measurement techniques with primary focus on Cybersecurity, IT, and metrology. AI professionals and management alike can refer and use this framework which was designed to help organizations across industries and domains to manage risks of AI systems and improve their trustworthiness.

Traditional audits will be more suitable for regular processes, however to audit an AI Infused process it would be challenging due to static check points. To audit a process which includes AI components, we would require a modernized & robust mechanism which includes dynamic decision making with evolving logics. The Expanded audit criteria for a process with AI components aligning with Business excellence include: Integrity of Prompt/Flow – Verify whether the prompts, logical decisions are properly version-controlled. Track Decisions – Is it possible to validate the model, logic or prompts that how the decisions are made by the model. Regulatory compliance – Verify that the AI models comply with all the required regulatory norms Bias – Does the models take the datasets & logic removing all implicit biases Strategic Alignment – Does the models are mapped & aligned with all Business metrics & KPIs Fragile or outdated model – Verify that the model still providing required outputs with current data or need to be updated. Escalations – Check the frequency of the errors or any repeated exceptions in the flow of the model Interpretation – Can all Business stakeholders understand the decisions taken by the model or any need for explanation Considerable risk factors: · Accuracy of the model goes down and lost its effectiveness · Logical flows not matching / aligned with change in business · Inappropriate mechanism to report issues / improvement suggestions · Outdated knowledge base feed into AI models To ensure Transparency, Fairness and alignment with Business Goals: · Frequent review of modes along with owners, data teams & the users · Maintain a clear version history of all the changes · Create surveys, dashboards and track all override logs to align with KPIs · Validate the models to understand the value it creates against the mapped metrics through different tools. Implementing the same in real-time scenario requires: · Templates with weighted scores across different criteria’s · Dashboard / scorecard to have a better tracking & alignment with Business goals · Train resources to include additional criterias for autiding

Auditing a process that uses AI needs a big change from how audits are usually done. AI introduces things that are changing, unclear, and flexible, which means we need to think differently, use more criteria, and set new checkpoints. This is a full and useful tutorial that was made to deal with these problems: 1. New standards for reviewing procedures that use AI a. The model should be easy to read and understand. Audit checkpoints: - Can folks who aren't tech-savvy understand and follow what AI says? - Are SHAP and LIME like simple models used to explain why it made its predictions? Risk Sign: Black-box models that are hard to understand but have a big effect on business. b. Points to verify for data integrity and governance: Audit checkpoints: - How good is the documentation and usage of data sources? - Do you routinely examine the quality of your data to see if it is biased or drifting? Risk Sign: Using datasets from other people without checking them or understanding where they came from. c. For LLMs, look at the flow and the prompt. Audit checkpoints: - Do individuals check prompts on a regular basis to make sure they are safe and work the same way every time? - Do you check and version prompt flows as you do with code? Risk Sign: Making important decisions (like investment advice or legal summaries) based on clues that haven't been checked. d. Checkpoints for the Algorithmic Fairness Audit: Audit checkpoints: - Are the results checked for demographic equality, equal opportunity, or other norms of fairness? - Has the group thought of a way to define "fairness" that works here? Risk Indicator: Different results for protected groups, but no proof that they were lowered. e. Checkpoints for Human-in-the-Loop (HITL) Controls: Audit checkpoints: - When do you need someone to look at your work, and when can you skip it? - Do individuals learn how to understand what AI can't do? Risk Sign: AI takes important decisions without someone reviewing them. 2. Putting it into action in the actual world a. Framework for Governance - AI oversight to be added to current risk and control frameworks like COBIT and COSO. - Give people jobs like data stewards, AI product owners, risk officers, and model auditors. b. A list of models and prompts - Write down all the AI parts you have, such as LLM prompts, fine-tuned models, and decision pipelines. - Add details about the purpose, owners, level of risk, and last validation date. c. AI Audit Trails - Keep track of user interactions, model versions, inputs and outputs, and decision scores automatically. - Make logs that can't be changed and that auditors can see. d. Revalidation every so often - Models should be re-audited if they are retrained, altered, or the data distributions change. - Set up triggers for things like a drop in performance, drift, or changes in the law. e. Toolkits and automation - You can use AI Fact-Sheets, Model Cards, and Audit-ML to check that all of your documents and reviews are the same. - Set up monitoring dashboards to obtain hazard notifications right away. 3. Some risks of AI and how to avoid them Type of Risk: Make a Plan to Reduce It - Data Drift Checking data all the time and making new levels of training - There is bias before and after model fairness testing, as well as during adversarial validation. - Not clear thinking Add frameworks for AI that can be explained and prompt injection. Cleaning and checking user input immediately - Don't put too much faith in AI; make sure there are clear guidelines for overrides and HITL checkpoints. - Not following the rules Check for legality and conformity at every stage of the model's life cycle. 4. Making sure that everything is in line with the goals of the business KPI Mapping: Link AI results to business KPIs like return on investment (ROI) and customer happiness. - Ethical Guidelines: Use AI in a way that is in line with your company's values and ESG goals. - Include people from other areas, such risk, compliance, and business, in the model's design and audit. - Scenario audits assess AI's ability to handle hard situations, like edge cases, stress tests, and other inputs that are meant to be hard for it to handle. Summary: The audit checklist now has new and significant topics to look for. Description of the model and why it was created Checks on the source and quality of the data Controls for fast engineering Fairness metrics and analysis at the group level Watching and logging in real time Figuring out who is involved and in charge of what By adding these AI-specific checkpoints to their audit frameworks, companies can design their AI appropriately while also keeping trust, compliance, and strategic alignment.

Thanks for picking the brain on this very important, relevant and much need of the hour topic. Auditing an AI system is very different from the traditional systems audit on various fronts. Traditional systems audit has a standardized framework widely accepted by various governing bodies across the world. By and large the process knowledge, the skill required , the checks and validations are mostly well defined and standardized across the globe. But for the AI systems audit, the above is not well defined and does not have an unified globally accepted or mandated requirements governed by a single entity, as it is very dynamic and ever evolving. I did some research and found that the below are some of the institutes trying to develop frameworks and standards for the AI landscape. 1. National Institute Of Standards and Technology (NIST)- America 2. Institute of Internal Auditors ( IIA)- America 3. International Organisation for standardization ( ISO)IEC 42001) - Independent 4. BSI (British Standards Institute) - UK 5. European Commission - Europe Union From the above it is evident that not a single institute governs the entirety of AI Auditing framework. Each country (US, UK and Europe Union) has separate institutes to standardise the auditing of AI systems. In my view, though most of the steps in traditional systems audit is still applicable to AI system audit, the main challenges will lie in the following 1. Quality of Data : How does one determine if there is bias in the data or not? How does one test the data on bias with which the Machine Learned? As we know the LLM model can use internet for specific searches and collate info based on models , what if the information that is available in the net by itself is bias?? Who knows the truth? This validation can be done only by qualified person who knows the unbiased factual truth. So error is inherent in AI..But what percentage of it is acceptable? Will it be measured by the impact that it creates in the business or the margin of error standardization irrespective of the impact? This is very challenging to decide. So I believe the companies and the qualified auditors have to reach a common point here which is easy said than done. 2. Reviewing a model: This is where the qualifications of the auditors come into picture. To review the AI model one must be an expert in statistics to point out of the results by AI is a flawed one. There are very complex algorithms involved and the time and effort required to decode and test the results will also be a key factor. What kind of sample would you choose ? Is sample relevant for AI auditing ?? How many kinds of adverse questions that can be framed for an AI to answer ?? What key elements should one focus on the adverse or trick questions to assess??.These are few questions I believe are very challenging to answer. 3. Data Governance and Security: AI can integrate with multiple systems through various interfaces, I even learn that AI can integrate with another AI agent form a different platform, while we can appreciate the capabilities one should also think how safe and secure the data is?. Can it be prone to hack, manipulation etc..I have no answer but I am sure the governing bodies will very soon. Even then it will still be an evolving one. 4. Cost of Audit: Any traditional system audit will not be more than couple of days or utmost a week. But auditing an AI system, since there is no standards the time spent on auditing will remain a debatable topic.More the time, more the money spent by the business on Auditing. I will not be surprised if Companies apportion significant amount of money in the budget just for auditing. 5. Ever Evolving Standards: As AI byi itself is evolving so will be the standards that govern them. This means the AI auditing institutes must also have resources who literally continuously research on AI systems and it's capabilities to set Standards. This is still theory not sure how it would be materialized. To conclude, I am also intrigued on how the global players are going to come out with an uniform standards that govern AI systems to address the concerns that I mentioned above effectively. Only time will tell.