Q 790. How Should an AI-Infused Process Be Audited? Traditional audit methods focus on inputs, outputs, compliance, and human decision points. But in AI-infused processes - where decisions may be driven by prompts, flows, or machine learning models - the old audit checklist may fall short. What new questions, checkpoints, or risk indicators should be included when auditing a process that includes AI components? How would you ensure transparency, fairness, and alignment with business goals? 

 

The best answer will be selected on the basis of: 

• Relevance and clarity of audit criteria for AI-infused processes  
• Practicality of implementation in real-world settings
• Insight into the unique risks introduced by AI

 

Note for website visitors -

• This platform hosts two weekly questions, one on Monday and the other on Thursday. 
• All previous questions can be found here: https://www.benchmarksixsigma.com/forum/lean-six-sigma-business-excellence-questions/. 
• To participate in the current question, please visit the forum homepage at https://www.benchmarksixsigma.com/forum/.
• The question will be open until Monday or Thursday at 5 PM Indian Standard Time, depending on the launch day. 
• Responses will not be visible until they are reviewed, and only non-plagiarised answers with less than 5-10% plagiarism will be considered for winner selection. 
• If you are unsure about plagiarism, please check your answer using a plagiarism checker tool such as https://smallseotools.com/plagiarism-checker/ before submitting. 
• All correct answers shall be published, and the top-rated answer will be displayed first. The author will receive an honourable mention in our Business Excellence dictionary at https://www.benchmarksixsigma.com/forum/business-excellence-dictionary-glossary/ along with the related term.

• Some people seem to be using AI platforms to find forum answers. This is a risky approach as AI responses are error-prone because our questions are application-oriented (they are never straightforward). Have a look at this funny example - https://www.benchmarksixsigma.com/forum/topic/39458-using-ai-to-respond-to-forum-questions/

• We also use an AI content detector at https://quillbot.com/ai-content-detector. Only answers with less than 45-50% AI-generated content will be considered for winner selection.

Thanks for picking the brain on this very important, relevant and much need of the hour topic. 

Auditing an AI system is very different from the traditional systems audit on various fronts.

Traditional systems audit has a standardized framework widely accepted by various governing bodies across the world. By and large the process knowledge, the skill required , the checks and validations are mostly well defined and standardized across the globe.

But for the AI systems audit, the above is not well defined and does not have an unified globally accepted or mandated requirements governed by a single entity, as it is very dynamic and ever evolving.

I did some research and found that the below are some of the institutes trying to develop frameworks and standards for the AI landscape.

1. National Institute Of Standards and Technology (NIST)- America

2. Institute of Internal Auditors ( IIA)- America

3. International Organisation for standardization ( ISO)IEC 42001) - Independent 

4. BSI (British Standards Institute) - UK

5. European Commission - Europe Union

 

From the above it is evident that not a single institute governs the entirety of AI Auditing framework. Each country (US, UK and Europe Union) has separate institutes to standardise the auditing of AI systems.

In my view, though most of the steps in traditional systems audit is still applicable to AI system audit, the main challenges will lie in the following 

1. Quality of Data : How does one determine if there is bias in the data or not? How does one test the data on bias with which the Machine Learned? As we know the LLM model can use internet for specific searches and collate info based on models , what if the information that is available in the net by itself is bias?? Who knows the truth? This validation can be done only by qualified person who knows the unbiased factual truth. So error is inherent in AI..But what percentage of it is acceptable? Will it be measured by the impact that it creates in the business or the margin of error standardization irrespective of the impact? This is very challenging to decide. So I believe the companies and the qualified auditors have to reach a common point here which is easy said than done.

2. Reviewing a model: This is where the qualifications of the auditors come into picture. To review the AI model one must be an expert in statistics to point out of the results by AI is a flawed one. There are very complex algorithms involved and the time and effort required to decode and test the results will also be a key factor. What kind of sample would you choose ? Is sample relevant for AI auditing ?? How many kinds of adverse questions that can be framed for an AI to answer ?? What key elements should one focus on the adverse or trick questions to assess??.These are few questions I believe are very challenging to answer.

3. Data Governance and Security:

AI can integrate with multiple systems through various interfaces, I even learn that AI can integrate with another AI agent form a different platform, while we can appreciate the capabilities one should also think how safe and secure the data is?. Can it be prone to hack, manipulation etc..I have no answer but I am sure the governing bodies will very soon. Even then it will still be an evolving one.

4. Cost of Audit: Any traditional system audit will not be more than couple of days or utmost a week. But auditing an AI system, since there is no standards the time spent on auditing will remain a debatable topic.More the time, more the money spent by the business on Auditing. I will not be surprised if Companies apportion significant amount of money in the budget just for auditing.

5. Ever Evolving Standards: As AI byi itself is evolving so will be the standards that govern them. This means the AI auditing institutes must also have resources who literally continuously research on AI systems and it's capabilities to set Standards. This is still theory not sure how it would be materialized.

To conclude, I am also intrigued on how the global players are going to come out with an uniform standards that govern AI systems to address the concerns that I mentioned above effectively. Only time will tell.  

 

 

Auditing a process that uses AI needs a big change from how audits are usually done. AI introduces things that are changing, unclear, and flexible, which means we need to think differently, use more criteria, and set new checkpoints. This is a full and useful tutorial that was made to deal with these problems:

1. New standards for reviewing procedures that use AI
a. The model should be easy to read and understand.
Audit checkpoints:
- Can folks who aren't tech-savvy understand and follow what AI says?
- Are SHAP and LIME like simple models used to explain why it made its predictions?
Risk Sign: Black-box models that are hard to understand but have a big effect on business.

b. Points to verify for data integrity and governance:

Audit checkpoints:
- How good is the documentation and usage of data sources?
- Do you routinely examine the quality of your data to see if it is biased or drifting?
Risk Sign: Using datasets from other people without checking them or understanding where they came from.

c. For LLMs, look at the flow and the prompt.

Audit checkpoints:
- Do individuals check prompts on a regular basis to make sure they are safe and work the same way every time?
- Do you check and version prompt flows as you do with code?
Risk Sign: Making important decisions (like investment advice or legal summaries) based on clues that haven't been checked.

d. Checkpoints for the Algorithmic Fairness Audit:

Audit checkpoints:
- Are the results checked for demographic equality, equal opportunity, or other norms of fairness?
- Has the group thought of a way to define "fairness" that works here?
Risk Indicator: Different results for protected groups, but no proof that they were lowered.

e. Checkpoints for Human-in-the-Loop (HITL) Controls:

Audit checkpoints:
- When do you need someone to look at your work, and when can you skip it?
- Do individuals learn how to understand what AI can't do?
Risk Sign: AI takes important decisions without someone reviewing them.

2. Putting it into action in the actual world
a. Framework for Governance
- AI oversight to be added to current risk and control frameworks like COBIT and COSO.
- Give people jobs like data stewards, AI product owners, risk officers, and model auditors.

b. A list of models and prompts
- Write down all the AI parts you have, such as LLM prompts, fine-tuned models, and decision pipelines.
- Add details about the purpose, owners, level of risk, and last validation date.

c. AI Audit Trails
- Keep track of user interactions, model versions, inputs and outputs, and decision scores automatically.
- Make logs that can't be changed and that auditors can see.

d. Revalidation every so often
- Models should be re-audited if they are retrained, altered, or the data distributions change.
- Set up triggers for things like a drop in performance, drift, or changes in the law.

e. Toolkits and automation
- You can use AI Fact-Sheets, Model Cards, and Audit-ML to check that all of your documents and reviews are the same.
- Set up monitoring dashboards to obtain hazard notifications right away.

3. Some risks of AI and how to avoid them
Type of Risk: Make a Plan to Reduce It
- Data Drift Checking data all the time and making new levels of training
- There is bias before and after model fairness testing, as well as during adversarial validation.
- Not clear thinking Add frameworks for AI that can be explained and prompt injection. Cleaning and checking user input immediately
- Don't put too much faith in AI; make sure there are clear guidelines for overrides and HITL checkpoints.
- Not following the rules Check for legality and conformity at every stage of the model's life cycle.

4. Making sure that everything is in line with the goals of the business KPI Mapping: Link AI results to business KPIs like return on investment (ROI) and customer happiness.
- Ethical Guidelines: Use AI in a way that is in line with your company's values and ESG goals.
- Include people from other areas, such risk, compliance, and business, in the model's design and audit.
- Scenario audits assess AI's ability to handle hard situations, like edge cases, stress tests, and other inputs that are meant to be hard for it to handle.

Summary: The audit checklist now has new and significant topics to look for. Description of the model and why it was created

• Checks on the source and quality of the data
• Controls for fast engineering
• Fairness metrics and analysis at the group level
• Watching and logging in real time
• Figuring out who is involved and in charge of what

By adding these AI-specific checkpoints to their audit frameworks, companies can design their AI appropriately while also keeping trust, compliance, and strategic alignment.

Traditional audits will be more suitable for regular processes, however to audit an AI Infused process it would be challenging due to static check points. To audit a process which includes AI components, we would require a modernized & robust mechanism which includes dynamic decision making with evolving logics.

 

The Expanded audit criteria for a process with AI components aligning with Business excellence include:

 

Integrity of Prompt/Flow – Verify whether the prompts, logical decisions are properly version-controlled.

Track Decisions – Is it possible to validate the model, logic or prompts that how the decisions are made by the model.

Regulatory compliance – Verify that the AI models comply with all the required regulatory norms

Bias – Does the models take the datasets & logic removing all implicit biases

Strategic Alignment – Does the models are mapped & aligned with all Business metrics & KPIs

Fragile or outdated model – Verify that the model still providing required outputs with current data or need to be updated.

Escalations – Check the frequency of the errors or any repeated exceptions in the flow of the model

Interpretation – Can all Business stakeholders understand the decisions taken by the model or any need for explanation

 

Considerable risk factors:

·         Accuracy of the model goes down and lost its effectiveness

·         Logical flows not matching / aligned with change in business

·         Inappropriate mechanism to report issues / improvement suggestions

·         Outdated knowledge base feed into AI models

 

To ensure Transparency, Fairness and alignment with Business Goals:

·         Frequent review of modes along with owners, data teams & the users

·         Maintain a clear version history of all the changes

·         Create surveys, dashboards and track all override logs to align with KPIs

·         Validate the models to understand the value it creates against the mapped metrics through different tools.

 

Implementing the same in real-time scenario requires:

·         Templates with weighted scores across different criteria’s

·         Dashboard / scorecard to have a better tracking & alignment with Business goals

·         Train resources to include additional criterias for autiding

As a former Quality Management System auditor and QMR (Quality Management Representative), I would still rely on ISO standard to facilitate audit on AI-infused processes. ISO provides internationally and consensus-driven management system guideline from strategic down to operational level that span to multiple sectors, not just industry.

 

 Introducing ISO 42001:2023, this new standard focuses on building trustworthy, transparent, and risk-aware AI system supported by good governance, responsible leadership, and continual improvement. This is also known as Artificial Intelligence Management System (AIMS).

 

Like any other ISO standard, AIMS is structured to facilitate thought-provoking questions such as but not limited to:

 

“How to monitor performance and ensure AI behavior aligns with intended purpose?”

“What are the requirements in conducting internal audit of AI Management System?”

“When and how to address non-conformities and take corrective actions?”

“What are the AI Lifecycle stages?”

 

These guide questions were embedded in ISO 42001:2023’s clauses to steer AI solution experts and management alike in establishing a fit-for-purpose management system manual.

 

In addition to ISO, organizations can also refer to NIST AI Risk Framework. NIST is a U.S. government agency that develops standards, guidelines, best practices, and measurement techniques with primary focus on Cybersecurity, IT, and metrology.  AI professionals and management alike can refer and use this framework which was designed to help organizations across industries and domains to manage risks of AI systems and improve their trustworthiness.

 

 

Audit of AI infused process is important to ensure compliance and accountability, since it verify that AI systems function or operate as per the regulatory requirement, and uphold transparency and public trust.

 

Followings are the key points that should considered during the audit:

Ø  Goal & Scope: It is important that goal and scope should be clearly defined. AI audit is very important to ensure effective examination of the system’s performance, compliance, and standards of ethics.

Ø  In-depth data and algorithm evaluations: It is important to confirm the accuracy and integrity of data and to rigorously evaluate algorithms in order to ensure the accurate functionality and uphold principles of fairness.

Ø  Implementation of continuous improvement: Action to be taken on audit findings and implement regular monitoring are critical step to maintain compliance and consistently improve the perfoirmance and reliability of AI System.

Followings are the main area’s for an AI Audit:

Ø  Define the audit scope by determining which AI system will be evaluated and identifying the specific component or process to be examined.  Establishment of clear objectives, evaluation criteria, and success metrics to ensure alignment with organizational goals and regulatory requirements.

Ø  Collection of relevant data related with the AI system i.e. input datasets, documentation related to algorithm, and output results. Make sure that the data is cleaned and validated to facilitate accurate and effective analysis.

Ø  In order to check whether the AI model function correctly and same are free from errors algorithms review play crucial role.

Ø  It is important to verify the AI systems in order to ensure that model comply with relevant regulations and standards, such as GDPR and CCPA.

Below outlined checkpoints should considered while auditing AI component:

Category	Checklist Item
Governance & Accountability	Is there a designated owner for the AI system?
	Have the role and responsibilities been clearly defined and communicated ?
Data Quality & Management	Has the source of the training data been properly documented ?
	Has the data undergone through evaluation to ensure that it is free from biasness?
	Is the data thoroughly tracked from its original source or point of origin?
	Are data privacy and consent requirements are met with the policy?
	Has the model been validated using the data set that are both diverse and representative of use case study
Explainability & Transparency	Can non-technical stakeholders can understand the decision of AI model’s ?
	Are prompts and flows (for LLMs) are accurate?
Security & Robustness	Have appropriate controls been implemented to protect AI System from adversarial attacks?
Compliance & Ethics	Is there a formalized process in place to conduct for ethical review of AI use cases?
	Are audit logs and documentation retained for compliance?
Business Alignment	Are the goals of AI initiatives aligned with the objectives of business
	Are Key Performance Indicators (KPIs) clearly defined and actively monitored to assess the performance of AI system?

 

In order to maintain transparency, fairness, and alignment with business goals, following approaches should considered:

 

1)       Transparency:

·         Maintain audit trails for data, models, and decisions.

·         Use model documentation tools (e.g., Model Card).

·         Provide user-facing explanations for AI decisions.

 

 

2)       Fairness:

·         Ensure that AI systems consistently uphold principles of fairness by avoiding discriminatory outcomes.

·         Conduct bias periodic audits regularly.

·         Engage diverse group of stakeholders throughout the model design and testing phase to ensure inclusivity, fairness, and broader perspective in decision-making

·         Implement feedback loops to catch and correct unfair outcomes.

3)       Alignment with Business:

·         Define clear KPIs for AI performance that align with business goals.

·         Ensure CFT collaboration 

·         Use AI governance frameworks

 

In the realm of medical coding, where transparency, compliance, and alignment with business objectives are paramount, the audit framework known as "RAISE" is tailored for processes enhanced by artificial intelligence.

 

The RAISE framework serves to assess the Reliability, Accountability, Impact, Security, and Ethics of AI systems within practical workflows.

 

Reliability

Is the AI system capable of consistently delivering reliable and accurate results, even in unusual circumstances?

 

Accountability

Is it possible to trace the origins of decisions, figuring out if they come from AI, a medical coder, or a system rule?

 

Impact on Business and Compliance

Does the AI provide results that comply with regulatory norms and fulfill ROI objectives?

 

Security

Are the components of the AI protected, allowing for regulated access and eliminating the possibility of misuse or unintentional data disclosure?

 

Ethics

Do AI systems make decisions that are fair, unbiased, and grounded in ethical principles?

 

Checkpoints:

We can determine if accuracy declines over time.
We can maintain a log that lists responsibilities and specifies who is responsible for AI malfunctions.
We can monitor regularly whether AI is compliant with the UHDDS standard, POA, and HAC regulations
We can follow up and make sure who has the authority to see or alter AI models, prompts, or outcomes, as indicated in the access logs
We can check whether the system explain the reason why it recommends a specific code, diagnosis, or DRG.

 

Risk:

The main risk is there is possibility of AI making high-impact decisions without any transparency.

 

Thus, RAISE helps us to keep AI aligned with both patient care and business outcomes.

 

Traditional audit methods emphasize procedural adherence of the process (Inputs, Outputs, Compliance). At times, it also audits human accountability. However, AI systems introduce dynamic, opaque, and non-deterministic elements that require a broader and more adaptive audit framework. However, this technique will not be more appropriate as decisions taken by AI are based on inputs given to the program.

 

Expanded Audit Criteria for AI-Infused Processes:

Model Design & Development: this is to understand flawed or biased training data can lead to systemic errors or unfair outcomes.

• Are the AI models trained on high-quality data?
• Did the model documentation covers the assumptions and limitations?
• Is the purpose of the AI model aligned with business objectives?

Decision Traceability: One thing that stood out — we’ve got to be able to explain how AI is making decisions, especially for anything high-stakes.

• Can we walk someone through the logic or input that led to a particular output?
• Are we keeping a log of changes to prompts or how the workflow is set up? That’ll be important later.

Fairness, Bias & Ethics: AI can behave oddly if the training data’s off. We don’t want something that treats one group unfairly without us even realizing.

• Have we checked if the model is biased — like giving different outcomes for different user groups?
• Is anyone reviewing this regularly, or is it a one-time test and forget?

Monitoring & Drift: AI isn’t static. Even if it works well today, it might not tomorrow.

• Are we keeping tabs on how it’s performing week over week or month over month?
• If things start drifting or it gives unexpected results, who jumps in and fixes it?

Human Oversight: Automation’s great — but we still need people in control.

• Do we have any control points where someone can step in and override a weird decision?
• Are users confident about when to trust the AI and when to double-check?

Some Observations in my Experience:

• Results that are unpredictable or inconsistent
• No one taking ownership when something goes wrong
• System changes happening but not being recorded
• Unclear governance, Details of responsible SPOC
• Full reliance on a tool, people response that they did not build and don’t understand fully

Real-World Audit Tips

I. Build on What We Already Do

We dont need a brand-new process. Just add sections to current audit checklists by looking at bias, model changes, decision logs, etc.

 

II. Assign a Go-To Person or Small Team

Someone needs to own AI governance. Ideally, someone who understands the tech but can also speak business risk.

 

III. Push the System — Hard

Try edge cases. Throw in unexpected inputs. See how the model reacts. We learn a lot from how it handles the “weird stuff.”

 

IV. Keep a Short, Clear Record for Each Model

Doesn’t need to be fancy. Just include the model's purpose, last test date, any known issues, and how often it gets reviewed or updated.

 

Make Sure It’s Transparent, Fair, and Aligned

• Transparent: Can someone outside the tech team follow what happened and why?
• Fair: Regular bias checks — not just one-and-done
• Aligned: Is the AI helping meet business goals or just running because “we have it”?

At the end of the day, the check points is not just about the technology. We need to be able to establish: “This is what the system did, here’s why, and we are confident it was the right call — or we fixed it if it was not.”

That’s true today; traditional auditing methods primarily focus on static inputs and outputs, as well as human decision points. However, when we discuss incorporating AI into business, we need to think beyond traditional methods. When we introduce AI into the mix, we need to consider its impact on business processes. We need to shift from static checklists to dynamic checklists, as the very nature of an AI-infused system is to evolve; so, our auditing should also evolve.

The following criteria should be considered when auditing the AI process.

1-      The quality of data is crucial, as it should be dynamic, and we continually ingest high-quality data.

2-      The performance of the process is increasing or decreasing.

3-      The process is handling the intent.

4-      Provide improved prompt templates and regularly test them.

5-      Continue testing new real-world cases, looking for new edge cases.

6-      Keep on updating documentation and SOPs.

7-      Revisit training data sources for accuracy and completeness.

8-      Evaluate prompt engineering regularly for safety.

9-      Keep feedback in the loop when AI outputs hallucinate.

10- Audit of change logs and tracking of model updates.

11- Does AI generate outputs following a logical path or in response to a prompt?

12- Is the system compliant with regulatory requirements, including HIPAA and GDPR, as well as ethical standards?

13- Does the system support business goals, e.g., customer satisfaction and revenue growth?

 

Practical Implementation of Controls to Ensure Sustainability.

1-      Firstly, testing should be comprehensive, using both scenarios and real-world examples.

2-      Continue to monitor the quality of outputs, user feedback, and hallucinations.

3-      Implement CI/CD for proper checkpoints and change management.

4-      Always have up-to-date documentation, version control, and traceability of prompts.

5-      Review of ethics and compliance regularly.

6-      Include SMEs from cross-functional teams, e.g., data scientists, legal teams, and ethical champions.

 

Due to the changing or dynamic nature of AI systems, it poses unique challenges and risks.

1-      Decision outcomes are often unclear, and unintended consequences can lead to discriminatory practices.

2-      AI systems are overly reliant on LLMs without a human in the loop.

3-      Low-quality LLM prompts can result in malicious manipulation.

4-      As it relies on the quality ingestion of data to LLM, performance will degrade if retraining is irregular.

 

Conclusion.

As mentioned earlier, an AI system audit should be dynamic and introspective, examining why AI did X instead of Y.

If we want our AI system to evolve with new real-world challenges while being safe, ethically aligned, and aligned with business goals, we should keep humans in the loop, maintain transparency, and closely monitor the system from day one.

 

From what I have read on this topic, most companies are going to more interested in how we implement AI governance (understand regulatory frameworks), spot and manage AI risks, ensure ethical AI behavior, build explainability and transparency into our models, and the being able to track the AI compliance with precision.

 

Regulatory Framework: No what policies that any AI solution must abide.  This mean, in the USA, Federal, State, and of course, corporate policy.   “Operationalizing” these laws means conducting privacy impact assessments, documenting AI lifecycle processes, ensuring human oversight, and keeping up to date as regulations evolve.

 

Some ways in which we can develop and enforce these policies into our organizations would be: 1) building cross-functional AI governance committee, 2) define ethical principles and acceptable AI use cases, 3) establish an internal review process for approving model, 4) integrating checkpoints (e.g., bias testing, data review, etc..) into the development process or workflow, and 5) provide organizational-wide training on AI compliance.

 

Risk management needs to always be in the forefront of any AI solution we create.  Creating the solution is not the difficult part.  Managing the risks will be and it takes a lot of due diligence, hard work and strict compliance.  Some of the tools we can use are 1) regular audits and compliance checkpoints, 2) thorough documentation of how the model was trained and how it performed, 3) implementing some 3 party monitoring tools to detect drift and anomalies, 4) embedding risk reviews into the agile development workflows, 5) collaborating early with the legal and compliance teams, and 6) designing a sandbox environment for ethical experimentation.

 

Ethical considerations need to be evaluated along with addressing potential ethical conflicts.  Some of the key considerations are 1) avoiding harm to both the company and the customer, 2) promoting fairness, 3) preventing the misuse or overreach of the AI solution.

 

Some of the documentation and reporting that can be produced to demonstrate AI compliance are 1) model cards (describing model intent, limitations, and metrics), 2) datasheets for datasets (providing detail such as origin, composition and bias review)

First of the the AI infused process should be audited for its customer centric solution as to if it is able to provide the solution to the customer requirements without human interventions.

Second part of the audit should be the scenarios which are unpredictable and can lead to incomplete cycle or solution provision for which the AI has been created.

The most challenging part is to draft or provide solution for all the possible scenarios and later on it can be identified during the Audit process.

When auditing AI-integrated systems, transparency on how the model works, the security of the data, and whether the prompt logic system is working properly all need to be evaluated. AI systems should be evaluated on whether their decisions are traceable, if there are risks of bias, and if the system meets ethical standards. Examining the sources of the training data, monitoring model drift, and examining output consistency with given prompts constitute critical tasks. Employing accountability-enhancing tools such as explainability mechanisms, inline controls, and collaborative processes bolsters overall accountability. We will ensure proper and responsible use of the systems through regular alignment checks with the overarching business goals.

For a AI infused processes there is a need to shift from traditional audit approach to a more dynamic audit approach which can models, prompts, flows. Since AI uses these approach for its BAU working so audits should be more focused on data integrity, accountability, ethical compliance.

Below mentioned are the approaches that can be used while building AI audit framework:

 

1. Prompts and Flow Designs

• Questions to ask - Are they documented and have a version control mechanism in place? How is the ambiguity handled?
• Check Items - Prompt repository, Flow Diagrams for execution.

2. Model Governance & Lifecycle Monitoring

• Questions to ask - What type of AI model is used? Who trained the model and what type of data is used? Is there any documents and updation policy?
• Check Items - Model validation, Testing, Drift detection, data revision mechanism, Schedules.

3. Data Integrity & Bias 

• Questions to ask - Is data input clean? Are there any detection system for detecting Bias?
• Check Items - Bias Audits, Data steps to be followed, data lineages.

4. Compliance & Ethical Alignment

• Questions to ask - Does AI comply with regulatory guidelines set up a board? Are guidelines embedded in the design and the system
• Check Items - Regulatory mapping, gap analysis,.

5. Human In Loop

• Questions to ask - Are there any human decision points integrated? is there any recovery mechanism if AI fails?
• Check Items - HITL workflows and paths to be checked. Feedback Loops.

Below mentioned are the best practices for ensuring fairness and transparency:

1. Documentation of AI processes should be mandate.

2. Continuous monitoring for drift, bias, performance for the AI model.

3. Proper aligned communication strategy with stakeholders and business objectives.

4. AI audits at a set frequency.

 

 

For an audit, there will be an audit checklist. The audit checklist is key to an audit process.  It does not matter whether its for a traditional process or an AI-infused process.

Having said that lets quickly see the response for these questions :                                                                                             

What new questions, checkpoints, or risk indicators should be included when auditing a process that includes AI components?

 

1.  Do all the AI components serve the intended purpose

2. Do all AI components adhere to the Data governance & data privacy policies and procedures (Risk indicator)

3. Do the AI components adhere to the Safety/Security regulations (Risk indicator)

4. Do all AI components are easy to diagnose/dissect if there are any issues to fix

5. Do we have the right metrics to measure the performance of the AI components (checkpoint)

6. Do we have a benchmark/threshold value(against the traditional way of doing the process) defined for the AI components as how effectively deliver in terms of timeliness, providing quality output/outcome etc..

7. Do the AI components as any ethical concerns (Risk indicator)

8. How much AI knowledge the auditor has, to audit an AI infused process – based on this only, the audit checklist can vary and the effectiveness of the audit itself will rely upon (Risk indicator)

   

For instance, if I want to create an AI systems for an insurance company for creating an insurance policy , I may need to have several AI agents. May have an Agent for underwriting work, An Agent for Premium, An Agent for collecting user (customer) data and an agent for Policy creation. Now I need to check if all agents serve its intended purpose and also all agents should integrate and work in unison so as to get a policy created properly and in a timely manner. The AI agents also need to ensure that the regulatory rules/laws and compliance to GDPR laws of the region/country should be captured as of on the date of policy creation. They also should focus on any ethical concerns that might pop out as part of this process. These are potential area of risks.

 

By ensuring the above audit checklist questions , checkpoints and risk indicators are addressed, we can ensure that this Insurance Policy Creation process can be smoothly done.

    

How would you ensure transparency, fairness, and alignment with business goals?

1. With the help of a visual radiating tools like Kanban boards, display the state of each AI-infused process, to the stakeholders

2. As we deal with AI infused processes, the BRD plays an important role here. We should have the goals, objectives clearly articulated and communicated to all the stakeholders

3. Periodic feedback of the AI-infused processes will provide a overview of each such process and comparing with the BRD can point out whether the current state of the processes are in alignment with their business goals

 

Conclusion:

In my perspective for AI infused processes, the fundamental difference in auditing is the AI knowledge possessed by the auditor.

 

IMHO, an ideal way to do an audit this type of AI-infused processes is to combine human with AI.  So when an audit happens for AI-infused processes, the audit checklist can be fed to an AI Audit agent which can handle that and which can self-learn over a period of time (inspect and adapt) and then that can be overseen by a human auditor.  This way the audit can be more effective while dealing with this AI-infused processes.

Now a days with the introduction of the Artificial Intelligence in Business processes has made the need of the Audit for the AI infused process more and more relevant.

The significance of Audit in the Business processes has always been there as the adherence to the standard operating process in normal audit process is checked and this is the way Organizations re able to maintain the standardization in the process.

The problem came when we introduced the Generative AI in our existing Business process of Sales and Marketing where we had aimed to utilize the AI abilities to read through multiple previous executed RFQ and RFPs so that the previous data and pricing points can be utilized.

When we had onboarded the Customer relationship management tool, Our Sales & marketing team had specific need to make the process for RFP submission faster and immediate need to include the Agentic AI capabilities of the tool to help them with identify past data points from any of the submitted RFPs and to help them analyze the failure point (of not receiving the order from client as winning bid).

Also, Gen AI was supposed to help the design team in reading through the specification documents and help in identifying the Client requirements so that design team can also ascertain the dimensions and hence the materials requirement for the SCM team to arrive at the BOM quantity and price.

The Audit team had this task of revisiting their Audit approach in which they had traditionally attempted the Audit processes of identification of the SOPs and then validating through random data points and requesting for the validating documentations.

The Audit team had to first understand the AI working, how does a AI model is prepared, its testing procedure and then validation procedure before which the AI capabilities are ready to be used by different users and then the revisited audit process of the Sales and Marketing business process enabled with AI capabilities needed to be revisited.

The Audit team was able to come up with the revised Audit process in which it had split its scope in four different categories:

1. Business process without AI
2. Business process with AI
3. AI process/suggestions/prompt
4. AI model and data source build ups

1. The First category was being audited with the help of manuals and SOPs.

2. The second category needed to test the reply suggested by AI for the prompts which were asked by the team members. e.g. The prompt entered by team members like 'Suggest me similar type of RFP submitted earlier with profit margin less than 5% in the total bid of 50 lakhs?'
The Audit team needed to check the Reponses suggested by the AI tool with the database and cross check and verify if the AI reposes are in line with the database and providing the same reply which could have been provided otherwise by manual searches (may be time taken would have been a tad longer ~10x).

3. Audit team also used their own prompts to check on the responses and used to validate wrt the standard documents, organizations guidelines, SOPs, user manuals, Companies Policy etc.

4. The most important parameters of testing needed to check the AI model and data sources and data validation, biasness in the system and AI ethical practices.

Audit had taken help of Chief Information officer to identify the right person from Information system who could validate the AI model as to how the model has been structured, has the model been in built with certain biases in terms of socio political Reponses to be filtered out as per the country of jurisdiction or is independent, whether the responses are biased or not and was providing scores (scoring methodology already defined) and then the team used to arrive at the individual rating for the four sub processes and then the final score of the Audit process.
Based on the Audit score there were three categorize of satisfied, needs improvement and needs immediate attention and improvement.

All the recommendations by the Audit team were explained in details so that the overall process adherence remains high across organizations including AI infused processes.

The Audit team was confident that even though they have put in a place a process to Audit the AI infused process but still this will be a continuous improvement  process as the AI model itself keeps on evolving through various developments day by day.

All the answers are worth reading. Some points might appear repetitive but there are some unique points that will appear in different answers. Answers from Yuvaraj and Jess are a must read.

 

Sumukha has provided the best answer to this question. Well Done!

 

My 2 cents - "Audit by Design" is a good approach to solve for this problem.