Skip to content
View in the app

A better way to browse. Learn more.

Benchmark Six Sigma Forum

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

Venessa Laval

Members
  • Joined

  • Last visited

  1. Can AI Truly Be Creative — or Does It Just Remix Human Ideas? I've been thinking about this a lot, especially in the context of building a unified venue assessment framework for 110+ examination venues across 7 regions. This isn't a theoretical question for me—it's directly relevant to how I approach a real problem: synthesizing border agency security requirements, awarding body invigilation standards, and country office operational realities into a single framework. There's no template for this. The stakeholders pull in different directions. The venues operate under radically different constraints. So what would AI actually do differently than what I do? And does that difference matter? What AI Pattern Synthesis Actually Looks Like If I asked an AI system to design the unified framework, here's what would happen: It would extract patterns from existing audit standards, synthesize the standard compliance checkpoints (CCTV requirements, staff supervision, materials custody), recommend uniform assessment tiers, and generate scalable templates. Cost-efficient. Comprehensive-looking. That's useful. It's also not creative in the way the problem requires. How I Solve Problems Across Competing Stakeholders My approach would be different in specific ways: Constraint reframing: I recognize when a standard framework has miscategorized a problem. A venue in Nigeria can't meet Italian CCTV infrastructure standards. But that's the wrong question. The real question is: does it achieve the security outcome through locally feasible means? Reframing shifts everything about what solutions become visible. Contextual judgment: I carry organizational memory that doesn't exist in data systems. I know which venue operators are genuinely reliable versus theoretically compliant. I understand that when Turkey's political instability affects business costs and staff availability, that's different from operational failure—a distinction that requires knowing the context, not pattern-matching the data. Value synthesis under conflict: When border agency security requirements, awarding body continuity needs, and country office constraints pull in different directions, I don't pick one constraint category to optimize. I create solutions that meet multiple objectives simultaneously—sometimes at the cost of standard efficiency. Owning the decision: The framework I create isn't "what the data suggests." It reflects my judgment about what matters, what's sustainable, and what risks I'm willing to bear. That's different from synthesis. The Turkey Case: Where the Difference Actually Matters Let me walk you through a real example. A venue's compliance costs spiked 40%. Here's how pattern-matching works: When an AI system flags high costs, it would recommend finding a cheaper alternative. The logic is straightforward: high costs equal risk, so seek a better option. Here's what I did: I investigated why costs spiked. Turned out it wasn't the venue operator failing—it was local political instability affecting site access and staff availability. That's a different problem entirely. Not "this venue is unreliable," but "this venue is reliable in an unstable environment." Then I asked the actual questions: What does border agency security need? Continuous exam delivery. What does the country office need? A venue that meets the standard requirements. What do I know about this operator? That they're dependable when things are hard, not easy. So I created something that doesn't exist in any standard framework: temporary fee adjustment + enhanced oversight protocol + timeline for stabilization. I'm paying more for reduced efficiency right now because I'm gaining continuity and preserving a critical relationship. I'm anticipating that political conditions stabilize within the timeframe I've set. That decision isn't in the audit standards. I invented it because the situation demanded it. And I own it—I'm the one defending it to border agencies and awarding bodies if things go sideways. Why this matters: The AI recommendation was logical but wrong. It treated "high cost" as the actual problem when the actual problem was "temporary external shock affecting a reliable operator." Once you see it that way, the solution space completely changes. What This Means for AI in Complex Ecosystems AI can do specific things well. It extracts patterns from existing frameworks, synthesizes standard compliance checkpoints, generates scalable templates, flags anomalies. All valuable. None of it requires creativity in the way I'm describing it. What AI struggles with: reframing what the problem is. Carrying contextual judgment built on months of audit work and relationship history. Making value commitments about what matters when objectives conflict. Bearing responsibility for novel decisions in ambiguous, high-stakes contexts. Those aren't limitations of current AI systems. They're structural. Reframing requires recognizing what standard frameworks miss. Contextual judgment requires experience. Value commitments require skin in the game. Responsibility requires consequence. Building the Unified Framework The unified venue assessment framework isn't just synthesizing border agency requirements + awarding body standards + country office operations. It's creating new categories that none of them recognize: Context-aware risk thresholds that vary by geography, political stability, and infrastructure reality. Venue reliability profiles that override standard cost metrics. Temporary operational adjustments as a legitimate framework category instead of exceptions to rules. Stakeholder conflict resolution protocols that acknowledge when standard frameworks contradict each other. These categories exist because I recognized what standard frameworks miss. That's the creative part. The framework becomes genuinely useful not because it's pure invention—it builds on everything that exists—but because it changes what categories of solution are even visible or permissible. That's constraint reframing. And that requires human judgment rooted in context, stakes, and responsibility. AI can help with the synthesis. But the framework itself? That's mine.
  2. Our examination administration ecosystem operates in 110+ venues across 7 regions which must simultaneously satisfy border agencies, awarding bodies, and country office managers operating under conflicting standards, cost pressures, and regulatory expectations. For example, a single venue in either India, China, Nigeria, Canada, Italy, Turkey might need to meet border agency identity verification procedures, comply with an awarding body's invigilation standards and work with operational challenges that a local country office manager will understand but not necessarily the external regulator. These different countries have also different fraud perception index and different operational infrastructure constraints. Right now, our function uses human judgement, observation to consolidate and translate between these competing requirements using a lot of engagement to satisfy multiple stakeholders simultaneously. For example, a human reviewer notices that Venue C in Turkey's compliance cost spike correlates with new local political instability—context an algorithm would flag as 'cost inefficiency' without understanding the why. An algorithm might recommend de-selection; a human might recommend temporary fee adjustment. If we were to have an AI to AI collaboration negotiating directly and reshaping these 6 diverse context: Competition dynamics; venues become algorithmic bidders. If autonomous systems start dynamically selecting venues, adjusting compliance requirements, and negotiating fees in real-time, venues enter a new competitive dynamic. Pricing dynamics: If exam fees or venue site fees are negotiated between organizational systems instead of human discussions/relationships, pricing become real time and dynamic. Exam fees, venue rental rates, and invigilator costs become subject to real-time algorithmic negotiation between awarding bodies, border agencies, and venue operators. The AI could converge on similar pricing in each geographies, with standardised pricing with elimination of competition and lack of negotiating powers from the venues because every awarding organisation algorithm quotes the same fee. Geographic competition distortion: If the algorithms assess "acceptable risk" differently across regions (because they're optimizing for different cost structures), venues in some countries could face systematically higher compliance burdens, reducing competitive access to exam delivery. Accountability in fragmented jurisdictions: When a security incident occurs at a venue in Turkey, and the compliance decision came from algorithm negotiation between an Indian awarding body system, a border agency system, and a local Country Office system—who is liable? The audit trail dissolves across jurisdictions. The trade-off: Of course, AI-to-AI coordination could optimize exam delivery efficiency—faster venue selection, cost reduction, better resource allocation across 110+ venues. But these efficiency gains come at a governance cost: the loss of human judgment about operational context, the risk of algorithmic tacit collusion on pricing, and the accountability gaps across fragmented jurisdictions. The ethical dilemma is stark: whether faster algorithmic coordination is worth the opacity and potential market capture it enables. The governance gap intensifies here: We are not managing algorithmic coordination within a single regulatory framework (like a domestic supply chain). We are managing it across 7 regions with potentially conflicting legal authorities, different data protection regimes, different audit standards. There's no supra-regional authority that can monitor whether algorithmic systems have implicitly colluded or prioritized certain geographies over others. Our safeguard proposal becomes jurisdiction-aware: Rather than a single governance checkpoint, we would need layered algorithmic transparency: At the individual venue level: algorithmic recommendations flagged for human review when they would increase compliance costs by >15% in any single geography, or when fee quotes from independent awarding body systems converge to within 10% variance in the same geography. At the regional level: quarterly analysis of whether algorithmic decisions show evidence of systematic bias toward or against certain geographies At the multi-region level: cross-border audit trail showing where algorithm recommendations from different stakeholder systems diverged, and how humans reconciled them The examination administration ecosystem is uniquely positioned to be an early test case. Unlike abstract AI governance discussions, exam administration involves real security stakes, fragmented authority, and measurable market distortions. The governance framework we design now—for algorithmic coordination across fragmented jurisdictions—will determine whether AI-to-AI collaboration creates market efficiency or market capture.
  3. Can AI Systems from Different Companies Collaborate Effectively? Context We work with awarding bodies across 7 regions for delivery and part of the process when selecting new venues or maintaining the delivery at existing venues is to approve them. We currently share audit findings via email We have a duty of care to customers for venue safety Scenario 1: What if our audit AI and the awarding bodies approval AI collaborated directly? Our audit AI could potentially identify venue security gaps or compliance issues across regions. The external awarding bodies’ AI then evaluates whether that venue meets their standards for running exams. Both systems need to share information to decide whether this venue would be approved or not. Both systems flag risks and assessments in real-time For instance, we had in a country in Southern Africa cluster a venue with inadequate fire exits. If we were to use AI, it would flag venue not approved due to inadequate fire exits. In the hypothetical case the awarding organization AI would say approve for delivery on the other hand, in case of a fire outbreak with victims hurt, we might face legal issues because the venue has been deemed as accepted but we didn’t actually have the authority to reject it. There could also be a scenario where the awarding organization would be sued because they approved it, but the liability could be shared as well. There is therefore misalignment in power and responsibility. In this current scenario, the awarding organization has regulatory authority because it officially approves or rejects venues, they have the power to make the final decision as they are the gatekeeper. On the other hand, we are responsible for our customers’ safety at the venue. If something goes wrong, the customer will sue us not the awarding organization. We have liability. The Opportunities: Speed: Instead of email correspondence, we can have real time exchanges on venue approval which would enable troubleshooting in case of serious issues Consistency: We will be able to implement a standardized approach across the 7 regions Better decisions: Both AIs are informing and collaborating with each other with a specific focus (our security, quality and compliance focus + their standards) Reduced administrative workload: Less administrative time on email coordination between the operations manager, regional manager and the awarding body. Earlier Risk Detection: This will enable real time monitoring and proactive detection instead of reactive reporting. Risks and Governance Challenges: Risks: Liability when the systems disagree: When they disagree on venue approval, who decides? If our AI flags a risk they ignore (or vice versa), who would be liable in case of major incidents/investigations? Authority Conflict: Should their AI override our assessment, or vice versa? We would need to protect our organization and ensure that the awarding body's AI doesn’t overrides our audit findings. We have a duty of care to our customers, and we need to ensure a clear line of accountability and authority when systems disagree. Data privacy breach: What compliance data can we share with them without breaching confidentiality and data privacy of customers as only specific customers for that awarding body would apply Misalignment of standards: Given that some of our standards in our frameworks are enhanced for integrity purposes, we would need to have a minimum of shared standards and understanding Loss of control: The AI of the awarding body making decisions that can compromise venue safety Governance Mechanisms to address those risks: Data Privacy: We need to engage with our GDPR team to decide which type of data we can share with the awarding body’s AI. We would need to have data agreements and business rule to ensure that only the relevant details are shared. We decide which section is confidential for our organization and the same will apply for the awarding body. Authority conflict: Our duty of care to customers means we can't let their AI override ours on safety, but they have regulatory authority. So, collaboration only works if we pre-agree on authority boundaries. For example, if our AI flags a health and safety gap and their AI approves the venue, we escalate to humans and override if needed, documenting the decision. This requires clear escalation rules, shared understanding of authority, audit trails, and pre-agreed governance. Standards misalignment: We can share our framework as this is visible in any case when sharing the report findings. However, we will need to agree on what is critical and minor gap. At times our standards are stricter than theirs, we would then explain the rationale in terms of this enhancement and see if they agree. The heads of assurance from both organizations should validate the standards alignment. Liability: In case of escalation, we need to have a documented process to ensure that we have acted responsibly in good faith. The risk assessments would be key and the rationale on the rejection. Loss of control: It is highly unlikely that we will be able to override their AI each time in case of disagreement on venue safety. There would need to be a mechanism on veto rights (escalation and documentation) once the risk is flagged. We would need to have a message notification should their AI override and approve the venue despite the concerns raised. We would need to engage also with our legal team to protect our reputation for the override possibilities if the regulatory authority kicks in.
  4. We have a global security standard on the security of materials whereby they need to be stored in a locked secure room with CCTV coverage and dual key access controls. Simple enough you would say, however, I had three different scenarios which showed that transparency was the whole issue: the right information needed to be shared with the right stakeholders. The first scenario revealed what worked. When a Country Office Manager in Region A pushed back on implementing dual-key custody, I initially saw resistance. But when I shared the specific audit evidence (materials left unsupervised in a similar region, which created an opening for unauthorized access) her response shifted entirely. She didn't argue; instead, she identified a genuine constraint in her region (insufficient trained personnel) and proposed a compensating control (split-location storage with role-based access) that was stronger than the standard. The transparency about risk evidence didn't produce compliance; it produced collaboration. This success led me to encourage team motivation and buy in by sharing cross region comparative data internally. I began sharing performance benchmarking across regions—which regions were meeting the dual-key standard, which ones were struggling, and which ones had adapted. I expected this to drive improvement. Instead, weaker-performing regions became defensive. They didn't engage with the underlying risk evidence; they argued that comparisons were unfair, that their region had different constraints, that the standard was arbitrary for them specifically. Adaptation requests shifted from "here's our genuine constraint and proposed solution" to "everyone else is just better resourced." I gained visibility into performance gaps but lost the quality of decision-making. When I stopped sharing comparative data, adaptation requests returned to being grounded in actual constraints rather than competitive positioning. The third scenario showed me what happens when both kinds of transparency fail. A Country Office Manager was told to implement a standard without context and without any pathway to propose adaptation. No audit evidence explained why the standard mattered. No acknowledgment that her region's constraints might be different. Just: implement this. She pushed back, felt unheard, escalated the issue. The solution wasn't more transparency—it was the right transparency. When I shared the reasoning behind the standard and created a formal process for her to propose compensating controls, the escalation resolved. Collaboration returned. It was therefore clear that transparency about reasoning and risk evidence enables better decisions. Transparency about comparative performance creates defensiveness. Absence of transparency about the 'why' produces escalation. This is true whether decisions are made by humans or by AI systems. In fact, the introduction of an AI system evaluating regional adaptations would make these transparency choices even more critical, because an algorithm can amplify both the benefits and harms each decision. When an AI system enters this environment, evaluating adaptation requests and flagging risks, the transparency choices become even more critical. For instance, if a lower-resource region requested to replace continuous CCTV with enhanced access logging and weekly administrative review, I would face a transparency decision. The AI system could evaluate whether this compensating control meets the risk thresholds, but I would need to decide: should the regional manager see the AI's risk assessment score? Should external stakeholders be told about the adaptation? The global team needs to see the AI's risk assessment and reasoning as evidence. The team would decide if an adaptation is acceptable. For example they would need the full picture: what happened in similar regions, what is the likelihood and impact if we were to approve this adaptation, and whether the compensating control is genuinely adequate. It is not necessary to see at this stage vendor costs or regional budget constraints; these are management decisions, not standards decisions. Regional managers need to know the "why" behind standards and have a clear pathway to request adaptation. When I've shared audit evidence (why the standard exist) managers move from resistance to collaboration. They need to know decision authority (can they adapt autonomously?) and consequences (will external stakeholders be told?). They should not see performance comparisons with other regions; my second scenario taught me that triggers defensiveness, not problem-solving. External stakeholders need transparency about the adaptation itself, not the metrics. They need to know: Region X operates under documented alternative controls due to infrastructure constraints; compensating controls are specific measures. This gives them assurance the gap is known and managed, not hidden. They don't need to see the AI's risk score. The AI system would flag the risks and surface the reasoning to the right stakeholder. It wouldn't hide risk signals and share numerical scores with audiences who will misinterpret them. It would also require human review before approving any adaptation. The algorithm identifies candidates and surfaces evidence, but humans decide because in an interdependent ecosystem, precedent affects everyone. Transparency becomes a design choice, not an afterthought. When you get it right, collaboration across regions is amplified. When it's wrong, escalation is amplified.
  5. At the beginning of the month, a whistleblowing malpractice was reported for one of our operations in West Africa. Anticipating the reporting assurance, I reviewed 3 different reports across our own audit system, regional powerpoint presentation and data captured in excel sheets to check whether this risk had been highlighted. AI would be useful to link and summarise this cross over information enabling me to easily take the decision and make recommendations. There are 3 different areas that will need focus to enable me to lead differently: 1. Enhancing Senior Management collaboration: Our audit work is very important and is the foundation in terms of providing assurance to various stakeholders and improving and showcasing best practice. We have a total of 7 regions and the scheduling of the audits depends on our contractual agreements, prioritized operations depending on business size as well as the risk level of that country. We currently have a system to capture the audit findings however the scheduling, evaluation and synthesis of complex, contextual information of audit findings, regional analysis, operations profile, compliance, staff practices and risk factors comparison are done manually. When there is a major investigation or results scrutiny, then 2 other different systems are being used. Very often, there is cross over of information depending on the incident and in terms of collaboration we rely a lot on the senior managers of that sub-function to provide an update in the senior management meeting. When we discover a major malpractice incident in one of our 7 regions, we have an escalation process where this is discussed as a triage and we involve other senior managers when relevant to contribute as per the subfunctions. AI could proceed with the analysis of our audit findings cross region, linking that to the analysis of the other sub functions system flagging patterns, trends, non compliance and risks supporting the whole decision making process. For instance if we have a major investigation in a particular region/operation following a major incident, the AI could flag whether an audit was scheduled for that operation/centre, whether any audit findings picked up any major concern in that operation and within the whole region, potential other operations where this could be a risk to enable prompt mitigation and assurance to stakeholders. The AI however could fall short in terms of understanding the reason why it happened leading ultimately to the non-compliance. For example, AI could highlight trends in the CCTV footage evidence, but we wouldn’t know there was non compliance by staff unless we did interviews to gather this information and agree on the final outcome. Thus although the AI would be key in drawing the dots, as a leader I believe that we would still need to proceed with the human intervention to agree on the outcome and decision. Henceforth if it was lack of training/understanding of the process which was the gap, we can re-define the governance and agree with the other senior lead on training initiatives, monitoring of understanding and checking of implementation through a second audit (face to face or desk based). Thus the ideal new workflow for this incident would be : AI triggers the trend, within 24 hours our regional manager and senior audit manager jointly review this checklist as per below: i) Did the staff follow the mandatory training? ii) Date and details of refresher training? iii) Is there any evidence of training that we could review (attendance register/online evidence)? iv) What is the evidence and gap leading to the conclusion that it was a lack of training from the interview meeting notes? v) Can the line manager confirm the training of this staff member? After this review, both should sign off the process. 2. Resource Allocation: Managing 30+ auditors across 110 locations with limited resources is our biggest challenge. Currently, scheduling is spread in 4 to 7 Excel files that occasionally get corrupted. Here's the decision that as senior managers we keep also asking: if AI recommends optimized audit scheduling, how do we know when it is trustworthy and can we reject/override it? We have an ambitious audit plan with limited valuable audit resources. We have 3 staffs supporting the scheduling which is demanding in terms of administrative effort. The AI system could optimize the scheduling and deployment of audit resources based on risk data, availability of auditors, past audit findings, etc. The only challenge for the AI however would be whether it would understand the specifics of the 7 regions as these have different context based on the country risk, business profile, fraud and perception index, etc. For example, we have some regions who are highly compliant and other regions who are compliant with high risk level associated with business or country risk itself. It’s still unclear whether the AI will understand if the highly compliant region would only need more capacity building while the other high risk region would need more enforcement hence more audit scheduling. Before finalizing the audit deployment, we could initiate a validation where regional leads must either confirm or reject the AI recommended scheduling with a justified explanation and relevant documentation. This would prevent the AI from overriding local knowledge of the context while still benefitting from the pattern analysis. The AI could also help us identify the locations where we could use desk based audits given we have limited resources and 110 locations. We would of course need to agree the criteria and business rule to enable this selection. Overall, AI could propose the scheduling but the allocations to high risk regions must go through a monthly audit meeting where risk, capacity, auditor profile are reviewed and debated/approved. AI doesn’t decide but the data it generated informs the decision. 3. Audit Observation and Quality Assurance: When we are writing our observations, we are stating factual observation, accountability, writing a professional tone and making recommendations. We have a quality assurance process reports are reviewed to ensure clarity to the relevant audience and benchmarking/standardizing our auditors performance as well. AI could potentially refine the draft by making consistency checks. However given the confidential nature of those reports, we wouldn’t want AI to decide or change the whole report which would impact our reputation. Thus AI could flag inconsistencies but the scheduled auditor reviews and signs off that they have verified its accuracy, observed facts while the confidential details remain protected. The lesson learnt from this incident is that AI’s real value isn’t about speed but preparation. My new role as a leader is to become a data driven leader focusing and interrogating the data AI surfaces, and not taking a decision quickly. These validations and sign-offs are controls that will let me actually trust what the system is saying.

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.