Leaderboard

Mentioned below is the break-up of the question for our discussion. 1. Describe a scenario where AI accountability can be a concern. 2. What responsibilities can be assigned to AI agent and stakeholders. 3. Design safeguard mechanisms to ensure transparency and traceability. 1. Scenario where AI accountability can be a concern: A Community Management company has developed and deployed an AI agent to manage their incoming maintenance requests from tenants across different sites by asking & recording required details, prioritizing requests, creating work orders and assigning available technician. The agent uses historical training data to identify the impact and priority. On a typical day, a tenant encounters electrical sparks in the electrical panel due to water dripping from the AC. The tenant follows the protocol, contacts and provides all details asked for by the AI agent. This is obviously a matter of danger. AI processes the recorded details and classifies this as “low priority” and schedules the workorder for the next day. Here the agent failed to sense the urgency of matter and act accordingly, which is a matter of grave concern. Who shall be blamed for this failure? How can be build responsibilities? 2. Responsibilities that can be assigned to AI agent and stakeholders: a. AI agent: It is not a legal being and hence cannot be made accountable for any of its outputs or results. b. Data scientist: Ensure to provide clean, contextual, clean, unbiased and adequate data for training the model. This is crucial to determine the accuracy of the model. Accountable for model performance. c. AI developer: Identify the right technology, platform, build risk-aware models and integrate fail-safe, traceability and explainability features. Assess the relevant model performance assessment metric such as sensitivity / specificity etc. Ultimately responsible for logic / code related flaws and failures. d. Operations team: Continuously monitor the system’s performance until satisfactory levels of performance are achieved. Intervene and override model outputs when the model provides incorrect outputs. Provide feedback to finetune the system to continuously improve its performance. e. Compliance & Audit team: Responsible for assessing all risks with its counteractions. Define acceptable levels of performance. Periodically audit to identify bias, flaws and security breaches. f. Core committee (Governance Team): Provide strategic directions and take ownership of AI initiatives and outcomes. Manage escalations and legal conflicts arising due to AI related crisis. 3. Design safeguard mechanisms to ensure transparency and traceability: Deploying AI agents replacing humans could affect quality and compliance of service. How can we build transparent and traceable systems? a. Transparency: i. Build Explainable AI: The AI agent must have the capability to explain the rationale behind the decision using simple language. ii. Realtime risk scoring: Display the risk score in real-time to the tenant and technician that AI computes based on the keywords in the problem description. This will help them refine their words to increase the risk score. iii. Safety layer & Human-in-the-loop (HITL): Add a safety layer to the model defining key words such as leak, spark, fire etc. where the AI agent can bypass and notify a human when the defined keywords are detected. b. Traceability: i. Detailed Decision logs: Maintain details logs of all version changes, access, feedback loops, communications, actions, decisions and decision overrides. ii. End to end tracking: Create unique records for each cycle, starting from tenant inputs, AI actions / decision, technician assignment and workorder closure with CSAT scores. This will help trace the workflow and perform forensics. Conclusion: The above case demonstrates the necessity of establishing clear responsibilities for stakeholders involved in the AI deployment. By building robust traceability and transparency in AI systems, organizations can ensure that the outcomes are both understood and accounted for. Ultimately, the Governance Team shall take responsibility for the outcomes and take appropriate measures to build a robust and successful journey.