Smitha Muralidharan

Rethinking the Compliance v/s Product boundary in US Mortgage Production. In Berkadia, Our team performs financial analysis, drafts Narratives, property and neighbourhood descriptions, prepare BOVs and offerining memorandums, pull reports like crime reports, rents and sales comps, abstracts Lease information, and Design marketing material for Investment sales team. The traditional boundary in our world is clear : Product/Production drives the speed, deal quality, and banker enablement. Compliance ensures regulatory adherence, marketing accuracy, and reputational protection. Product optimizes for velocity and win rate. Compliance optimizes for risk mitigation and defensibility. Today, AI is beginning to disrupt the separation and particularly in Narrative drafting and Quality audits. Where the boundary breaks Historically anlaysts draft Narratives manually, Drafts are reviewed by Quality/Compliance team, Edits are made and Turn around time stretches. Compliance operates after creation. But with AI integrated into drafting and auditing, that linear workflow no longer makes sense. AI can: 1. Draft property narratives from structured underwiring inputs. 2. Cross check claims against comp databases 3. Flag unsupported market statements 4. Standarize disclaimers 5. Detect fair lending and advertising risks in language before release. Once AI is embedded, "review" is no longer a downstream function. it becomes part of creation itself. The boundary shifts from: "Did Compliance approve this?" to " Was this generated inside AI controlled compliance framework?" How roles would need to evolve Compliance shifts from reviewer to system architect. Instead of revieweing documents line by line, Compliance would : Define guardrails embedded in AI prompts Approve training data sources Set thresholds for acceptable risk language Co-design automated audit logic Montior exception dashboards. They move from reactive oversight to proactive control design. Production team becomes AI orchestrator, Not document creator Analysts no longer primarily draft. They validate AI outputs, Escalate edge cases, Apply deal judgment, Refine prompts when necessary, Ensure banker intent is preserved. Middle management shifts from workload supervision to exception management and risk calibration. Production, Compliance, and Technology must form a joint operating model The biggest structural change is: AI collapses the separation between building and approving. We cannot embed compliance logic into AI without compliance being upstream in system design. That means: Shared ownership of AI tools, Joint governance committee, Continous audit model, Clear accountability for hallucinations and misstatements. The old model: Production creates -> Compliance polices The AI model: Compliance codes guardrails -> Production operates within them. What happens to the Roles? They dont disappear but they shift. Analysts becomes AI validators and market intelligence interpreters. Compliance professionals becomes risk system designer. Middle managers evolve into risk performance balancers. Technology team becomes core to revenue generation, not support. The boundary between compliance and product does not vanish. It becomes codified inside systems rather than enforced between departments. The Strategic Impact: In Commerical mortgage production, speed wins deals but reputational risk can destroy them. AI allows us to 1. Increase velocity 2. Reduce inconsistency 3. Lower manual rework 4. Standardize risk controls But only if Compliance is embedded upstream. The real automation is not of Narratives and its audit, it is th redesign of accountability. AI shifts compliance from a checkpoint to an infrastructure layer. And in a highly regulated financial environment, that changes how teams must be structured.

In Narratives process for US commerical mortgage banking deals, AI(Berkie) is being embedded directly into analysts day to day workflow to help draft core Narratives sections. Original (Pre-AI) workflow and performance expectations Before AI tool called Berkie was used, the Narrative draft workflow followed a manual, time sensitive sequence Document Collection & Review Analyst manually gathered and studied multiple source documents: Offering Memorandum, Appraisal report, Property website, Borrower website/Sponsor details, Rent comps, Sales comps, expense comps from external websites, Crime reports, Google/Google Maps for location & aerial review, and latest financial analysis. Drafting Narrative sections manually Analyst wrote every section from scratch: Property overview, Location overview, Borrower/Sponsor overview, Management overview, Market Overview, Rent/Sales/Expense comps, Strength & Weakness, Risk & Mitigants, and Crime reports Performance expectations Accuracy of extracted data, Consistency in writing style, ability to identify key risks and themes, 100% manual verification, Turnaround time was typically longer(5hours to a full day depending on deal complexity) The process heavily relied on the analysts attention to detail, writing ability, and familiarity with CRE underwriting. What AI (Berkie) now does in the workflow With introduction of Berkie, analysts now use a structured marketplace of prompts for each narrative section. Analysts upload relevant documents (Offering memorandum, Appraisal, reports), URLs where possible(property site, crime data, etc), Screenshots or extracted information where login access restricts data. Berkie's role: Reads the attachments and URLs, Uses predefined prompts for each section, Produces a first draft narrative, structures the content according to the standard CRE narrative format (Freddie, Fannie agency template), Extracts factual information (property details, comps, management info, location attributes, market trends, etc) Analysts role afte AI input: Validate numerical accuracy, Fix missing or misinterpreted insights, Add deal specific perspective, Ensure compliance with underwriting and agency guidelines, and Finalize risks, mitigants, and subjective assessments. This has shifted analysts responsibility from writing everything to reviewing, correcting, and fine tuning. One situation where AI improves Improvement scenario : Enhancing speed & consistency in Market overview section. The market overview section often requires synthesizing market rents, vacancy rents, Employment trends, population growth, local economic drivers, competitors property performance. Before Berkie(AI), Analysts spent significant time pulling this from multiple sources and writing a clear Narrative How Berkie(AI) improves it: Berkie extracts and organizes market stats quickly, generates consistent writing style across deals, Highlights macro trends an analyst might overlook, saves hours of manual research & writing. Impact: Faster turnaround, reduced analyst workload, and more uniform quality across the team. One situation where AI could introduce risk, bias, delay or hidden errors Risk scenario: AI misinterpreting a financial data or comps Berkie may mis-read or mis-interpret Rent comps, Sales comps, Expense line items, NOI or DSCR calculations, Sq footage discrepancies between OM V/S Appraisal, Property photo context or map locations. Example: If Berkie incorrectly intreprets rent comps(e.g mistakes asking rent for effective rent, or uses older data from attachments), the narrative could inaccurately reflect market positioning leading to misinformed lender decisions. Why this creates a risk: Financial misreads may not be obvious during a quick review, Berkie/AI sometimes hallucinates missing data, Analysts may overtrust the AI draft, Incorrect comps analysis affects valuation, underwriting, and risk assessment. Potential outcomes: Undetected errors -> Misleading Narratives, Delays if analyst must significantly rewrite sections, Bias if AI leans toward overly positive/negative language, and Risk missing key red flags( e.g deferred maintenance, tenant rollover, poor crime trends) To manage these risks and bias, the workflow must treat AI as a drafting assistant with clear expectation that analysts must Cross check key data points against source documents, consciously adjust for optimistic marketing language v/s independent data, Document any data conflicts or uncertainities in the Narrative or internal notes. This balance using AI for speed and consolidation, while keeping human analysts fully accountable for accuracy and judgement is what turns AI from simple automation tool into a genuinely collborative part of the Narratives process.

Narratives + Berkie AI MarketPlace ( Internal AI) Berkie pulls data where possible and drafts/reshapes sections (Property Overview, Market Overview, Borrower & Sponsor overview, Maps & Aerials, Strengths & Weakness) Analyst will have the final say on what to keep, change or delete before anything goes to Mortgage Bankers and Underwriters Here AI is clearly recommending content and structure, not deciding. The risk is when: Some analysts might over trust (pass AI text through minimal checks) Others might ignore the AI ( rewrite everything) and lose the time savings we are aiming for. Below is how we defined when to accept V/S when to override and the practical rules of thumb to keep the balance. AI assisted Narrative drafting For clarity, here is exactly what AI is recommending: Data interpretation and summarisation: From property details, sales/loan data, crime reports, comps, market stats, sponsor info AI proposes : A property overview, Market Overview, Sponsor description, and Initial strengths and weaknesses wording( based on the inputs shared by analysts) Language polishing and standardisation: AI rewrites analyst's bullet points into Narrative form, enforces US english, clarity, tone, and structure. Section organisation & emphasis: AI may choose what to lead with, how much weight to give to crime v/s amenities, etc , based on the prompt. The AI recommendation is: "Given the provided inputs and prompts, this is the Narrative you can send to Mortgage Banker" The Human decision should be: " Is this narrative accurate, appropriate, and aligned with banker expectations for this specific deal?" Decide when to accept and when to override We trained analysts to make the accept/override decisions in three simple phases. Facts & input checks: " Is everything that looks factual, actually true ?" Default Stance: NEVER blindly accept AI's factual statments. Verify first. Accept AI's wording when, every fact and statement is directly traceable to Berkie data, Analyst provided notes, Attached reports(crime, market study, rentroll, etc). AI is merely re-phrasing things you already confirmed.(e.g summarising CRIME report which analyst already read). Override or heavily edit when: 1. AI infers anything beyond the input e.g a) "The sponsor is highly experienced".. when the input only says "2 prior deals" b)"The area is considered safe".. based on limited or ambiguous crime info. c)"The market is booming".. without strong support in the market data. Numbers are rounded/changed or additional figures appear which are not in reports AI uses vague, unanchored claims like strong demographics, high demand etc without references. Rule of thumb: If we cannot pin point the exact sources, we do not accept as-is. Either adjust to match what we can back up with source or remove it. Emphasis & Framing: "Is the story this drafts tells is right for this deal?" Here is the AI is recommending about how the deal is positioned, not just the facts. We accept AI's framing when a) The draft matches analysts judgement of the deals story. For a strong deal, it should lead with genuine strengths and mentioned balanced risks. For a challenging deal, it should acknowledge core risks clearly but professionally. b) The ordering feels right when high impact positives are clearly highlighted and Major risks are not buried at the bottom or softened to the point of misleading. Override and reshapen when a)The AI is too optimistic relative to the risk profile(e.g crime or sponsor concerns are downplayed) b)Important banker specific angles are missing, example MB has explicity asked to stress sponsor's track record or focus on a particular market risk.c) You read the narrative and feel, "If I were the banker, this would give me the wrong impression." Rule of thumb: Ask "If the banker only reads the page once, would they walk away with the right high level stroy"? If the answer is anything other than Yes, then we must override AI's framing in that section. Language, Style and efficiency: "Is the AI doing better than I do manually?" Here AI is the strongest and we want to avoid overriding out of habit. Accept AI's language when a) We have already validated the facts and framing (1&2) b)The text uses US english, Is grammatically correct and clear, avoids redundancy and overly salesy language. c) Our edits are purely minor preference tweaks(e.g swapping a word, reordering a clause) Overrider or partially edit when a) The tone does not match Berkadia style ex: Too casual, Too promotional or too negative b) There are repeated phrases or clunky sentences that could confuse a banker c) The same sentence can be flagged by error matrix for language/format. Rule of thumb: Once facts and framing are correct, bias toward accepting the AI's language unless it clearly violates style guidelines or would confuse the reader. Dont rewrite just because we would phrase it differently. Safeguards to prevent over trust To stop analysts from rubbing stamping AI output we must: Mandatorily review checkpoints in the tool. Before marking a draft as "ready for MB review", analysts should check boxes likes a)" I have verified the factual accuracy of AI generated section" b)I have ensured that key risks are neither omitted nor downplayed. This keeps responsbility clearily with analyst. System warnings for high risk sections. We flagged certain sections as " always require extra check" a)Sponsor description (risk of overclaiming experience or financial strength) b)Crime/Safety commentary c)Any inference heavy market commentary. In the UI, These sections are a) Highlighted and b) Have a small message " High jugde area - verify carefully, do not rely on AI alone" Random sampling and feedback. Weekly once a random request of each analyst is reviewed to see a) Where analysts accepted AI text verbatim b)Whether those sentences were indeed fully supported by sources. If patterns of over trust is detected during audits, we will use them in training and tweaking the prompts if necessary to be more conservative. This way, In the Narratives + Berkie Market Place ( Inhouse AI of Berkadia) Analysts know exactly when to trust AI(polishing, structuring, supported summaries) They know exactly when to override(judgement heavy areas, sponsor risk, crime, deal story) And we keep pratical balance between quality, speed, and human responsbility, instead of drifting towards blind trust or total disregard of the AI.

Project : Reduce cycle time and improve quality of Narratives draft for property offering final memos. Problem statement : Current average time to produce a first draft Narrative ( Property overview, Market Overview, Borrower Sponsor, Maps & Aerials, Demographics, Crime Reports, Strenghths & Weakness) is 5hours per deal, with 16% of drafts requiring major rework due to : Missing or inconsistent data Grammar, UK/US English used interchangeably Misalignment across sections(e.g strenghths not matching the market facts) Goal : Reduce average drafting time by 30 - 40% and reduce work by 20 - 30% while maintaining or improving Narratives quality( rated by originators & underwriters) Where AI Fits : Auto generating data from known sources ( internal systems & vetted external websites) Gnerating a first draft of : Property overview, Market Overview, Borrower Sponsor, Maps & Aerials, Demographics, Crime Reports, Strenghths & Weakness Improving grammar, UK/US English usage, Overall stylistic alignment Analyst still : confirms source credibility, checks factual accuracy, adds nuance specific to the deal, approves/ edits strengths & weakness. Hypothesis Formation : In this project, AI outputs are best treated as structured hypotheses, for example: The property's location near XYZ business park is a key strength due to strong employment growth Crime rates in the submarket have declined over the five years, supporting stability of the assest Market rent growth has moderated but remains above long term averages. From an MBB standpoint: These are not facts, they are claims to be tested AI's role in Define/Measure is propose: Here is what might be true, here are patterns implied by the data I see. Guidance to analysts: Treat AI text as hypothesis rather than validated conclusions Use prompts that frame AI output as provisional. For example: "List 5 potential strengths of this property based on the data below, and mark each as High confidence or Needs verification with a short reason. Hypothesis testing : AI can help structure tests, but LSS will be led by humans. For Narratives quality, Hypothesis will be like H1 : Using AI to draft market overview reduces average drafting time by 30% without lowering quality scores H2 : AI generated strengths/weaknesses will be atleast 80% aligned with analysts final conclusion AI can help design experiments or sampling plans( eg: How many deals do we need to test to detect a 20% reduction in cycle time at 95% confidence level) and Suggest such metrics to track ( Time saved, number of factual errors, number of grammar corrections, etc) But the actual test( data collection, calculations, significance tests) should be performed in transparent tools like Excel, Minitab, etc. where we can see the formulas and logic. AI derived statistics ( eg: This is 90% likely) should not be treated as valid inferential statistics unless we have a clearly documented model and method behind it. Hence MBB needs to have rule of thumb that : AI is acceptable in forming hypotheses and in helping us set up tests AI is not acceptable as the only source of evidence that a hypothesis is confirmed. Statistical validation must be done through transparent methods. Statistical Confidence: We are building marketplace with in Berkie (In house AI) for all the prompts to be used for Narrative drafting In the marketplace initiative, there are two layers of confidence Confidence about workflow change (Process Improvement) : Does using AI actually improve the quality/productivity? Confidence about individual deals output ( deal level narrative accuracy) : Is the AI generated narrative for this deal accurate enough to use? Confidence in workflow change: Here as an MBB, we should use classic LSS experiments. Define Metrics : Drafting cycle time per deal, Number of factual corrections per Narrative, Number of grammar/style fixes, and Originator & analysts satisfaction score Design : Pre/Post study like N deals before AI, N deals after AI or Some analysts use AI, some dont, over the same period of time Analyze: Use standard tools (T tests, control charts) to confirm 1. Is there a statistically significant decrease in drafting time 2. Is quality maintained or improved Document : Effect sizes, confidence intervals and any risks observed(e.g common type of AI errors) Confidence about individual deals output: Here instead of 95% confidence in a strict statistical sense, we define operational acceptance criteria. For example : For an AI assisted Narrative to be acceptable as a draft, we require 1. 0 critical factual errors (data that could mislead credit decision) 2. less than or 2 minor factual discrepancies ( currently aiming for 97% accuracy, acceptable error is three low critical errors) 3. Grammar and usage of UK english errors below X per 1000 words 4. Analyst can review and finalize within 30mins for 90% of the deals. For an MBB perspective, we can: Use sampling and inspection like 1. Randomly sample AI drafts 2. Score them against a checklist 3. Track defects per unit and apply control charts Overtime we can characterize AI as a process with a certain defect rate, Just like any human process. Assessing data quality and credibility when AI is in the loop This is critical as the data for Narrative is pulled from multiple sources (Internal systems, public websites, vetted websites, etc) Separate source credibility from AI credibility : A) As MBB, We should enforce a clear hierarchy Source credibility rules(upstream of AI) : Internal : Salesforce/Omniview, Berkadia internal property/loan systems, internal market research has highest trust, Trusted external: Offical census data, reputable third party market reports, public crime data portal has highest trust, Open web or generic searches (google search) has low trust unless specifically validated. AI Usage rules: AI is allowed to Summarize and rephrase known, structured inputs from trusted sources and Highlight patterns or inconsistencies across those sources. However, AI is not allowed to freely invent data not present in the supplied sources and acras the primary source of record for quantitaive facts. B) Data lineage and traceability: To keep Lean Six Sigma like rigor, we need traceability For each factual statement in the AI narrative, analysts should be able to trace source and transformation We should ask AI to annonate paragraphs with references e.g(Source : Crime Report 2024 Q2) or (Source: Internal rent roll 2025-01) or generate a separate evidence log for the narrative From an MBB angle, this is like keep data collection forms and measurements system documentation for our Y's and X's. C) Measurement system analysis for AI and analysts : We will take AI and analysts as measurement devices for Narrative content. We wil run Gage R&R style excerise : we will select a set of deals we will have a) AI only draft(first pass) b) Analyst only draft(without AI) c) AI+Analyst (AI draft, then analyst edit) We will have independent reviewers ( seasoned and quality specialist) rate accuracy, clarity, grammar/UK US english usage, alignment between sections We will assess variations attributing to AI v.s Human v.s Combination. Where AI improves repeatbility/consistency(e.g style/alignment) v.s where it adds risk ( e.g subtle factual errors) This quantifies data quality and helps set where AI is safe v.s risky Where AI should accelerate decisions, and where traditional validation is non - negotiable A) Areas where AI can safely accelerate and automate: In the Narrative AI marketplace, AI is well suited for: Drafting and rephrasing : a) Converting bullet research into fluent US english paragraphs b)Enforcing consistent style and structure Summarization: Summarizing long third party reports like Crime Reports, Market Reports, Sponsor histories, etc Formatting and aligning : a) Aligning terminology across sections(e.g consistently referring to submarket names, asset class labels b)Ensuring internal consistency between section (e.g strength and weaknesses must refer to facts already stated in the narrative. Highlighting potential strengths and weaknesses : a) Suggesting deal strengths/risks based on the data provided b) Tagging statements as "requires analyst verification" where sources are weaker or ambigous Quality checks : a) Flagging likely grammar issues b)Enforcing US spelling conventions c)checking for obvious contradictions (We said crime is low in one section and high in another section of Narrative) Here an MBB can reasonably reduce manual efforts while keeping risk low, especially with defined guardrails and human review. B) Areas where traditional validation is non negotiable From a Lean Six Sigma and risk perspective the following should not be delegated solely to AI Quantitative facts that influence credit/investment decisions e.g Occupancy rates, rent levels, reports fulled from trusted source Casual statments and forward looking claims e.g Because of X, We expect Y, Crime reductions mean lower risk for the asset. AI may propose such statements however Analyst must evaluate casuality using domain knowledge and data. Use traditional reasoning and where relevant, simple statistical checks(e.g trend analysis) rather than trusting AI's intuition Compliance, regulatory and reputation risk : Any statement that touch on fair housing, sensitive demographic descriptions, legal regulatory interpretations must be reviewed and where necessary, crafted and reviwed by analysts. AI may help draft neutral language but it should not be the final authority. Final sign off and accountability: The accountable owner of the narrative is the analyst ( and ultimately the deal team) not the AI. MBB guidance should codify that a) AI is a tool in the Measure/Analyze steps b)Approve/Reject decisions remain with analysts. In conclusion, AI should treated as a powerful assistant, not a replacement for Lean Six Sigma rigor. It can speed up research and drafting the Narrative, but the standards for data validity, statistical confidence, and final judgement must remain with the analysts. Where the impact is high, traditional verification is still essential and where risk is low, AI can safely help us work faster and more consistently.

In my process, We are adapting DMAIC to support the introduction of in house AI for recurring and repetitive tasks, while analysts retain ownership of making final judgemental calls, in depth research and deal interpretation. Previously, the problem statement was Defined around cycle time and quality issues. With AI, now Define phase is more focused on 1. Analysts spending excessive time of recurring & repetitive tasks and refinement of the Narratives draft 2. Inconsistencies and information repetition errors 3. Reduced time of analysts on higher value research and insights. AI in this phase will strengthen clarity on which activites are AI eligible (Usage accurate Language, structure of the draft, removal of repeated information from draft) and which activites will remain under anaylsts control ( research from govt websites, county websites, narrating property nuances and deal story teeling) AI will enable in keeping the Measure phase more objective. In our process, We measure beyound TAT and it will include reduction in drafting time, reduction in language related errors and repeated information, consistency of tone and structure across drafts, Audit feedbacks before and after AI introduction. Analysts will continue to have authority of accepting AI improvements and also to check if it aligns with MBs expectation of Narratives draft In Analyze, AI can help identifying patterns such as 1) Common repetition issues (within draft) 2) Frequently used unclear and/or overused phrasing 3)Structural inconsistencies Analysts will continue to ascertain the rootcause, deal specific risks and do SWAT analysis of property, understanding nuances of property that AI cannot While in the Improve Phase, AI will be used or tested as main processesor for repeated and recurring tasks including language. Improvements will include 1) Standardized prompts for drafting Narratives as per research material and instructions provided about the deal 2) AI driven/checked language clarity, grammar, redundancy removal 3) Faster draft creation, allowing analysts to focus on research and accuracy Analysts will have complete ownership of writing prompts, deciding if AI output is acceptable, determinings for tasks AI cannot be used. With, the Control phase will move away from regular flow and now we are focusing more on goverance of 1) Prompt libraries and updating library when required 2)Mandatory review of AI assisted drafts 3)Random audits of 10% drafts of each analysts each week by audit team to ensure Narrative quality and prevent/stop over dependence on AI. In conclusion, AI can add to DMAIC but cannot replace it.