Building Trustworthy GenAI for Regulated Industries
Abhi Rajan
·
2026-04-28
·
via Artificial Intelligence in Plain English - Medium
A Practical Playbook for Financial Crime Teams In this article, we set out how financial crime teams can move beyond experimental uses of generative AI and establish governance that regulators, auditors and analysts can genuinely trust. As generative AI finds its way into more financial crime systems, a theme that keeps resurfacing in discussions with practitioners is trust in the outputs, processes and decisions. What we’re looking at is the type of trust that regulators expect to see documented and auditable. The technology is powerful, but its adoption in regulated industries cannot rely on optimism or technical enthusiasm alone. It requires structure and discipline, as well as a clear understanding of where risks genuinely lie. This is why we need a practical playbook for deploying generative AI responsibly in financial crime teams that is focused on real operational considerations rather than theoretical frameworks. Why Trust Still Matters in 2026 Financial institutions have mostly moved past early experimentation with generative AI and into cautious production use. In many anti-money laundering (AML) and know-your-customer (KYC) teams, large language models (LLMs) are now being used to support screening. Financial institutions are even going so far as to use them for investigation and case preparation in live environments. It’s no coincidence. In fact, the change has landed in our laps as a result of quite a noticeable change in regulatory posture. Supervisors across major jurisdictions are no longer debating whether AI will be used. In simpler terms, gen-AI is no longer an “if.” What we’re seeing instead is that they’re asking how firms govern it and test it. We’re all looking to establish how firms show evidence of control. That regulatory shift has only sharpened in recent months. In January 2026, the House of Commons Treasury Committee published its report on AI in financial services and recommended that the FCA issue practical guidance by the end of 2026 on how existing consumer protection rules apply to AI as well as on senior manager accountability under the Senior Managers and Certification Regime (SM & CR) (Treasury Committee, 2026). Just days later, the FCA launched the Mills Review, which is a long-term examination of how AI may reshape retail financial services through to 2030 (Financial Conduct Authority, 2026). Nikhil Rathi has reaffirmed that the FCA is not planning to introduce AI-specific rules and is doubling down on its principles-based approach. However, guidance on audit trails and human-in-the-loop protocols is now expected later this year. Across the Channel, the EU AI Act becomes fully applicable on 2 August 2026 (European Commission, 2026), which means that firms operating in both jurisdictions will need to reconcile a still-flexible UK framework with a much more prescriptive EU one. The Bank of England’s Financial Policy Committee has also weighed in. Its April 2026 record notes that, while generative and agentic AI have not yet been adopted at a level that would present systemic risk, the risks are likely to grow rapidly as deployment expands (Bank of England, 2026). The simple takeaway is that the window for firms to define their own GenAI governance posture before supervisors define it for them is narrowing. This is where it gets heated for financial crime teams because generative AI increasingly shapes regulated outcomes even where final decisions remain human. An AI’s summaries will influence escalation. Its suggested narratives will frame that final judgement. The retrieved context that it provides will set guardrails around our interpretation. Be that as it may, the risk doesn’t fall on automation alone but in how AI steers attention and reasoning inside of these very controlled processes. That is exactly why trust is still so important. We can’t see trustworthiness as a software property. It has to be an operational condition that is intentionally created through documentation, evaluation, oversight and accountability. I believe that firms that treat trust as a technical problem will struggle to justify their outcomes. However, those that treat it as a governance discipline will likely be better positioned to defend their decisions if (and when) they come under scrutiny. The GenAI Risk Landscape for Financial Crime Teams in 2026 Generative AI has a distinct risk profile in AML and KYC use cases that truly differs from traditional statistical models simply because we’ve never had anything like it before. One of the biggest concerns at the moment is subtle hallucination because even a small deviation in an output can completely change the contextual understanding of a case. The devil here is that even though outputs may appear coherent, they might be reliant on extremely weak inference or just incomplete evidence. Unfortunately, this can distort summaries, which, as I’ve just now pointed out, will affect case narratives. Related to this issue is shallow reasoning, which happens when models produce seemingly fluent conclusions without adequately reconciling conflicting sources. This only gets riskier with expanding context windows. Models that process lengthy onboarding files or adverse media packs often struggle to synthesise multiple narratives consistently (Li, 2024). This can lead to selective emphasis, which goes completely against the grain of the type of balanced interpretation that firms actually need. Behavioural drift only worsens this. Even without major retraining, changes in prompts or data sources can totally alter the tone and structure of outputs over time. The same goes for orchestration logic. These risks will stack up in multi-agent systems because when retrieval, classification and summarisation elements are all combined, the errors will propagate across the pipeline. The whole point is for the system not to be siloed, but it’s that very design that will spread the error across the board. This isn’t some far-fetched worry. Supervisory bodies have also brought up these concerns, one of which is the Financial Stability Board, which has documented system level governance challenges that are tied to AI behaviour and control gaps in complex deployments (Board, 2024). The BIS Financial Stability Institute has also examined model governance failures in AML contexts where interpretability and oversight created lagged implementation (Settlements, 2015). Public facing giants like OpenAI and DeepMind are just as concerned about large language model behaviour, which just shows the need to treat these risks as serious structural instead of treating them as purely incidental (OpenAI, 2025). A Pragmatic Evaluation Framework for GenAI in AML/KYC This all has serious consequences for gen-AI in financial crime use in that we need to move away from narrow accuracy measures. The simple reason is that language models rarely fail through obvious errors. It’s the subtlety that makes this all the more dangerous if not properly guided. The risks that we’re seeing come up are inconsistent behaviour as well as selective reasoning and even weak handling of edge cases. This is why I believe evaluation should focus on behaviour under realistic conditions instead of focusing on benchmark performance alone. My opinion is that scenario-based testing and adversarial reviews are needed in order to understand how these models respond to things like ambiguity and contradiction. Knowing what an LLM does with incomplete evidence is a must at this stage. We have to make it so that test data reflects operational reality. This should ideally include multilingual adverse media, common name collisions, conflicting reporting across sources and changes in narrative tone across jurisdictions. These should go hand in hand with acceptance criteria, which should prioritise consistency above everything else. Variance control and traceability of outputs should come in at a close second and third on that priority list. We can’t only base acceptance on whether an answer appears correct in isolation. There are several practical frameworks that can be applied here. The MAS Veritas methodology is the first of which that provides structured templates for explainability, fairness and model testing. These are well-suited to screening use cases (Singapore, 2022). This is how core Veritas dimensions can be plotted to concrete evaluation checks within AML screening workflows. Table 1. Mapping the four core MAS Veritas dimensions to concrete evaluation checks within an AML screening workflow. The point of this mapping is to make abstract governance principles testable in production rather than leaving them as design intentions. The FATF Digital Transformation guidance sets clear expectations for applying AI within AML processes while retaining human accountability. This works well too (Force, 2025) and can be applied using the following logical stream. Figure 1. A simplified end-to-end view of where generative AI sits within an AML screening pipeline. Note the placement of AI Enrichment between Screening and Analyst Review. The model adds context and structure but never replaces the analyst review or the audit trail that follows it. For broader system level risk assessment, the NIST AI Risk Management Framework supports outcome based evaluation across the full AI lifecycle (Technology, 2025). While these make for good frameworks, there are still certain governance components that should really be non-negotiable. Governance Components Regulated Institutions Cannot Skip When all is said and done, trustworthy use of gen-AI in financial crime depends on governance fundamentals that we simply cannot treat as optional add-ons. Documentation is the first component that firms must have in order. They need to clearly define intended use, known limitations, data provenance, expected model behaviour and fail-safes before ever thinking of deployment. The reason for this is that downstream assurance will quite quickly collapse without it. In simple terms, if you don’t describe what you expect the model to do, you can never determine whether it did what it was meant to. Explainability standards are linked to this and they’re especially important where retrieval-augmented generation is used. Teams need to evidence how conclusions are reached. They need to be able to show which sources were relied on as well as where uncertainty might still be lurking. This is why ongoing monitoring is so necessary. Essentially, it completes the control loop. Behavioural drift, tone changes, reasoning shortcuts and output instability have to be tracked. In addition to this, there needs to be strong version control that is paired up with audit trails. That way, models can be compared, reproduced, tailored and rolled back as and when necessary. The last of the governance components are escalation and override mechanisms. These must be explicit because analysts need clear authority to intervene when confidence is low in the output or when the evidence is inconsistent. This all lines up with the Prudential Regulation Authority’s (PRA’s) model risk management principles, which emphasise clarity of use, control and accountability. In the United States, SR 11–7 lays out model governance as a system responsibility instead of a technical exercise (Currency, 2011). Similarly, in the UK, the FCA continues to reinforce the expectations of oversight, explainability and human control in its AI guidance (Authority F. C., 2025). The only major difference between us and them is in how these expectations are expressed and enforced. Designing Human-Centric GenAI Systems This begs a new question. How do we design human-centric gen-AI systems that match up with this governance? Firstly, in regulated financial crime environments, gen-AI must be designed to support human judgment. It needs to be a tool and not a substitute for human decision-making. At the end of it, accountability for decisions is still placed on analysts and this doesn’t change when AI is introduced into the workflow. Gen-AI is there to provide context. At least, that’s what it should be there for, along with structure and clarity. It must help investigators make better informed decisions within a set of established controls. In essence, this is where the idea of AI as a companion can become useful in an operational sense. This is primarily because well-designed systems can bring relevant evidence to the surface and organise complex information. That helps analysts see into nooks and crannies where uncertainty may have been instead of hiding it behind wrongfully overconfident conclusions. This just means that outputs should highlight gaps, conflicting sources and areas that need human judgment. The goal here really is to reduce the cognitive load that analysts deal with but without narrowing their analytical discretion. Again, the MAS FEAT guidance speaks to this as it sets out guidelines for fairness, ethics, accountability and transparency and places them as practical design obligations instead of abstract values (Singapore, 2022). The upside is that investigators and analysts agree with this as is seen in the ACAMS publications on AI in AML operations, which supports the same point from an investigator perspective. The overall takeaway from the article is that analyst trust is built when systems are predictable as well as explainable and clearly subordinate to human control (Yücesoy, 2025). Integrating GenAI into Existing Risk and Control Frameworks With these parameters for human-centric gen-AI in mind, it’s a must that we acknowledge that it’s easiest to govern when it’s treated as an extension of existing model risk and operational risk practice. It’s not a special case and I don’t feel it should be treated as such. Governing it as that said extension means that firms must start with recognising that prompts, retrieval logic, ranking layers and guardrails are model artefacts. We can agree that they can shape outcomes and they can drift. They also require versioning, testing and change control. If they sit outside the formal inventory (or authoritative register of models and model-like components that firms recognise as being subject to governance), then any oversight will be nothing more than performative. To avoid, I’d recommend that teams map the full pipeline into established governance processes, including data ingestion, retrieval, inference, analyst review and audit logging. That seems to be the most logical way forward because this also forces a clean separation between enrichment and scoring, as well as between interpretation and decisioning. It makes which components can be automated and where accountability must stay explicitly human much clearer of a line in the sand. That said, practical integration of this quite often depends on aligning gen-AI controls to the same evidence standards used for traditional models, including reproducibility and rollback. Firms should also strongly consider aligning their GenAI controls with wider ICT risk expectations, including security, resilience and supplier governance where third-party services are involved. A good guide for this is set out in the European Banking Authority guidance on ICT and security risk management (Authority E. B., 2025). These parameters will continue to evolve as we garner more lessons from early deployments and those that come after. Lessons Learned from Early Deployments (2024–2025) Early production deployments of gen-AI in financial crime have exposed a set of recurring lessons that have become what we can say is difficult to ignore. One is the risk of overreliance on the models. We’ve seen analysts place rather undue confidence in generated summaries. Weak or conflicting sources were sometimes misinterpreted as coherent narratives and the issue was not model intent, although that’s what it might look like on the surface. What we were actually witnessing when this happened was just how easily fluent language can hide evidential gaps if there aren’t rigorous human fail-safes. Long context summarisation created even more challenges in that even though larger context windows gave models the ability to ingest more material, the teams were often unprepared to monitor how conclusions were formed across lengthy and inconsistent inputs. Narrative emphasis can be altered so subtly over time that it doesn’t trigger existing controls. The gist of this is that, operationally, we’ve seen that hybrid architectures perform better than pure gen-AI approaches. Rules that produce deterministic checks in collaboration with traditional models provided stability and generative components added interpretation as well as context. In sum, the institutions that seemed to embed analysts deeply into validation in the form of testing and ongoing monitoring reported higher trust and better outcomes. A Playbook for 2026 and Beyond: What “Good” Looks Like At this rate, we should see more effective use of generative AI in financial crime in 2026. Said use will most likely depend less on model capability and more on disciplined execution that falls in line with good governance. I would suggest that teams should begin by establishing domain-specific evaluation sets before any system reaches production or gets anywhere near it, for that matter. These sets will provide a stable foundation for testing in real anti-money laundering (AML) and know-your-customer (KYC) conditions, which will be far better than generic benchmarks. Of course, rigorous documentation and behavioural monitoring must come soon after in order to ensure that assumptions as well as limitations and observed behaviour are continuously recorded. From here, explainability and evidence traceability should be treated as first-class design requirements. In plain terms, if outputs can’t be traced back to sources and reasoning paths, they can’t be defended, which is exactly why strong human oversight is still of the essence. This needs to include structured review in coalition with escalation and override workflows to ensure that analysts can maintain control when uncertainty is high or evidence conflicts. Which brings me to safe failure modes. Systems must degrade gracefully without disrupting investigations or breaching regulatory obligations. Last but not of least importance on the agenda for firms has to be formalising internal gen-AI governance frameworks aligned to MAS FEAT, NIST RMF and FATF guidance as well as established PRA expectations. Making generative AI trustworthy has less to do with eliminating risk and more to do with managing it systematically and transparently. References European Banking Authority (2025) . Guidelines on ICT and security risk management. EBA: https://www.eba.europa.eu/activities/single-rulebook/regulatory-activities/internal-governance/guidelines-ict-and-security-risk-management. Financial Conduct Authority (2025) . Artificial intelligence in financial services. FCA: https://www.fca.org.uk/firms/ai-financial-services. Financial Stability Board (2024) . The financial stability implications of artificial intelligence. FSB: https://www.fsb.org/2024/11/the-financial-stability-implications-of-artificial-intelligence/. Office of the Comptroller of the Currency (2011) . Supervisory guidance on model risk management (OCC bulletin 2011–12). OCC: https://www.occ.gov/news-issuances/bulletins/2011/bulletin-2011-12.html. Financial Action Task Force (2025). Digital transformation of AML/CFT. FATF: https://www.fatf-gafi.org/en/publications/Digitaltransformation/Digital-transformation.html. Li, Y. W. (2024) . Long-context LLMs still struggle with long-context understanding. arXiv: https://arxiv.org/abs/2404.02060. OpenAI. (2025) . Research. OpenAI: https://openai.com/research/. Bank for International Settlements (2015) . Use of machine learning for anti-money laundering and model governance practices. BIS: https://www.bis.org/fsi/fsipapers11.htm. Monetary Authority of Singapore (2022) . Veritas document 3: FEAT principles assessment methodology. MAS: https://www.mas.gov.sg/-/media/mas-media-library/news/media-releases/2022/veritas-document-3---feat-principles-assessment-methodology.pdf. National Institute of Standards and Technology (2025) . AI Risk Management Framework. NIST: https://www.nist.gov/itl/ai-risk-management-framework. Yücesoy, N. (2025) . AI and the future of compliance: Tools, risks and human oversight. ACAMS Today: https://www.acamstoday.org/ai-and-the-future-of-compliance-tools-risks-and-human-oversight/. Bank of England (2026) . Financial Policy Committee Record — April 2026. Bank of England: https://www.bankofengland.co.uk/financial-policy-committee/financial-policy-committee-meetings. European Commission (2026) . AI Act: Regulatory framework for artificial intelligence. European Commission: https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai. Financial Conduct Authority (2026) . The FCA’s long term review into AI and retail financial services: designing for the unknown. Financial Conduct Authority: https://www.fca.org.uk/news/speeches/fca-long-term-review-ai-retail-financial-services-designing-unknown. Treasury Committee (2026) . AI in Financial Services (Fifteenth Report of Session 2024–26, HC 684). House of Commons Treasury Committee: https://publications.parliament.uk/pa/cm5901/cmselect/cmtreasy/684/report.html. Building Trustworthy GenAI for Regulated Industries was originally published in Artificial Intelligence in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。