Governing the Ungovernable

AI governance, the EU AI Act, and the new operating model for regulated financial services

Drawn from a recent industry roundtable co-hosted by Delta Capita and Kore Labs, supported by Cantor Fitzgerald and the Compliance Institute, with regulators, compliance leaders, technologists, and board directors in the room.

This whitepaper reflects the themes and perspectives that emerged from the discussion. In accordance with Chatham House Rules, no comments are attributed to individual speakers or their organisations.

Artificial intelligence has moved from the experimentation phase into the operational fabric of regulated financial services faster than any technology shift in recent memory. Mortgages are being underwritten, suitability assessments triaged, incidents managed, and reports drafted with AI in the loop — sometimes invisibly, often through third-party vendors. As one panellist put it, AI is already "inside the fibres of your organisation right now. It's happening, and it's invisible." The question facing boards, compliance teams, and operations leaders is no longer whether to govern AI, but how to govern something they often cannot see.

This piece distils the discussion from a recent industry roundtable on the EU AI Act and regulatory readiness. It sets out what the Act requires, the common challenges firms are wrestling with, what they are doing today, what they should be doing, and where vendors such as KORE and Delta Capita can help close the gap.

What is the EU AI Act?

The EU AI Act is the European Union's flagship horizontal regulation for artificial intelligence. It introduces a risk-based framework that classifies AI systems by the level of harm they could cause and imposes proportionate obligations on providers and deployers. High-risk use cases — including many financial services applications such as creditworthiness assessments and fraud detection — sit at the heart of the regime, with the most stringent obligations around data quality, transparency, human oversight, technical documentation, and ongoing monitoring.

Implementation timelines have shifted recently, with high-risk obligations now expected to bite around December next year. That extension should not be mistaken for breathing room. As one panellist observed, the announcement "reinforces the fact that we should be preparing for this, because if you leave it too long and the rules change quickly, you have to do more and more work." The Act does not sit in isolation either: it reads across to GDPR, the Digital Operational Resilience Act (DORA), senior manager regimes, and emerging supervisory expectations from global regulators. Each of these intersects with how firms acquire, build, validate, monitor, and explain their AI systems.

Just because you can use AI doesn't mean you should. It may not necessarily be appropriate. 

Common industry challenges

Despite the diversity of firms in the room — banks, insurers, asset managers, fintechs — a remarkably consistent set of pain points emerged.

Invisible AI, everywhere

Most firms accept that an accurate AI inventory is essential, yet very few have one. AI is no longer something a firm chooses to deploy; it arrives implicitly through vendor stacks, productivity tools, and partnerships layered on top of partnerships. ServiceNow's tie-up with Anthropic, Anthropic's relationships with Microsoft, AWS and Google, and the cascade of fourth- and fifth-party dependencies mean AI is rarely procured deliberately. As one participant put it, when asked which AI tools their organisation used, most answer "we've got Copilot, we're fine" — missing the wider implicit footprint entirely.

The pace of change has broken the old playbook

Several panellists noted that firms have rewritten their AI policies three times in twelve months. Traditional model risk management — review the model once a year, validate it, sign it off — is incompatible with systems that learn continuously and that come in probabilistic rather than deterministic flavours. Compliance frameworks built for an industrial era of rules-in, decisions-out struggle when the underlying model is fluid by design.

Ownership and accountability are unresolved

AI as a "cross-cutting" risk often translates into nobody owning it. Compliance teams find themselves handed responsibility for a domain originally driven by IT, then escalated through risk, then escalated again to the board. First-line teams are reluctant to take ownership of AI risk because, unlike a defined product line, AI exposure has no obvious end. As one panellist put it, if you accept personal responsibility for AI, "you're not going to know where that ends. It doesn't end at the end of someone's business day."

Explainability, audit trail and the data foundation

True mathematical explainability of large models is, for practical purposes, unattainable. What firms can — and must — demand instead is visibility into the data and decisions surrounding AI use: what data was used, what events led to a decision, which model was applied, and a full audit trail before and after the decision was made. Without that, "ownership" is a word without meaning.

Cross-border friction and shadow usage

Firms operating across the EU, the UK, and North America face a regulatory patchwork that is sharpening rather than harmonising. Add to this the practical reality that staff at every level — and especially junior, AI-native colleagues — are already using AI on personal devices to draft reports and decks, often outside any sanctioned environment. Some US banks have reportedly built AI usage into staff retention targets, intensifying the pressure to adopt without the matching investment in guardrails.

Regulators are catching up too

It is not just industry that is behind. Supervisors are themselves working out how to police AI, and there is a knowledge gap between policy teams and frontline supervisory teams. The risk-based architecture of the Act is widely viewed as the right approach, but enforcement will lag capability for some time yet.

What firms are doing today

Across the panel, a pattern of early-stage responses emerged — sensible, but rarely sufficient on their own.

• Tightening governance and policy. Most firms have stood up AI policies, refreshed them repeatedly, and pulled compliance, risk, and IT into recurring forums. Some have built dedicated AI validation functions modelled on traditional model validation teams.
• Building model and tool registers. Firms are attempting to inventory the AI in use, though most acknowledge their registers are partial — particularly when it comes to third-party and implicit AI.
• Locking down corporate environments. Tools such as Copilot have been deployed in controlled enterprise instances to prevent sensitive data being used to train external models, while personal-device usage remains a visible blind spot.
• Investing in AI literacy. Some firms are beginning to roll out training analogous to AML training — mandatory, recurring, and stretching from junior staff to the board. Others are establishing AI advisory councils to support boards that openly acknowledge they do not yet understand the technology they are being asked to govern.
• Pulling DORA and AI workstreams together. Resilience and AI are increasingly being run in tandem, particularly around critical and important services, third-party risk, and incident management.

These are reasonable starting points. But they share a weakness: they remain process- and policy-led in a world where the underlying technology is decision-led and continuously evolving.

What firms should be doing

Six themes from the roundtable point to a more durable operating model for AI governance.

1. Lead with strategy, not compliance

If AI is approached purely as a regulatory exercise, firms will miss both the opportunity and, paradoxically, much of the risk. Strategy must come first, followed by policy, and only then compliance and control. The biggest cost is the missed opportunity — better products, better service, more competitive economics — not the regulatory bill.

2. Fix the foundation before chasing gadgets

AI generates noise. Each week brings a new model, a new application, a new vendor demo. The firms that will pull ahead are those that resist the gadget cycle and instead invest in the foundational layer: data clarity, data classification, data lineage, model validation discipline, and a defensible AI inventory. Get this right and roughly eighty per cent of the AI governance challenge becomes tractable.

3. Govern decisions, not just processes

Perhaps the most important shift articulated on the panel was this: stop trying to govern the workflow, and start governing the decision. It is impossible to inventory every AI tool, every micro-feature, every implicit agent. It is, however, possible to identify the decisions that matter — granting a mortgage, declining working capital, hiring a candidate, assessing suitability — and to wrap controls, evidence, and human oversight around those decisions, regardless of whether they were made by a human, a model, or a chain of agents.

You can't govern every single decision. You've got to zone in on the ones that actually impact the customer — that is what is common across every organisation.

4. Use the customer outcome as the organising principle

Every previous step-change in regulatory implementation — Sarbanes-Oxley, GDPR, CCPA, Consumer Duty — became tractable when firms stopped optimising for silos and started organising around customer impact. The same logic applies here. "Great technology should be in the service of superior customer outcomes, not customers in the service of superior technology." Customer outcomes also give the second line of defence the language it needs to bring first-line owners with it, rather than against it.

5. Build continuous monitoring and integrated compliance

Annual model reviews are not enough. AI requires continuous monitoring, real-time transparency, and cross-boundary visibility — across products, geographies, and business units. The historical preference for siloed compliance breaks down precisely because AI ignores those silos. Integrated compliance, long talked about and rarely delivered, has finally found its forcing function.

6. Treat AI literacy as a board-level discipline

Boards openly admit they do not know enough to govern AI. The remedy is not for every director to learn to code, but for organisations to embed AI literacy — focused on principles, impact, ethics, and appropriate use — from the most junior analyst to the chair. Training should be recurring, mandatory, and treated with the same seriousness as AML.

7. Start capturing the audit trail now

By the time the Act's high-risk obligations are fully enforced, firms will need a history of decisions, not just a forward-looking policy. The firms that begin capturing decision-level evidence today will be the ones able to demonstrate, retrospectively and at scale, what was decided, by whom (or what), on which data, under which policy.

Where vendors like KORE and Delta Capita help

Most firms in regulated financial services are not, and should not aspire to be, AI infrastructure builders. They are sophisticated consumers of technology, operating under intense regulatory scrutiny. That makes the choice of partner critical.

Delta Capita: the regulatory and operating-model partner

Delta Capita's regulatory and risk consulting practice helps firms cut through the AI Act's complexity in a way that connects strategy, policy, and compliance. Drawing on its experience supporting global financial institutions through DORA, Consumer Duty, GDPR and other major regimes, Delta Capita helps clients define the scope of AI in their organisation, identify the decisions that actually matter, design the ownership model across the three lines of defence, and build the practical frameworks that translate board-level strategy into supervisable operations. Importantly, Delta Capita's approach starts from the customer outcome — the connective tissue that breaks down silos and gives second-line teams the mandate to engage first-line owners constructively.

KORE: the AI control plane

KORE provides the technological backbone that the new governance model demands. Today, KORE operates as a system of governance and record for human decisions — intercepting decisions in real time, enforcing policy at the point of execution, triggering oversight where required, and recording every action with a complete audit trail. KORE is now extending that same capability from human decisions to AI decisions, becoming a mandatory gateway between AI agents and core enterprise systems.

In practical terms, that means every consequential AI-driven decision can be validated, intercepted where necessary, escalated when policy requires it, and recorded with a full audit trail. It is not a reporting tool retrofitted onto an AI estate; it is a live control layer at the point of execution. For firms wrestling with explainability, ownership, and supervisory readiness, this is precisely the foundation that makes "governing the decision rather than the workflow" technically feasible.

Organisations that establish an AI control plane now will move faster, and with greater confidence, than those that don't — because they'll have the governance infrastructure to back it up.

Together: strategy plus infrastructure

Delta Capita and KORE are increasingly engaged together because the problem itself is hybrid. AI governance fails when it is a consulting exercise without infrastructure, and it fails just as surely when it is a platform without an operating model. Pairing Delta Capita's regulatory and transformation expertise with KORE's decision-level control plane gives firms a single, coherent path from board strategy through to evidenced, supervisable execution.

A closing thought

The winners in the next phase of AI in financial services will not be the firms with the biggest models. They will be the firms that have built a controlled space within which AI can be adopted safely — a control plane that lets the business move fast precisely because the guardrails are real, evidenced, and live at the point of decision.

The EU AI Act is the regulatory forcing function. But the deeper opportunity is operational. Firms that treat the Act as a chance to fix the foundation, govern at the level of the decision, and capture a real-time audit trail will not just be compliant when the rules bite. They will be quicker, more confident, and more trusted than competitors who waited for clarity that, by then, will arrive too late.

The question is not whether you will need an AI control plane. It is whether you build it now, or spend the next two years catching up.