The opinions expressed here are those of the authors. They do not necessarily reflect the views or positions of UK Finance or its members.

The question at the top of mind for firms and regulators is whether results generated by AI can be understood easily, or will a choice always have to be made between models that work well and models that are explainable?

This is particularly true of comparisons between black box and white box AI models, where black box AI refers to systems, such as large language models (LLMs) that produce results using billions of parameters to inform their outputs, while white box AI is more binary, interpretable, and traceable but with limited parameters. The trade-off here is that white box models tend to be less powerful, and black box models are highly accurate but often seem opaque. Like Nazy Alborz Group Product Manager, AI, at Global Relay states: “False positives in surveillance review queues can lead to increased costs, as compliance teams must review every flagged message, often requiring additional personnel. AI-enabled surveillance reduces false positives and accurately pinpoints undetected real risks, which enhances the efficiency of the review process”. However, as pressure grows to deliver more accurate surveillance outcomes, firms may start to question whether unparalleled accuracy overshadows the need for explainability.

Global Relay’s 2025 State of AI in Surveillance report lists this tension as a key barrier to AI adoption by financial firms. 91 per cent of survey respondents rated explainability as highly important, with an average score of 9.14 out of 10, but many professionals also acknowledged that there are major accuracy gains from LLMs that are impossible to ignore. A proponent of this view is Global Head of Communications Surveillance at Barclays, Ugne Willard, who commented: 

“Accuracy outweighing explainability has to happen. Anyone who has played around with Gen AI- especially in the comms space- will see how much it could add on top of traditional methods”.

However, trust in AI remains an issue, where the report found that, on average, industry respondents rated their trust in AI to make context-driven decisions at just 4.92 out of 10. One survey participant stated: 

“You need to have both (accuracy and explainability), I believe it is very dangerous to have anything you cannot explain to a regulator or internally to a trader”.

Alongside this, a 2024 report published by the FCA and Bank of England revealed that only 35 per cent of firms that use AI within their systems are able to “confidently explain” how it works. Therefore, there is a long way to go to foster trust in AI and this starts with education about its benefits and transparency around how it functions to reach conclusions in a way that is smarter and more efficient.

The solution

Establishing a compliance program in which AI can be utilised effectively, without jeopardising critical components of transparency and accountability remains a challenge. Firms may benefit from integrating stronger governance frameworks that allow black box systems to operate within white box boundaries through human oversight (human-in-the-loop AI) to complement machine learning models, documentation and communication of algorithms to build trust and understanding amongst stakeholders, and explainability techniques. 

It is becoming more common to involve model governance teams, not just compliance teams, in the vetting of AI tools. As Senior Manager, Surveillance and Market Abuse at PWC, Hannah Bowery stated: 

“The key is having the vendor documentation and vendor information upfront when you’re choosing to implement these tools. That’s the key. Regarding explainability, from our perspective, we’re seeing clients coming to us to independently test and assess how these models are working. We are being approached by model governance teams – not just compliance and surveillance teams – regarding independent testing.”

Global Relay’s report found that the importance of a transparent third-party framework was rated at a 9.4 out of 10 on average, as compliance teams feel more confident when they employ models that have been carefully assessed and reviewed and are able to gain a holistic understanding around how AI models have been trained and how information is documented. This is because third-party providers have a large impact on a firm’s business operations.

The future of AI in surveillance

The ideal solution, moving forward, is not choosing one over the other, but lies in building a hybrid approach that balance both accuracy and transparency. Models that not only detect context-driven risk but offer a clear and auditable reasoning behind every alert and flag. AI in surveillance must be both accurate and accountable, and both thorough and transparent.