The opinions expressed here are those of the authors. They do not necessarily reflect the views or positions of UK Finance or its members.

If you think that the Bank of England’s new regulation on model risk management - SS1/23 Model Risk Management, which came into force last month, does not apply to you, then you may be in for a nasty surprise! 

On first reading, the Prudential Regulation Authority’s (PRA) supervisory statement seems to concern only those banks and investment firms which have “internal model approval to calculate regulatory capital requirements for either credit risk, market risk, or counterparty credit risk.” However, nowhere else in the document is this narrow definition repeated. Could it be applied to any models used anywhere in a bank?

Do you have the list?

I checked with technical officers at the Bank of England, and they confirmed that SS1/23 Model Risk Management principles apply to all models wherever they are used in the bank. My contact went onto clarify that this meant not just AI and not just models used to calculate capital requirements. He specifically called out marketing, fraud and credit risk as areas that ARE included in this regulation.

In a perfect world, all banks would have well documented, auditable and managed governance processes for all the analytical models they run. In reality, after more than 30 years of building, buying and using analytics models, we all know that even creating a list of them all is a huge challenge. Yet, that is exactly what is required to comply with SS1/23 – meaning that, as of 17 May this year, many banks may be technically out of compliance.

Beware the Iceberg

The problem lies not so much with the high profile, high-risk models used in the highly governed areas of treasury operations. There are well established processes for the documentation and management of models used to calculate capital requirements, for example. Following the requirements to include periodic independent audits of these models to check that they are well constructed, trained on unbiased data, and operating in line with acceptable parameters etc.  it is relatively more straightforward to implement. 

But, as with icebergs, it’s what’s below the surface that can hurt you. The problem comes with the potentially thousands of other models built and deployed all over the bank. And it’s not just the models you have built. As the statement makes clear, the expectations apply to all models used for business decision-making, risk management, and financial reporting, including those acquired from external vendors. This includes models embedded in third-party solutions such as customer segmentation or fraud prevention applications. 

The vital first step

The requirements from the PRA outline five core principles for Model Risk Management. The very first principle is to define what constitutes a model and maintain a comprehensive model inventory. Taking the first step is often the hardest. 

The regulation states that banks must make a board member responsible for model risk management to ensure sufficiently high-level scrutiny and focus on the issue. That board representative will need to seek assurance not only from analytics and data leads responsible for creating models, but also from business teams using them. After all, few data scientists focus much on the models they or their predecessors have already created – their time and energy is applied to creating the next models. 

It is individual business process leaders who need to ascertain which models are used where in their operations, and as noted above, also query any applications from third parties to see if they also rely on analytical models. 

That is, of course, only the first step as the principles then require assessment and management of those models in a transparent and auditable way. Nothing in the regulation is anything other than data science best practice – but we all know how often we follow best practice.

Risk and reward

If you are thinking that this sounds like a heavy lift, then you might be right. But these requirements must be the catalyst to extend the best practices most likely already in place in the most highly governed areas of your operations. It also establishes the foundations for future governance and compliance. Regulations are beginning to overlap; these ones that reference model risk controls; incoming ones that require evidence on the way AI models are trained plus well-worn BCBS 239 and Consumer Duty regulations applied to automated decisions, to name but a few. Having a definitive list of what models are used and where, will be the key to compliance with many of these requirements.

Creating and then actively managing that inventory of models is essential, but where do you start? By following the data: if you can trace the data, you can find the models that reference it. If you would like to understand how Teradata can help you be compliant to SS1/23, please get in touch with me.