Written by:
Simon Hills, Director, Prudential Policy, UK Finance

I was delighted last week to host a UK Finance webinar on ‘Model Risk Management – Effective practices for stress testing’ with expert colleagues from KPMG and Thomson Reuters.

The PRA had released its Policy and Supervisory Statements just  a couple of weeks before, so it was very timely and attracted a large number of participants. You can listen to the webinar below.

The Supervisory Statement is a synthesis of model risk management (MRM) good practices that the PRA has observed during the annual cyclical stress testing process, which is applied to the largest UK banks. The four principles are intended to be relevant to all model types, not only those used in a stress testing context. And similarly, the PRA’s expectation is that other banks and building societies should draw upon them, in a proportionate way, depending on their size, complexity, risk profile and the materiality of the model, as their own stress testing practices mature.

We discussed the evolution of stress testing, which is at its most advanced in the US where the Fed’s statement SR11-7 provides quite a broad definition of what a model is. It requires banks to separate out model use and development from validation and independent review, maintain an inventory of models used within the banks and ensure that an effective governance process is in place.

So, it is welcome that the PRA’s Supervisory Statement, against which banks will have to assess themselves in their ICAAP process from 2019 onwards, is broadly aligned to the Fed’s views on stress testing.

During the webinar we posed a number of questions to the audience, the answers to which reflect the relatively recent nature of the debate on model risk management in the UK. For instance, 40 per cent of respondents felt that they still had some way to go to achieve compliance with the Principles. IT infrastructures and data quality were identified as the number one challenge for banks as they implement them.

But perhaps the most interesting question asked was ‘to what extent are components of your MRM framework (e.g. model documentation, testing, inventory etc.) currently automated?’. Almost 70 per cent of those that responded said that their MRM automation processes were as yet in the foothills. Again, the weight of regulation and internal changes means that automation is a thing for the future, as simpler solutions and low-hanging fruit are currently the top priorities. The challenges for the future include grafting machine learning on to legacy systems and assurance about how machine learning reaches its conclusions – issues that are also discussed in the UK Finance Sustainable Finance in a Digital Age report released today, which discusses the benefits of using machine learning to speed up lending decisions, while potentially reducing credit risk.

Overcoming those challenges should bring rich rewards as automated documentation and the use of challenger model benchmarking or auto validation for statistical models becomes more prevalent.

We live on the brink of significant progress in the introduction of machine learning and artificial intelligence to modelling. The webinar concluded that machine learning will be more accepted and adopted by banks only once regulators themselves become more au fait with the benefits, and risks, that they can bring. UK Finance will be at the forefront of encouraging this dialogue.

Managing a Model Risk