A guide to understanding and managing model risk

Model risk occurs from the use of models that have fundamental errors against intended use and design objectives and from models being used incorrectly or inappropriately.

Very often, issues remain hidden due to inadequate or flawed testing and validation. Sometimes, a pair of fresh eyes will spot long-standing problems. For example, during a recent model review, we identified incorrect coding, an incorrect data field being used and an out of date expense assumption – all on the same model!

Effective control and management of model risk requires a robust governance framework; i.e. a model risk management framework.

Despite the increasing importance of, and reliance on, models there has been little regulatory guidance on model risk. The Prudential Regulation Authority (PRA) has published** some model risk management principles for stress testing for banks and similar firms, but insurance-related material is limited to supervisory statement SS17/16, which focusses on Solvency II internal models.

But firms can’t limit model risk management to specific models or a subset of models. The framework should cover all relevant models across all business units.

In our opinion, there needs to be central model risk management framework that covers the following areas:

• Model risk governance: involves setting up the firm’s policy that clearly identifies the roles and responsibilities of different stakeholders and defines the standards and controls that must be observed when developing and operating models.
• Model risk appetite: sets out the extent of willingness by the board or management body to accept results from models based on the type of model and compliance with the firm’s model risk governance standards. For example, will the firm accept the use of models that do not fully comply with its governance standards and, if so, in what circumstances?
• Model risk identification: a key part of the framework is to identify, record and maintain a comprehensive model inventory capturing, for all models, risk ratings such as materiality, complexity and validation status etc., alongside more mundane items like model owner, users and functional area.
• Model risk monitoring and reporting: ongoing monitoring of models is necessary to ensure models remain fit for use. Monitoring includes assessing the need for model validation and maintaining appropriate documentation including limitation logs. Management information is required to ensure effective oversight. Reporting should cover, for example, key developments and limitations, the evolving model risk profile and recommendations from owners and external validators.
• Model risk mitigation: actions are needed when a firm’s model risk profile falls outside its risk appetite. Mitigation may include model development, use of expert judgement, or additional validation in light of emerging or changing risks such as pandemic risk.