Practical issues in implementing Solvency II for Lloyd’s Managing Agents: Model Validation and Independant Review

Paul began the discussion by looking at current articles, guidance notes and principles relevant to independent review requirements.

It was noted that Article 124 of the framework directive requires firms to have a regular cycle of model validation and to include performance monitoring and review of the ongoing appropriateness of model specification. 

Further details are provided in Lloyd’s detailed guidance notes.  The section 5 level 3 text (available from Lloyd’s or ABI) on independent review covers the scope of the validation policy and discusses independent review and what its purpose and role should be.

The CRO Forum have also published an IM control framework and in principle 7 they state that firms should subject their models to independent review. Here they also talk about the need to demonstrate that the model remains “fit for purpose”.

The Lloyd’s Guidance provides examples of what a firm might look for in the area of validation (including Calibration, Validation and Profit Attribution). By way of example, when considering Data, the relevant investigation should include:

• How accurate is the data?
• Is it complete?
• Does it reconcile?

In terms of the methodology used, criteria should include:

• Does the data support the statistical distribution?
• How does this compare with historical outputs?

Other relevant questions include:

• Have the assumptions been validated against actual experience?
• How effective is the challenge between actuarial, risk and the underwriters?

Paul discussed the scope of validation activities which will include: model governance; data management; model development; ongoing monitoring and model re-validation. Outcomes include review of the effectiveness and adequacy of controls; strategic alignment and accuracy of outputs; investigation, research and development; supervisory review. He suggested that these reviews can be resourced from a number of areas including internal audit; risk and compliance monitoring; internal experts and/or; independent third party. 

The presentation then led to a discussion on the role of the independent reviewer and the challenges that this represents for internal audit.  This was of particular interest to the audience which was largely made up from representatives from Internal Audit.

The three key areas to address in the internal audit plan are:

1. Audit of the risk function;
2. Solvency II programme assurance i.e. - will it deliver? and
3. What is the role of internal audit in the model validation?

It was noted that firms may well face a shortage of internal resources to address these three areas and some will find that they will need to invest in recruitment, training and culture to bring the internal audit function up to the required level.

The internal audit function needs to think about what resource model it will deploy i.e. whether to a) build capability internally b) outsource or c) co-source.

Gerald then went on to discuss what lessons can be learnt from the banking sector. The points that he highlighted included:

• Getting early involvement of the validation team in the internal model development  process;

• Aligning the model with the strategy/business model;
• Ensuring that the review is thorough and independent;
• Ensuring that the audit committee challenges content and substance;
• Model review and validation should not just be on the calculation kernel;
• Documenting what has been done but also what hasn’t been done;
• Recognition that internal audit does not necessarily have the skills to get involved in the quantitative areas;
• Monitoring three key issues a) independence of the reviewer b) review should encompass end to end process c) quality of the documentation.

You can download a copy of the presentations here.

Our next Roundtable meeting will be held in June and the topic will be announced shortly.