The Policyholder Puzzle

Why is modeling policyholder behavior so damn hard, and what can we do about it? A bit of background on why I’m asking this. I recently had the privilege of leading a study of dynamic lapses for general account products across 10 companies. For the first time since many investment-oriented products have come on the market, a significant shift in interest rates and rapidly evolving equity markets have made it possible to directly observe policyholder reactions, and it was too good an opportunity to pass up. The study was both fascinating and frustrating, and I’d like to set aside the actual results of the study and focus on thoughts regarding technique.

First, a little bit of background on where the industry is coming from. Until recently, actuaries had to construct dynamic lapse formulas with little to no tail behavior data to fit to. If you are reading this and have a high degree of confidence in your formulas, please write me a note! The rest of us, in my experience, have been forced to make educated guesses with limited data, leaving us with a high degree of model risk. Additionally, a broader trend toward predictive analytics techniques opens the door to directly incorporating dynamic behavior within those algorithms.

Based on the team’s work with the study, we assert that newer techniques, combined with careful actuarial judgment, have the potential to produce more accurate estimates than most models we have seen in the industry.

Features of a good dynamic lapse formula

A dynamic lapse formula¹ is typically integrated into actuarial models, often stochastic models that incorporate exogenous market factors. Given this, what should I expect from a good formula?

1. It should explain the data we do have, however limited.
2. It should incorporate professional judgment in areas where data is lacking.
3. It should use input variables that the model can access (usually from the in-force file or the economic scenario generator).
4. It should be computationally efficient.

In other words, if you have an excellent formula, but it uses variables that are not included in your actuarial models, it’s not very useful. Another example is artificial intelligence machine learning (AIML)—you may be able to train an AIML model to predict lapses, but what exactly should you do with the AIML model? Can an actuarial model access it on the fly, and does it pass No. 3 above? We will return to this topic later in the section “Alternatives to a GLM.”

Where our GLM approach left us hanging

Indeed, we used a generalized linear model (GLM) to develop the dynamic lapse formulas in our study. The GLM approach allowed us to quantify the existing relationships to both market and nonmarket variables over the last few years (for greater explainability, we actually split this into one GLM for nonmarket variables, then a second GLM for residual market-related behavior). For some products, we found very clear relationships that matched our intuition and, therefore, refined our understanding of policyholder behavior. For others (products with Market Value Adjustments (MVAs), for example) we had a much poorer fit and were left with as many questions as answers.

Four specific questions we were left with:

1. Is this a one-time effect, like a shock lapse relieving pent-up demand, that would then come down even if interest rates were to remain elevated?
2. How do we extrapolate from the current environment, where the biggest shifts have been in the short-term rates, to more typical changes in the level of the whole interest rate curve?
3. How much of what we observed was due to the pandemic and the many other (hopefully) unique circumstances of the study period, and can one dismiss such effects when modeling future dynamic behavior?
4. How much of what we observed was due to new product offerings and opportunistic agent behavior? And how should we predict such future behavior?

These questions all point to an acknowledgment that we observed a single market path, and that we shouldn’t get too excited thinking we have all the data we need to make definitive conclusions. (I get excited anyway, but at least I know better.)

Let’s talk a little more about each of these questions.

Is there a shock lapse effect going on?

Probably, but our data was not sufficient to prove it. This has been theorized. Termed “burnout,” it’s where a number of people take advantage of the higher interest rate environment, and then the residual population that’s left after a couple of years doesn’t react strongly to market conditions. To study this properly, you need to pay attention to distribution channels and how insurance agents interact with their clients. Many policyholders don’t pay attention to their financial products on a day-to-day basis, but their agents will reach out to them when there are new offerings on the market at more attractive rates. The agents will eventually run through their respective cohorts, at which point the overall lapse rate should theoretically revert to some equilibrium.

This also goes to the heart of my frustrations with the GLM-based approach. A basic GLM will determine a fixed relationship, such that a particular interest rate level will result in a particular number of lapses. The reality is far more complex and path dependent.

If one wanted to retrofit this burnout concept into a GLM, it is possible but runs a risk of overfitting due to the higher number of variables required. In other words, you can make the lapse at time t a function of interest rates at times t, t-1, t-2, t-3. It should be possible to additionally constrain the lapse sensitivity parameters such that they are monotonically decreasing. It should also be possible to have a predefined shape of burnout, e.g., linear, which can then be modeled with a single slope parameter rather than separate parameters for t-1, t-2, etc.

As of the time our analysis was done, I don’t think burnout could have been modeled based on the data, and it would have been a judgmental add-on. At this point, it is, in fact, possible that higher interest rates have been in place for long enough that we can incorporate them into the GLM and possibly even observe them directly in the data. Alternatively, it may be better modeled using other techniques, as described below in the section “Alternatives to a GLM.”

Which interest rates do we care about?

I’m mostly used to seeing the 10-year Treasury as a benchmark for the “competitor rate” in dynamic lapse formulas. I don’t know the history of how this practice emerged, but it’s safe to acknowledge that this is very simplified. In theory, when interest rates rise and a policyholder wants to take advantage of that, actuaries have assumed that the policyholder would hop over into a similar product at higher rates. Suppose we have a fixed indexed annuity where the general practice is to invest with an overall duration of seven years and an average rating of BBB. I would think that the competitor rate should then be tied to seven-year BBB bonds, as this will be most highly correlated with what competitors are crediting.

Layered on top of that is what we’ve seen in the last few years. Unlike most historical interest rate movements, we saw a flattening of the yield curve where shorter-term rates rose more than longer-term rates. This resulted in a degree of policyholder lapses that are difficult to explain using seven- to 10-year rates; in fact, six-month Treasuries gave us a better fit in most products. Our simple understanding is that this happened because policyholders hopped over into a wider range of products than we expected, like high-yield savings accounts, taking advantage of short-term rates. The reality is far more nuanced, with policyholders reacting to a large number of inputs, including headline news, insurance agents, family, personal liquidity needs and more.

Does this mean that everyone should be using six-month Treasuries as the basis for our dynamic lapse formulas? Probably not. This was unique to this particular time period, and our modeling of the future should reflect the fact that parallel shifts explain over 80% of historical interest rate movements.²

In our GLM modeling, we were able to improve our fit by incorporating a variety of rates, but ultimately followed the principle of parsimony and settled on single rate with the best fit across the most products: the six-month Treasury, with no lag. This was purely for the purposes of maximizing fit; setting the actual competitor rate(s) intended to be more predictive of the future would be a second critical step in the assumption-setting process.

How much of what we observed was unique to this period?

We are all aware of the changes that have occurred in many people’s lives since 2020, apart from the market movements I have been discussing to this point. There was COVID-19, working from home, government relief and stimulus spending, a mental health crisis and significant changes to residential real estate markets. I won’t offer any answers to this one; we barely had enough data to create a predictive formula based on market data. One would likely need a much larger number of scenarios to play out before sufficient data existed to isolate the impacts of market variables from these other factors. This is yet another reason that actuarial judgment will continue to play a significant role in setting dynamic lapse formulas.

How much of what we observed was related to new product offerings?

A number of companies reported that they were losing policyholders to newer offerings. An example would be premium persistency bonuses that offset surrender charges or MVA losses. I also don’t have a way to directly isolate the impact of newer products. However, I also think that we can expect companies to behave opportunistically in the future, which means that it may be safe to disregard new product offerings when setting up a GLM (in mathematical terms, I am arguing that the effect of new products is highly correlated with the rise interest rates, and therefore, a regression should actually not include both variables).

Alternatives to a GLM

The above discussion largely relates to the difficulties and uncertainties that we dealt with when fitting our GLM process. I also want to pull back for a moment to consider whether we’re investing in the right approach altogether. When you begin your own analysis, here are other approaches that you can consider.

Machine learning. AIML is worth exploring for any high-dimensional problem, and this qualifies. Machine learning can parse all of the relationships and likely create a stronger prediction than our GLM. The main downsides to an AIML model are (a) difficulty explaining or interpreting the model’s results, (b) practical difficulties retrofitting it into a typical actuarial modeling process, and (c) difficulty layering actuarial judgment onto the resulting function. Of course, the way around (b) is to use the AIML model as more of an educational tool, but that brings us to problem (a). (Note that Python and R have a number of libraries aimed at explaining the predictive power of variables, even coming from AIML models, so this is not insurmountable.)

Agent-based modeling (ABM).³ This has a wide variety of applications where the overall behavior is an emergent property of a complex set of underlying rules operating at the individual agent (policyholder) level. GLMs allow us to account for a much larger number of variables than a more traditional experience study process; ABM would be a similar leap over GLMs, allowing for more complex and granular relationships. As I mentioned, our GLM simply could not capture the full range of effects for certain products without using enough variables that we considered it to be overfitting. ABM allows for complex underlying rule sets, with the potential for both a higher fit on the same variable set, and possibly the ability to make better use of more market variables. Two main downsides to the ABM approach are (a) this approach may need significant investment before we learn how to apply it successfully, and (b) it may require more data than we have available to successfully fit the many underlying relationships that need to be modeled.

Final thoughts

I believe we should all be careful not to underestimate the importance of actuarial assumptions to insurance companies. This process, hidden away in the heart of the actuarial department, drives product decisions and company valuations and often determines which companies succeed in the market. Prior to 2022, there was simply not enough data available to fit robust dynamic lapse formulas. We now have a modest amount of data available, and it is well worth investing in and improving our predictive capabilities to help management make smart decisions.

That said, we can expect to face continued challenges. Our experience data is still limited, and we are attempting to predict relationships in both ordinary and extreme environments. However, with newer techniques, I believe we can still make a large step forward in predicting behavior and quantifying uncertainty.

Statements of fact and opinions expressed herein are those of the individual authors and are not necessarily those of the Society of Actuaries or the respective authors’ employers.

Copyright © 2025 by the Society of Actuaries, Chicago, Illinois.