Storytelling With Numbers

As a kid, I loved math. The answer was right or wrong, black or white. You could easily see where you went wrong if your answer wasn’t correct and could remedy that to arrive at the one expected

In college, I decided to abandon my calculus career path and delve into statistics. Statistics, I quickly determined, is not math but data—namely the analysis and interpretation of data. While I loved the black and white nature of math, the gray nature of statistics intrigued me, because of the narrative it allows. A percentage itself is black and white—40%, for example—but how it is presented, the stories it is used to convey are gray—only two out of five, or nearly half or less than half. The positioning begins the narrative.

If it wasn’t clear already, in many ways, the presidential election showed all Americans how easily data and statistics can get the analysis wrong. Data can be very useful, but it can also provide a narrative that may not be always correct.

Take the way in which the decline of defined benefit (DB) pension plans has been discussed. I believe valuable data has led to a narrative that’s a bit disingenuous. Although it is true that the number of defined benefit plans has decreased, this hasn’t translated to a loss of income for Americans, as is frequently implied. 

Dallas Salisbury, former president of the Employee Benefit Research Institute (EBRI), used to frequently point out that, although pensions are a good source of retirement income for workers who stay with one company for many years, they will not necessarily provide for those who switch jobs often. In fact, data show that most American workers, always have changed jobs multiple times over the course of their career. This means that, for most retirees, the corporate move away from DB plans has not had a significant—or potentially any—impact on their long-term retirement savings or financial well-being.

In addition to questioning the narrative that derives from interpretation, it’s important to be mindful of year-over-year reporting when the questionnaires change. This is because statistics are very much dependent on the inputs. Take EBRI’s recent criticism of the U.S. Census Bureau’s Current Population Survey (CPS). EBRI alleged that the recently released CPS, which estimated major declines in employment-based plan participation, miscounted how many Americans participate in employer-sponsored retirement plans. Workers with the sharpest drops in participation include older employees, higher earners and workers with larger employers, all of whom are actually those most likely to engage in retirement plans. EBRI says these miscalculations are due to a major redesign, in 2014, of the CPS’ questions pertaining to income. The Census Bureau initiated the redesign following previous research indicating that the survey misclassified and generally underreported income—particularly pension income. The EBRI reports that, while the changes seem to have improved the accuracy of data on pension income, they also resulted in the appearance of historically sharp reductions in the levels of worker participation in employment-based retirement plans. 

So what’s the point of all of this for you and the retirement plan industry? We’ve written in these pages about plan advisers and sponsors’ ability to evaluate retirement plans by leveraging industry data and company data. But the concept of a data narrative reminds all of us that, in order to properly leverage that data, we have to look at the inputs and be mindful of how the numbers are then interpreted. 

For example, when it comes to inputs, think about evaluating the use of catch-up contributions. If only 10% of participants take advantage of them, that might appear to be low usage. But what if that 10% actually represents 40% of all eligible participants? Then that would show strong usage.

At PLANADVISER, we believe it is incumbent on us to help you make sense of the deluge of research and commentary available in the industry every day. We write articles about research reports, and conduct and produce our own proprietary research. When we communicate what we’ve learned, we consider not just the numbers themselves, but how they are presented, and by whom they are being reported. Is a financial wellness company behind the research showing such programs work? Is an asset manager showing the value of active management? 

The more the industry looks to data to help address plan-design or demographics issues, the more important it is for all involved with the running of retirement plans to ensure that we are accessing the right research and that the numbers are being questioned where appropriate.