Can Big Data Make Risk Tolerance Questionnaires Obsolete?
This week, we remember a seminal event in the history of artificial intelligence. It was the twentieth anniversary of world chess champion Garry Kasparov’s loss to IBMs Deep Blue supercomputer.
I was fortunate to be in attendance when Kasparov gave a keynote speech at the 2017 Envestnet Advisor Summit, where he spoke about his latest book, Deep Thinking. The book is the first detailed account from Kasparov of his battle with the IBM machine as well as his recommendation that we should be prepared to adjust our lives to future advances in technology.
With every new encroachment of machines, the voices of panic and doubt are heard, and they are only getting louder today, Kasparov writes.
While the ability to play chess does not, in itself, make a computer as intelligent as a human, this milestone was just one in a long string of examples where a machine outperformed a human in a specific task. As more and more complex tasks are taken over by software, we should be looking ahead to the next breakthrough and considering the benefits as well as the potential drawbacks.
This started me thinking about how artificial intelligence combined with Big Data could improve wealth management. We have witnessed the automation of almost the entire end-to-end wealth management process, from proposal generation to new account opening, portfolio rebalancing, and performance reporting all the way through to billing.
But one key aspect has stubbornly remained a (mostly) manual process and highly dependent on human input: Risk tolerance questionnaires (RTQs).
The purpose of the RTQ is to create an accurate risk profile of the client, which becomes the primary factor in selecting their investments. Yet, the entire industry still forces clients to take what amounts to a multiple-choice pop quiz, one for which they have not studied but relies on their answers to build a portfolio that they may be invested in for decades.
Replacing RTQs has been on my mind for a while, and I have written about a number of reviews of different risk profiling software. Now seemed to be an excellent time to explore the idea of whether the concept the RTQ could, at some point, go the way of the Dodo.
The Sorry State of Risk Tolerance Questionnaires
My friend, Michael Kitces, wrote an aptly titled blog post, The Sorry State Of Risk Tolerance Questionnaires For Financial Advisors:
the reality is that it’s difficult to measure the subjective aspects of risk tolerance itself, simply because it’s the representation of an abstract psychological trait in the first place. In other words, we can’t just objectively look into someone’s brain and figure out what their risk tolerance is. Instead, we have to ask questions, evaluate the responses, and try to figure out how clients feel about their willingness to take risky trade-offs, and how they perceive the risks around them.
Unfortunately, though, many risk tolerance questionnaires (RTQs) don’t actually do a very good job of helping to predict a client’s actual investment behavior during volatile markets, particularly when they ask about how the investor believes he/she would behave in the event of a significant financial loss. In part, this appears to be due to differences from one investor to the next as to what constitutes a risky and undesirable loss in the first place, which can be based on sometimes-arbitrary reference points.
Before we continue, let’s break out the three key components of the risk profile. I’m paraphrasing below from the Kitces article:
- risk tolerance – the client’s willingness to take on risk
- risk capacity – the client’s ability to endure a potential financial loss and still be able to achieve his/her goals
- risk perception – the client’s understanding of how risky they think their investments are
There is a wide variety of risk profiling tools that all take varied approaches to the above components. Some focus primarily on risk tolerance, while others try to capture a combination of all three using different techniques. I’m going to skip all the details of the different techniques for assessing risk. Kitces does an excellent job of this in his article.
What we do know is that most risk tolerance questionnaires suffer from three problems:
- They are overly technical in nature;
- Response bias, which results in an investor responding differently when stressed;
- The difference between risk tolerance and risk capacity is not well-understood by clients or by many advisors, for that matter.
A 2012 study of risk tolerance questionnaires showed that many software products failed to accurately predict client behavior when the clients were under pressure.
What I would like to discuss is the possibility of automating the entire process and cutting the client out of the loop. Is this possible? If so, would it be recommended or would clients accept it? While some advisors and most of the software vendors are against it now, time and technology may soften their views.
To try and gather a range of opinions on this topic, I spoke with representatives from some of the leading providers of risk profiling software:
- Aaron Klein, CEO, Riskalyze
- Tyler Nunnally, US Strategist, Finmetrica
- Hugh Massie, CEO, DNA Behavior
- Mark Friedenthal, CEO, Tolerisk
- John Ndege, CEO, Pocket Risk
I was also able to speak with Thomas Oberlechner, Founder at FinPsy LLC, a consultancy specializing in behavioral research.
Can AI + Big Data Replace RTQs?
To restate my original question: Could artificial intelligence combined with big data provide a better understanding of an investor’s risk profile than a questionnaire?
I’m thinking that companies like Yodlee, Quovo, etc. could provide access to an enormous quantity of very useful information about a prospective client. It could include the client’s entire investing history plus their personal finance history, including every transaction they ever made, and throw in social media, too. Could an algorithm be written to develop a personality profile for the client that would be a better representation of how they would deal with different market scenarios?
Let me begin the discussion with my favorite vendor response. This is from John Ndege of Pocket Risk:
We will have a manned mission to Mars before an algorithm replaces all risk profiling tools. Even with access to all the client data, it will take years (decades) to gain widespread trust and usage.
In response, a quote from Kasparov:
It’s remarkable how quickly we go from being skeptics to taking a new technology for granted. Given honest data, machines can make us into better decision makers.
How long did it take for us to switch from physical maps to GPS or to dump the Yellow Pages for Google? Not very long at all. Once we realize that a particular technological advance can perform a task faster, more accurately and/or with less effort, most people eventually accept it. Soon it becomes hard to remember what life was like without the technology. Anyone remember what it was like before smartphones were ubiquitous? (See How Risk Tolerance Software Is Disrupting Wealth Management)
Massie from Financial DNA agreed that technology cannot replace the questionnaire, but believes it can provide added value:
The right Big Data optimized with personality insights through an algorithm will provide a big leg up in the power and the ability to apply the behavioral insights across a whole data base of people. However, on its own it will not achieve the level of accuracy if a correctly structured pyschomterically designed questionnaire is used.
Can Humans Be Objective?
My original thought that an algorithm could do a better job at estimating someone’s risk profile was because of the extensive research that shows that humans are unable to answer financial questions objectively. Our innate biases always cloud our judgment.
One of the challenges with RTQs is that they are heavily based on the individual’s personality, or perceived willingness to accept investment risk, explained Tolerisk’s Mark Friedenthal. [C]cognitive biases have been fairly well documented to suggest that a person’s recent experience (such as a positive or negative stock market or portfolio experience) will influence how they answer personality questions about taking risks, he noted.
In other words, Friedenthal believes there is no completely objective way to translate someone’s personality into a risk profile just by answering questions. This is why his software includes a client’s cash flows into the process.
As told by Kitces:
The situation is further complicated by the fact that when we take RTQs, we tend to answer the questions calmly and rationally, but when risky events occur, we may respond emotionally (literally using a different part of our brain). Known as the dual self or dual process theory, this disconnect between how we react to risky events in real time, and our (rational) expectation of how we will react, makes it challenging to simply ask consumers (in the hopes of getting a good answer) about their tolerance for taking future risks.
So, why not eliminate people from the process entirely? Why not go full machine?
Understanding The Data
Well, it may not be that easy to correlate the data with human intentions. Even if you have every financial transaction someone ever made, there is still ambiguity to deal with. How do you accurately translate transactions into an investor’s preferences 
As Aaron Klein of Riskalyze points out:
When you look at a log of trades, holdings or spending activity, are you looking at the client making their own decisions, or were those decisions executed by [an advisor] on behalf of a client? Does the client embrace that decision still, or do they regret it?That’s the real problem, we see the answers to those questions are not evident in the records of transactions, and thus, it’s really difficult to ascribe risk tolerance to someone based on that data.
Even with the complexities of analyzing the data, there must still be a value to the data that can be mined by the right algorithm. Artificial intelligence has tremendous potential because of all the variables involved in determining risk capacity, Finmetrica’s Nunnally pointed out.
Nunnally continued:
It is very difficult to ask an investor how much risk they can afford to take in a questionnaire because most people don’t know. There are so many variables involved, including time horizon, and countless complexities that relate to future projections like investment returns, income over time, health concerns, etc. that it goes well beyond what the human mind can fully comprehend.
AI is able to crunch numbers and decipher patterns that can lead to investment optimization that far surpasses what humans are capable of.
The passage of time throughout the data set also needs to be taken into consideration. Some decisions to increase or decrease risk are prompted by changes in the expected time horizon of usage, Friedenthal noted.
While it is true that an investor’s decisions when he/she is 25 will be much different from those when they are 55, I think there still should be some way to adjust the results to compensate for this.
Computers Plus Humans = Something Better
Perhaps the answer is not computers alone, but a combination of human intelligence enhanced by machine intelligence
Soon after Deep Blue beat Garry Kasparov, a new type of chess tournament called freestyle became popular. In freestyle events, teams compete using any combination of humans and computers. Kasparov observed that the teams of human plus machine dominated even the strongest computers.
This is echoed by Massie from Financial DNA:
The holy grail of behavioral insight is aligning and blending demographic and financial related big data (the external view of the client) and psychometrically measured personality traits (the internal view of the client), and then deploying tools to utilize the blended information to guide the planning process.
Massie believes that firms could leverage big data to give advisors a head start on knowing the client during the prospect phase. But he warns against building the planning process solely on data:
The advisor/firm will never know the complete picture until they have the client complete a psychometrically designed assessment, which if structured correctly will provide a strong insight to client emotions. Two clients can have similar Big Data attributes but a very different personality, which means they need to be communicated with differently and offered different solutions to meet different risk profiles and suitability requirements.
(See 3 Ways Personality Testing Crushes Risk Tolerance Questionnaires)
Thomas Oberlechner from FinPsy LLC also insisted that computer learning and expansive databases must be combined with specialized questionnaires in order to be effective at determining an investor’s?underlying behavioral traits and tendencies:
These are two main pathways to determine behavioral characteristics, both of individuals and of companies. You can collect new information by asking directly (for example using surveys) and you can analyze existing expressions or symptoms of past behavior. These two layers of information, explicit and implicit, complement each other. Moreover, possible differences between the two are actually important in understanding risk tendencies and dispositions more fully.
Of course Big Data analysis and AI can help determine behavioral characteristics and preferences. But you need to be able to integrate statistical applications with behavioral science expertise (personality, individual differences, motivation and cognition) to generate the most meaningful insight.
I’m not surprised that firms whose main product or service is built around questionnaires, either behavioral, psychometric or otherwise, are insisting that machines cannot replace them, but must be used in conjunction with them.
Friedenthal offered an alternative option:
Perhaps the better usage of [big] data might be in objectively analyzing savings and spending patterns. Consider that a thoughtful analysis of risk tolerance includes not only a measurement of personality (willingness to accept risk) but also the expected chronology of cash-flows (ability to take risk). Individuals often represent expected cash-flows based on memory and not a data driven account of their actual inflows and outflows. These imperfections may contribute to inaccurate analysis of their ability to take risk, which translates to a sub-optimal risk directive.
If your risk profiling process includes asking the client about their cash flows, then having direct access to their investment and bank accounts would provide you with much more accurate data than the person could provide from memory.
The Future is Now
To demonstrate that this concept is not as far in the future as some people think, Massie noted that his firm has already developed some algorithms for calculating an investor’s risk tolerance and risk capacity based on their financial history. They have even performed some analysis on the accuracy of the results, as Massie reported:
Big Data can predict [risk tolerance] to around a 30% level if the right Big Data attributes are used, although this can be increased to up to 60% if there is some data base optimization using behavioral insights. However, adding a correctly structured RTQ to the mix will lift the accuracy to 80%+
This is exciting news! No word on whether Massie’s firm is planning to deploy these new algorithms or if they are just an experiment. But one thing is clear, if they are released, they will be as a compliment to questionnaires, not a replacement for them.
Massie stated:
Overall, we do not think that RTQs should be eliminated as they reflect the internal view of the investor’s risk tolerance. Big Data (if accurate) will reflect the external view and particularly [the investor's] financial capacity. So, the RTQ and Big Data are designed to measure different risk profile elements. If both are used in tandem it would be a lot more helpful than if only one or the other were done.
Finmetrica’s Nunnally agrees that RTQs should stay:
At this point in the evolution of AI, an algorithm may be able to analyze transactional data to develop a profile relative to an investor’s personal financial history, but it cannot truly understand the emotional and psychological aspects that are uncovered through the course of a conversation (between the advisor and client]. In that regard, risk tolerance assessment is best served through a RTQ that is centered on human interaction.
A New Age of Human and Computer Cooperation
This is the point of the article where I usually include some generic statements about not being able to know what the future holds or if things continue this way we might see a major change, but then again it’s possible that everything will remain the same.
Not this time.
I’m going to predict that we will see some form of Big Data analytics included in risk profiling software becoming mainstream within the next 3-5 years. It will be a blending of questionnaires, data analysis and advisor insight to create a risk profile.
Even though only Financial DNA said they actually had something ready, all of the other vendors said that they were doing research into this area. If Financial DNA’s product starts to take off and becomes a reason that firms switch from competing products, we will quickly see a wave of similar product announcements.
I will leave you with this quote from Michael Kitces, the greatest opportunity of improving risk tolerance questionnaires and overall risk profiling may be the way it helps financial advisors to better manage ongoing client relationships.
All of this software should be focused on helping the advisor to improve the overall client experience. Isn’t that what it is all about?
Leave a Reply
Be the First to Comment!