individual-entry-BA-Blog

« More Market Consolidation | Main | October Rules Fest »

The Role of Predictive Analytics in the Sub-Prime Crisis

-- Posted by Carole-Ann

Being part of Fair Isaac, I have never really doubted the critical role predictive analytics could play in business.  I see real-life examples day in and day out but when I read the New-York Times article http://www.nytimes.com/2008/10/05/business/05fannie.html?partner=permalink&exprod=permalink, I found one that was painfully telling…

Let me point your attention to a few points that CHARLES DUHIGG makes in this investigation.  I will not comment though on the politics involved.

Fannie Mae has been playing a crucial role for the lending industry as we all know.  Their early success relied on their ability to predict which borrowers would be able to repay, assessing the premium required to compensate for the risk they took.  With an effective model, you can pinpoint good risk versus bad risk and therefore make safe decisions that ensure the business will prosper.  This is pretty much the essence of predictive analytics: once you know the probability to repay for people with a given set of characteristics, you can extrapolate and estimate how much reserves you need to build in order to beat the odds that you will not get you money back for each population segment.  This is how premium are calculated.

Unfortunately with new types of mortgage product and the lack of associated data, Fannie has not been able to produce a robust predictive model.  Not being able to tell how borrowers would behave in the long run, there was no way to estimate how much risk they were exposing themselves to.  When you combine that with the absence of a CRO (Chief Risk Officer), which is like driving with your eyes closed, no wonder we got where we are.

This is fairly atypical black and white example but it drives the point: with a robust model, you thrive / without a model, you’ll hit a wall… eventually.  I am not saying that predictive models are the only way to make good decisions but they definitely help manage the uncertainty.

One might wonder how they could have built a robust model in absence of data.  This is a fair question.  Well, how about better integrating a feedback loop into the system to detect the early signs of the meltdown?  This is what we call Decision Improvement.  Another potentially complementary approach might have been to take a less aggressive stand: knowing that we don’t know for sure, premiums could have been calculated with more room for caution.  This is where business rules and predictive models work hand in hand.

Again, this is not taking into consideration other facts that influenced the situation such as the mandates from Capitol Hill or the laxity of credit agents.

First time on the EDM blog?
Subscribe to the EDM blog feed or check out some other recent posts:

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451629b69e2010535dd8317970c

Listed below are links to weblogs that reference The Role of Predictive Analytics in the Sub-Prime Crisis:

Comments

Aleksander Dragnes

Some banks did have predictive analytics, but decided to lie to them:

http://bits.blogs.nytimes.com/2008/09/18/how-wall-streets-quants-lied-to-their-computers/

We probably need a really statistics-savvy governance function to stop such things.

--
Aleksander

FICO

You are totally correct. I am expecting some serious regulations in the near future, especially in terms of governance.

Neil Raden

When it comes to analytics, Wall Street is clearly the leader. The best of the best head there after school to grab six-figure starting salaries. Some even see seven figures, based on their performance. They are the rara avises, the crème-de-la-crème, and whenever we speak about "Competing on Analytics," it goes without saying that Wall Street analytics represent the exemplar of what is possible for an analytic culture.

So why is Wall Street melting down?

Clearly, analytics aren't everything. Our financial system is pretty complicated and subject to abuse and fraud. The current crisis is aligned with the greed of the mortgage brokers and the mortgage bankers. Once in a while the financial press will point the finger at the hedge funds and investment brokers that shoved mortgage-backed securities down the throats of other investors.

Hmmm.

Wasn't anyone watching this? After all, interest rates started to creep up a few years ago, the economy started to turn down, default rates started to appear at around the same time. Is it possible that the quants were so buried under leveraged layers of derivatives and other exotic instruments that they didn't see the coming storm? This seems like a pretty big movement to miss. After all, if you're sitting on top of a few hundred billion in debt that is on the razor's edge of liquidity, wouldn't you spend some time looking at it more closely, especially with such broad macroeconomic factors staring you in the face?

Maybe the problem was just that — too much attention to the monetary and business-related factors and not enough attention to the movement of markets on a broader scale.

In the early '70s, I had the unique opportunity to take economics classes from the legendary (but until recently, obscure) Hy Minsky. Minsky is known for the "financial instability hypothesis," which proposes economic expansions become unsustainable booms ending in crisis and economic unraveling. Speculation. Greed. Disaster. I first heard the phrase "Chaos Theory" from Minsky thirty-five years ago.

Minsky has suddenly become very popular (unfortunately he passed away in 1996). One of his memorable quotes in class (there were many) was: "All panics, manias and crises of a financial nature have their roots in an abuse of credit." He used the Dutch Tulip mania of the 1600s as an example. He believed that financial systems experience rounds of speculation that, if they are severe, end in crises. Minsky was considered a radical for his stress on their tendency toward excess and upheaval.

Minsky showed that bubbles are an inevitable result of market activity. Buyers who show gains with a successful strategy encourage other buyers until it stops working. When investors have to empty their portfolios of even their prime holdings to cover their positions, markets start to circle the drain. At that point, the "Minsky moment" is obvious.

And it's here — Bear Stearns, Countrywide, Lehman Brothers, Merrill Lynch.

So the question is, why didn't the best and the brightest see it coming? We need to do some soul-searching. Is there really any benefit to advanced analytics and an analytics culture if doesn't see the train coming through the tunnel? Or is it something else? I think that no one wants to believe a "Minsky Moment" is coming.

You only see what you're looking for.

-Neil Raden
twitter: nraden

Aleksander Dragnes

Neil,

There is a book by Nassim Nicholas Taleb, a former trader, called "Fooled by Randomness" that sheds some light on this. Before the credit crunch we've had a long period of expansion and very cheap money. Many of the analytics used worked remarkably well in this environment, but they had not been tested properly for different market conditions and under rare events. As the article I linked to above mentions this was in some cases intentional.

Adding to this many of the financial instruments were new and there was no real world experience on how they would behave when the economy soured, and of course it was not only the analysts who lied to themselves when testing their models, but also mortgage brokers trying to flip as many mortgages as possible. The GIGO (garbage in, garbage out) principle holds just as much true for analytics as for any computer program.

--
Aleksander

wilbur

I am glad Aleksander quoted Nassim Taleb. He brings a dose of reality to all this.
The key here - the way I see it - is that it is in fact as important to manage risk by being good at handling what we don't anticipate than to manage risk by attempting to predict.
Predicting is tough, and even tougher - this is I think Taleb's point - is to understand what we are predicting and measure the limitations of the conditions under which we predict.
But handling a "black swan", on the other hand, may be easier - less subject to bias - and yield much better results when the "black swan" occurs.
Somebody at BRF actually made that point in one of the panels.
All this to say that decision management will continue to be key, and it won't be just rules, just analytics.

Neil Raden

Aleksander,

Did you see the excellent article by George Soros in the New York Review of Books? Here is the link: http://www.nybooks.com/articles/22113

Soros borrows some of Minsky's ideas (without attribution, sadly) and coins his own term, "reflexivity" to describe how supposedly efficient markets act poorly and other actions support it. He even refers to the mortgage crisis as a bubble within a "super-bubble." Provocative piece, but not completely original.

Yes, I know Taleb's work. Too bad more people don't. I will take issue, though, with something you said. I still believe that the problem is that you can't see what you're not looking for. If you believe in market fundamentalism, you will likely miss these dysfunctional tendencies until it's too late, whether they involve new or mature mechanisms.

Neil Raden
twitter: nraden

The comments to this entry are closed.

Search Site


  • dmblog.fico.com

Subscribe

  • enter your email

Upcoming Events

  • FICO Tools & Analytics User Forum 2012
    BERLIN: September 11-12, 2012 LONDON: September 18-19, 2012 Gain new insights for improving business performance through advanced analytics and decision management tools.