Putting Common Sense Back into Credit Scoring
Today's credit scoring system does a lousy job of predicting customer defaults. SAS's finance architect is out to overhaul it.
- By Ted Cuzzillo
- October 1, 2008
Even before the Wall Street meltdown that's been grabbing headlines recently, a financial expert has been asking a question of lenders that is equally applicable to BI professionals: are metrics running your life, or are you running the metrics?
Almost no one remembers when loan officers decided who could borrow money. They used their own judgment to pick worthy applicants, often guided by rules of thumb. Some went by "The Three B's: never lend to beauticians, bartenders, or barbers" or "The Three P's: never lend to preachers, plumbers, or prostitutes."
To lenders in the new age of computers, that was all too human. Computers -- still running on vacuum tubes in the early '60s -- running automated metrics could do better, said experts. In fact, they probably did do better than free-wheeling loan officers -- just as well-tuned BI does better than "gut feelings" -- but credit scoring has also failed in significant ways that, I suspect, parallel many BI failures.
Now one veteran risk-management banker has a better idea. Clark Abrahams recently saw the current credit system's shortcomings first hand when he left a successful 30-year career in banking for his current position as SAS's chief financial architect. By changing jobs and moving to another state, he paid more for his new mortgage despite an ample ability to repay and a good credit record. It made no difference to the system whether he had changed jobs because he had been fired or (in his case) accepted a promotion.
"That makes no sense," he says. "I'm all for data-driven solutions, but not to the point where it doesn't make sense."
He's on a campaign to replace the current credit-scoring system with one he calls CCAF, for comprehensive credit assessment framework. Any institution that adopts it -- none has so far, though four banks are now financing its adoption -- will put human judgment back into credit scoring, guided by metrics and business rules.
The current system -- also used by investors to rate many kinds of bundled loans -- looks at each factor in isolation, he says, and rates the risk of each without considering the context.
The system is somewhat responsible for today's sub-prime mortgage problem, he says. "Had CCAF been in place [when today's troubled mortgages were made], we would not be facing the current massive market bailout."
Back when lenders decided to judge borrowers more objectively, they deemphasized the traditional "Five Cs of Credit": capacity (generally, the cash flow available for payments), capital, collateral, conditions (the loan's intended use and factors that may affect its use), and the borrower's character.
Unfortunately, early model builders lacked sufficient data to predict defaults, the model's ultimate job. They found a convenient proxy in a borrower's rate of late payments. The new system threw those who paid late -- 90 days delinquent once, 60 days twice, or 30 days three times -- into the "bad pile."
However, that proxy was imperfect. Lenders soon observed that some borrowers who had been rated as a higher risk were actually wealthy and had a low risk of default. Late fees meant little to them, and they paid when they felt like it. "Some people don't care," Abrahams says. "Their late payments are irrelevant, just noise."
Common sense would have made lenders revise their model to give more weight to capacity and capital. Instead, they stuck with the errant proxy metric and threw out capacity and capital. They reasoned that if people who paid late had capacity and capital, such factors must mean little.
Another poor proxy, he says, is the number of credit applications one makes. "Why does how much you look for credit mean you're riskier?"
Overall, current scoring does a lousy job of judging borrowers. Scores are averages of factors considered separately, stripping the data of context. "If you have the identical givens you could have the same score," he says, "but if you have the same score, you could be vastly different."
"The whole thing is really in need of a fresh look," he says. "We need not let the models control us, we need to control the models."
CCAF would first use the traditional "Five Cs" to sort out borrowers. Borrowers would be judged primarily by their capacity, capital, and repayment history.
Under CCAF, a wealthy, high-income applicant who had never defaulted would be rated lowest for risk. Current systems, on the other hand, would penalize the same applicant for his moderate delinquency and lack of installment debt -- even if he had never defaulted -- and make him pay a substantially higher rate.
First-time borrowers could also benefit. CCAF could evaluate payment history for rent, utilities, and telecom. Ratings in CCAF would be simpler than current scoring. For example, each applicant's payment history would be put on a simple scale: good, fair, and poor. Three factors would be considered: whether he had defaulted within the last five years, the age of his credit file, and his delinquency record.
Capital and capacity would be rated on similar scales, and ultimately all the ratings would be rolled up to determine his likelihood of default. New applicants with little credit history could be judged instead on their history with utilities and rent.
Unlike current systems, CCAF will be easily adaptable, he says, to reflect changing conditions to be used in a wide range of credit evaluations. It will also be more accurate, flexible, and consistent than the current ones. With fewer models, it will be cheaper. It will be easier to understand, among other advantages.
"One should not stray from the basics. We're talking about high tech and our whiz-bang solutions," he says, "but what I'm preaching is returning to good common sense: looking at judgment, not letting the models run the show."
The sub-prime mortgage crisis has given him a soapbox, and he says, "I will not be deterred." I hope not. It's time to abandon faith-based analytics in lending practices as well as your own BI applications -- the kind that allow the kind of backwards reasoning and Catch-22 logic that puts faith in metrics instead of their human makers.