We discuss adjustments a lot on this blog. Today there’s a remarkable example and brewing scandal involving first a computer error in which Moody’s incorrectly graded certain debt obligations. The real story comes after the problem was identified – as it is alleged that Moody’s altered its models to avoid having to fully undress its error in public.
On May 21, 2008, Bloomberg reports:
May 21 (Bloomberg) — Moody’s Investors Service said it’s conducting “a thorough review” of whether a computer error was responsible for assigning Aaa ratings to debt securities that later fell in value.
Some senior staff at Moody’s were aware in early 2007 that constant proportion debt obligations, funds that used borrowed money to bet on credit-default swaps, should have been ranked four levels lower, the Financial Times said, citing internal Moody’s documents. Moody’s altered some assumptions to avoid having to assign lower grades after it corrected the error, the paper said.
I wonder whether they used principal components or multiple inverse regression.
A financial blogger acidly observes:
The implications of that last paragraph are absolutely insane. “Some senior staff at Moody’s were aware in EARLY 2007 that CPDOs rated Aaa the previous year should have been ranked as many as FOUR LEVELS LOWER, the FT reported today, citing internal Moody’s documents. The firm ADJUSTED SOME ASSUMPTIONS TO AVOID having to assign lower grades, the paper said.”
First they make a mistake they shouldn’t have made. I mean, where were the checks and balances? Where were the reviews? Second, they then take that honest mistake and go criminal with it. Just brilliant. Amazing really.
Moody’s didn’t mention anything about not jousting with jesters.
Another financial blogger comments:
However, the implication that Moody’s had to fudge the numbers in order to come up with AAA on these deals but S&P came up with AAA with a “correct” model is something I for one am having a hard time with
.
Climate Audit 
29 Comments
No matter how Bloomberg tries to spin it, AAAA was well within the bounds of the ensemble of the 169 models used to determine the value of the debt obligations. It is perfectly reasonable under slightly different market initializations, eventually the holders of 100% of the subprime mortgages would have converged to full payment.
Don’t forget the teleconnections to the price of oil and corn futures.
Somebody’s going to wish he was hiding with Waldo.
Maybe Moody’s will say that these errors occurred in reports that were written a long time ago and since then they’ve “moved on”.
Or that other agencies got the same results using different methodologies.
This is the HUGE problem with behaving unethically.
You start off doing it in a little way to improve the citation rate on your paper. And it’s not as if the signal isn’t there, it’s just a little hard to see. So we’ll help it a bit.
And then you become famous, and people demand more and more of you, and you have to compound the error in the base data, or try to hide it.
Then a ‘competitor’ in another university sees what you’re doing, and publishes an even more extreme example so that he can gain some plaudits as well.
Then the politicians get interested, and soon everyone is generating more and more impressive figures. The press will only publish if your data shows something which is physically impossible!
And after a few years of this, the virus spreads outside Climate Science, and the money markets are affected….
I’m getting out of here before people start doing this with bridges and airliners…..
Perhaps the AAAA rating was a “scenario” not a “forecast”?
Aaa, not AAAA.
Which in Moodyspeak is equivalent to S&P’s AAA.
CPDO’s are the other side of credit default swaps.
Really not much to do with sub-prime in most cases.
Except that some of the companies included were places like CFC and BSC…
I’m not going to take the time to find the individual issues they were talking about,
but it seems likely that S&P were also rating them.
If so, I wonder whether the errors were really errors,
or someone just decided to change some fudge factor in new valuations going forward.
That happens from time to time, these models aren’t like your global warming heat transfer models.
Both the sellers and the buyers would scream bloody murder and refuse to have anything to do with the
rating agency if they retroactively changed the rating on something already sold because of a fudge factor change.
Steve, your ads from Google overwrite the end of this text entry box. Most annoying.
http://calculatedrisk.blogspot.com/2008/05/which-ratings-model-is-broken.html contains some more information and speculation about this case. Whether Moody’s altered their model to bring it in line with Standard and Poors or failed to catch the “bug” because it was what they were looking for or expecting, it still seems familiar.
Surely the observed rating fell within uncertainty intervals of the ensemble mean of the ratings. If it didn’t, well then, they just need to introduce more models in order to broaden the uncertainty intervals and therefore make more certain that the models are ‘consistent with’ the observed rating in the future.
That’s their problem, they used SEM instead of SD, and didn’t have a wide enough range of possible results to pick from to prove a lack of inconsistency (“consistent” as defined as in the range, not in a meaningful range as most would use it in a policy sense, one would imagine).
They just don’t have their terms defined properly. It’s the wrong test for this comparison of credit ratings!
Yes, Moody’s “tuned” its model to match observations in the market, and S&P’s model. So now you have two models agreeing with each other, lending credibility in numbers–an overwhelming consensus among major ratings agencies–as to their forecasting ability that complex debt instruments will perform as projected into the future.
The problem was really just market “noise”.
“Instead of idly debating the precise extent of global warming, … we need to deal with the central facts of rising temperatures, rising waters, and all the endless troubles that global warming will bring. We stand warned by serious and credible scientists across the world that time is short and the dangers are great. The most relevant question now is whether our own government is equal to the challenge”. -John McCain
Instead of idly debating the obscure financial details of certain maligned securities, we need to invest in them now. Credible sources like Moody’s have spoken, so we shouldn’t let deniers funded by their competitors spoil the situation.
As The Wall Street Journal commented, “His plan is ‘market based’ insofar as it requires an expensive, invasive government bureaucracy to interfere with the market”.
I’m sorry, Moody’s does not use a model that is sophisticated enough to be susceptible to these kind of errors. They are just trying to cover their tracks in the credit scandal.
Another financial blogger comments:
This sort of reminds me of the Mannian argument that criticisms of his Stick are “wrong” because other people get a similar answer. Only in bizarro-world would this sort of logic be allowed to stand. This only means that the criticisms of the Stick do not necessarily apply to the other strands of spaghetti – but then you have to wonder if they have perhaps made similar or their own errors (something that we’ve obviously discussed at length on the blog.)
Thinking out loud – when someone in climate science claims narrow confidence intervals, isn’t that a bit like saying that they have a AAA-rated model? If the true confidence intervals are little less than natural variability, that would be a Baa-rated model and not of investment grade. Maybe there’s a useful image here.
For those knowledgeable of Models the Aaa was obviously a PREDICTION based on a SPECIFIC SCENARIO, NOT a FORECAST!! Moody’s is not responsible for anyone not understanding this distinction and acting in ways that are INCONSISTENT with the FACTS!!!!
“Instead of idly debating the precise extent of global warming
Isn’t that kind of important? Like maybe the extent is currently less than zero?
Re # 12 Gunnar
Allied to Ross McKitrick’s proposal that carbon emission imposts be related to actual climate change, surely one can see the possibility of huge gains by betting on stocks that will benefit if the AGW bubble bursts. At present, people think they are doing well by investing in companies that trade emissions and plant trees, etc.
We should be conniving to identify stocks the opposite of this, stocks that will increase in value when we finally learn that we shall not have to fill the stratosphere with aluminium foil or sulphur. Stocks like China Coal & Electricity.
About Moody’s, one of my favorite authors (Carroll) might have mumbled
Steve — when you gonna let this issue of the stick die? It must wake you from your sleep at night. Take care.
I can’t speak for Steve, but I suspect he would happily let the hockey stick die. But despite the fact it is clinically brain dead, people like Mann, Tamino, Wahl, Amman etc, and others keep wanting to resuscitate it.
Besides which – it is good to learn from past mistakes. Should we forget about other disasters – say Enron while on the financial theme – or should we keep reminding ourselves what went wrong and why, so we can avoid making the same mistakes again?
Eric,
It’s nearly impossible to do any cross checking without a hockey stick
The problem is the financial markets apparent blindness to the problem of kurtosis. (A mathematical term for “when the smelly stuff hits the fan, it does so all at once”.) Financial market movements do not follow a normal distribution, and have not done so at least since the 1987 crash – probably longer, for all I know.
And yet, all these silly credit instruments still work on the basis of gaussians, and everyone acts surprised when supposed thousand-year events continue to come along every five years.
Bah. Even sillier than “the market has no memory”.
Both stock markets and climate influenced Mandelbrot – some of his comments on markets are relatively well-known; his comments on tree rings aren’t. But he looked at some of the tree ring series that we’ve discussed here. The Nile River series (Hurst exponent) is one of the first series studied for fractality. One of Mandelbrot’s most memorable images linked two aspects of fractal series (the 7 years of lean, 7 years of fat – Mandelbrot’s “Joseph Effect”) and the occurrence of huge anomalies (Mandelbrot’s “Noah Effect”). All expressed in not easy math, but, even without following the math, one can squint at some of the strange behavior of time series once i.i.d. Gaussian ceases to apply.
Ah, but two of the models (Fitch and DBRS) couldn’t produce a AAA scenario never mind how much they fiddled with the initialisation criteria. DBRS in all seriousness said that any instrument which can afford to pay Libor +2% can’t possibly be AAA.
Eminently sensible attitude.
. . . or that, taken in aggregate, everything evens out, so what’s the big deal?
so many adjustments . . . maybe they are really chiropracters, not climate scientists ?
Climate “Science” (BH,ICS) is nothing without adjustments.
Fred,
Sounds familiar. Did someone once suggest that the Hockey Team could name themselves the Climopractors? Motto: Adjustments are us!
They obviously failed to take into account the “urban over-heat island effect”. The island in question, of course, being Manhattan.