<blockquote>
'As a result of the near-universal use of VaR-based risk management systems, even as early as August 2007, David Viniar, chief financial officer of no less august an institution than Goldman Sachs, was moaning that he was seeing "25-standard deviation events" several days in a row. As the lucky chap (because still employed at a senior level by the richest financial institution in the world) probably now realizes, if he was seeing 25-standard-deviation events, his model was wrong. In a truly Gaussian system, you would be unlikely to see a 25-standard deviation event during the entire history of the universe.
...
We had better start hiring some mathematically trained skeptics. As a former mathematician myself, who had even constructed a primitive computer model, I was able to spot the modeling flaws in an early presentation of the "Club of Rome" global catastrophe theory of the 1970s. Shortly thereafter I participated in the other side of the process. For a term paper, I built an econometric model of the Malaysian economy that contained a hopeless theoretical error (my one lifetime course in economics, being at Keynesian Cambridge, had not included any discussion of money supply) but produced a highly plausible and indeed in the event correct prediction of a decade of non-inflationary growth for Malaysia.
The latter example indeed is instructive. I was fairly sure from my reading that Malaysia would do OK and so constructed an (utterly flawed) model to "prove" it. Wall Street mathematicians seeking to show that a new derivative had little risk, or climate scientists seeking to prove that Manhattan would be under water by 2075, would have been proud of me.
Only those who have themselves built these kinds of models know how misleading they can be. The rest of the world is "blinded by science" and trustingly accepts the prognostications of the charlatans at face value. The result is a more or less infinite potential for loss, except to the designers of the models, who become rich and famous, the mixture according to taste.'</blockquote>
Mad MathesisBy Martin Hutchinson
December 14, 2009
SourceWe've now lived through the same new disaster twice. Computer simulations, more or less universally adopted as the solution to a major problem, turned out to have been based on flawed assumptions and faulty data. As a result policy or markets became heavily skewed in an inappropriate direction. Wall Street's risk managers and climate change scientists both acted as super-salesmen for a paradigm that turned out to be flawed. After two examples of the same error have each cost the world a substantial percentage of a year's GDP, we'd better figure out how to avoid further examples of this syndrome.
Alexander Pope put the problem best, in his 1728 "Dunciad:"
<blockquote>
"Mad Mathesis alone was unconfined,
Too mad for mere material chains to bind,
Now to pure space lifts her ecstatic stare,
Now, running round the circle, finds it square."</blockquote>
Pope never had access to computer technology, or he would have realized that proving circles to be square was only the beginning of the chimerical wonders that could be created with the right software.
As the credit crisis of 2008 recedes into history, the part in it played by misguided computer models, particularly in the risk management area, is becoming generally agreed. Rating agencies made assumptions about the probabilistic independence of different home mortgages that were unfounded. As a result many of their AAA ratings proved to be completely spurious, particularly in the subprime area where the loans' vulnerability to a house price downturn was especially extreme.
Investment banks managed their risks based on the "Value-at-Risk" risk management paradigm, which assumed that the distribution of securities' returns was approximately Gaussian (normally distributed), with a very low probability of high losses. The "Basel II" system of global capital adequacy standards for banks, which came into effect in 2008, just in time for the crash, was so impressed with these models that it ruled that any bank using such obviously sophisticated and superior modeling techniques could calculate risks on its own, without reference to the crude guidelines deemed appropriate for smaller, less mathematically attuned houses. The Securities and Exchange Commission (SEC) essentially agreed with the Basel Committee; from 2004, it allowed the largest U.S. investment banks to manage their own leverage, under the theory that no mere regulator could match the exquisite precision of a modern VaR-based risk management system.
As a result of the near-universal use of VaR-based risk management systems, even as early as August 2007, David Viniar, chief financial officer of no less august an institution than Goldman Sachs, was moaning that he was seeing "25-standard deviation events" several days in a row. As the lucky chap (because still employed at a senior level by the richest financial institution in the world) probably now realizes, if he was seeing 25-standard-deviation events, his model was wrong. In a truly Gaussian system, you would be unlikely to see a 25-standard deviation event during the entire history of the universe.
It's not as if Wall Street had no warning; mathematical models based on modern financial theory had caused huge losses as far back as 1987, and had caused the collapse of Long Term Capital Management in 1998. Yet the world's best remunerated people went on using the mathematical models that had caused moderate sized disasters before, only to watch them cause a truly impressive disaster in 2008. It must have been some kind of compulsion.
Turning now to my other example, that of global warming: the possibility that excess carbon dioxide, through a "greenhouse effect" might cause a global rise in temperature is based on well-established chemistry and physics. Deniers of the possibility of global warming are thus being as irrational as the extreme eco-alarmists; global warming is indeed possible because of physical and chemical processes that are perfectly well understood, indeed fairly elementary.
The difficulty arises in estimating whether it is actually happening. The rise in temperatures so far observed is well within the level of "noise" in global temperatures over a period of a century or so, let alone the more extreme fluctuations that have taken place when the observation period is extended to millennia. It is thus necessary to match the very limited temperature data we have, stretching back no more than a century on a worldwide basis, with secondary observations of such things as tree rings and ice cores, synthesizing the result with a computer model of what is believed to be the carbon forcing process in order to predict the range of possible future warming effects.
This is of course a very similar process to that undertaken by Wall Street's rating agencies and risk managers. Assumptions and simplifications are made, without which it would be impossible to construct a model. Then the model is matched up against a few years' observations in real time, being "tweaked" as real data comes in that does not quite fit with it. By the time this has been done, careers have been invested in the model, institutions have been built around its predictions and eminent people have become enthralled by its results. It thus takes on the appearance of a scientific reality as solid as Newtonian mechanics.
The shakiness of the mathematics underlying the global warming "consensus" was highlighted by the recent "Climategate" e-mails and computer tapes. Like Wall Street risk managers, climate scientists pooh-poohed the obvious flaws in the assumptions underlying their mathematical models. Like Wall Street bankers, they asserted a consensus behind those models – in Wall Street's case, to win from regulators a profitable loosening of their leverage limits; in climate scientists' case, to persuade politicians to provide them with hugely profitable research opportunities and capital for their "new energy" start-ups. Like Wall Street traders, they rejected any modifications of the models that had served them well, and pushed those models to their outer limits, to trade ever more exotic derivatives, or to justify ever more alarmist predictions of climate change.
The denouement in both cases may also turn out to be similar. In Wall Street's case, the faulty models have led to losses in the financial system totaling in excess of $1 trillion. In the climate scientists' case, the precise degree of error in their assumptions is not yet apparent. It is only clear that dubious methods were used to cover up the flaws in their models and observations, and that the more extreme predictions ("6 degrees Celsius by 2100") were made up out of whole cloth to justify gargantuan economy-destroying projects of government control.
Should the Copenhagen conference produce anything beyond alarmist blather, the net cost to the global economy is likely to exceed by far that of the subprime mortgage fiasco. The difference between the two cases is that in Wall Street, the first decent-sized downturn showed the models to be rubbish, although admittedly that took 21 years to happen after the first demonstration. On the other hand with climate models we will have to wait even longer, until 2100, to find out whether they were completely spurious or merely exaggerated.
Since this has now happened to us twice in the same generation, we had better assume it is a trend, and decide how to combat it. Even today, in some other sector of economic activity, "scientists" are doubtless creating further mathematically-based predictions with the potential to destroy yet more of our wealth. We will again be asked to admire the beauty of the output and the sophistication of the model, while being carefully steered away from the highly dubious assumptions on which the model is based. Doubters of the model will again be dismissed as ignorant peasants, too poorly educated to understand the sophistication of the analysis.
We had better start hiring some mathematically trained skeptics. As a former mathematician myself, who had even constructed a primitive computer model, I was able to spot the modeling flaws in an early presentation of the "Club of Rome" global catastrophe theory of the 1970s. Shortly thereafter I participated in the other side of the process. For a term paper, I built an econometric model of the Malaysian economy that contained a hopeless theoretical error (my one lifetime course in economics, being at Keynesian Cambridge, had not included any discussion of money supply) but produced a highly plausible and indeed in the event correct prediction of a decade of non-inflationary growth for Malaysia.
The latter example indeed is instructive. I was fairly sure from my reading that Malaysia would do OK and so constructed an (utterly flawed) model to "prove" it. Wall Street mathematicians seeking to show that a new derivative had little risk, or climate scientists seeking to prove that Manhattan would be under water by 2075, would have been proud of me.
Only those who have themselves built these kinds of models know how misleading they can be. The rest of the world is "blinded by science" and trustingly accepts the prognostications of the charlatans at face value. The result is a more or less infinite potential for loss, except to the designers of the models, who become rich and famous, the mixture according to taste.
The solution is simple. Every time a group of scientists produce a mathematical model on which depends some economically serious outcome, decision-makers must hire a rival team of scientists, promising to reward them richly should they prove the model to be rubbish. Only by bringing the icy blast of competition to the overstuffed halls of academia can spurious computer-generated mathematics be fought off. Otherwise, it threatens our civilization.