Garbage In, Garbage Out

“They wanted to keep their capital base as stable as possible so that the limits they imposed on their trading desks and portfolio managers would be stable.”

One way they did this, Mr. Berman said, was to make sure the computer models looked at several years of trading history instead of just the last few months. The most important models calculate a measure known as Value at Risk — the amount of money you might lose in the worst plausible situation. They try to figure out what that worst case is by looking at how volatile markets have been in the past.

But since the markets were placid for several years (as mortgage bankers busily lent money to anyone with a pulse), the computers were slow to say that risk had increased as defaults started to rise.

It was like a weather forecaster in Houston last weekend talking about the onset of Hurricane Ike by giving the average wind speed for the previous month.

(Saul Hansell at the New York Times' Bits Blog: How Wall Street Lied to Its Computers)

On two occasions I have been asked,—"Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" [...] I am not able rightly to comprehend the kind of confusion of ideas that could provoke such a question.

(Charles Babbage, quoted in the Wikipedia page for Garbage In, Garbage Out. The original source of the quote is Babbage's Passages from the Life of a Philosopher, which can be found at Google Books. Note also that the second reference mentions the survey essay Computer Applications in Particular Industries: Securities [1981].)