Our Need for Fictions.
We like to think that all problems have solutions. The reasons may be complex and difficult to grasp, but our faith is that if we understand the causes, we can cure the effects.
And the simpler answers are better. The appearances of things are often confusing, yet if we dig deeply enough we will get to the core of truth that will give us more control over events. But that belief is getting harder and harder to sustain.
Investors, for example, used to try to focus on "the fundamentals" of a stock to determine if it would grow in value. Was the company well capitalized? well managed? strategically positioned in the market? Brokerage firms used to employ analysts to study such factors in order to advise their customers. But, then, economists looked at the data more closely and discovered that no analyst actually out performed the market systematically. The best an investor could do was to diversify, assemble a large basket of securities that covered the range of possibilities. That discovery led to the birth of index funds. They don't solve the problem for investors, but they make it easier to live with it.
Recently Jonah Lehrer wrote in Wired about how science too is failing us. He focused on the pharmaceutical industry's efforts to produce drugs that target specific ills, and he gave the example of an extremely promising drug to treat cholesterol developed by Pfizer. In the final phase of clinical trials it turned out that the drug did not work. Indeed, in many cases, it harmed patients. (See, "Trials and Errors: Why Science Is Failing Us.")