Programmer Smack Talk and Global Warming

I've been amused to watch some of the arguments going on out in the blogsphere as discussion of the hacking of the climate change servers moves off into a discussion of the quality of the code being used by climate researchers to model global warming.

Example:

Commenter One: Much of the code in the academic world tends to be written by grad students that have taken a class in programming and get told to write it.

Commenter Two: This is totally untrue. I never took a class in programming before writing my crappy undocumented code.

There's a certain wry self recognition for me here as well: I've never taken a class in programming, and I build mostly undocumented models to predict revenue and profits at specific price points based on past data. My results are directionally correct when you look at whole categories of products, but can be wildly off when projecting specific instances. (I try to make this clear to those who use my data, but people are always looking for certainty in life, even if they have to imagine it.)

The difference is, of course, that I'm seeking to mitigate the risks people take in making decisions that they're going to make anyway. "Gee, I really feel like we need to turn this product 50% off for the holidays." "Well, past experience shows that we wouldn't sell many more units, but would lose a whole lot of money. Let's try something else."

You would think that if you were going to, say, recommend that the entire world ratchet levels of CO2 emmissions back to the levels of the 1800s (with all the impacts to living standards and, let's be honest here, human life, which that entails), you would aspire to higher levels of accuracy and transparancy.

In a sense, I would imagine that these climate researchers have much the same justification for their actions that I do: They're just giving people common sense advise. I advise people not to waste too much profit margin. They advise people not to emit too much CO2.

Waste enough profit margin and your company goes out of business. Get enough CO2 in your atmostphere, and you get to enjoy the kind of climate that Venus has. From the point of view of serious environmentalists, who often seem to assume that any change made by humans to the planet is pretty clearly a bad thing, it may not seem like one needs to bring a lot of rigor to advising people to not burn fossil fuels. From that point of view, of course doing all these "unnatural things" will have bad consequences.

However, for the rest of us, the fact that modern industrial technology allows six billion people to live on this planet -- and for many of them to do so in greater material comfort than at any previous time in history -- is pretty clearly a good thing. And standing in front of that yelling "Stop" requires some pretty rigorous evidence. This isn't something that can be left to buggy code whose results are massaged into shape manually when they're going to come out into the light.

Adventures in Regression

Megan McArdle provides a perfect example of why a turning a fool loose with a regressiion or graphing tool can get you all sorts of strange places.

I've seen a lot shakier plots used to justify some sweeping conclusions, and if those were justified, well, then I'm forced to conclude that Mexican lemons have improved highway safety a great deal. The vitamin C, maybe? The fragrance? Bioflavanoids?

This is particularly tricky when you bring time into it, because things trend--as we get richer, we buy safer cars, get better emergency rooms, etc. We also import more lemons to make our chi-chi cocktails and lemon meringue pies. Overlay the two, and you've got a hell of a causal relationship.

But I expect that four years from now, we'll still be having the same conversations with proponents of "cancer clusters" and Democrats convinced that they can scientifically prove that Democrats are better for GDP by doing ham-fisted regressions of Democratic presidencies with a few tightly correlated economic variables. What's the mechanism? What makes electric power lines cause cancer, but not the earth's vastly more powerful magnetic field? What policies did Harry Truman and Bill Clinton have in common (but not with Richard Nixon) that caused this marvelous confluence? Well, maybe we don't know the mechanism exactly, but never you mind: just look at that bee-yoo-ti-ful correlation!

On less political issues, I find myself dealing with these kinds of "analysis" at work all the time. As in all things, common sense is in order. If there's no reasonable explanation for any sort of causal relationship between two factors, then consider very seriously the possibility that there is none.

I wish I had the years behind me to get away with the Professor's recurring line from The Lion, the Witch, and the Wardrobe: What do they teach them in schools these days?

Fraud, Folly or Probability

As the government continues to pump money into AIG, the foundering insurance giant which found itself at the center of the real estate and financial crashes, I've seen increasing numbers of commentators demand to know why no one is calling for the jailing of AIG executives on charges of fraud. How, the argument goes, was their selling of financial insurance products any different from the sort of fraud Maddoff carried out? They sold insurance policies they couldn't cover! They took money and gave nothing in return!

I think this tends to underline that people don't actually understand insurance and how it works very well. This is doubly concerning in that insurance has become increasingly central to people's ideas of economic security in the last few decades. Indeed, we've reached a point where lacking health insurance is itself considered a health problem, regardless of whether this actually results in someone failing to receive needed treatement.

What is insurance? Basically, insurance is a way of extending your savings for unlikely but high cost eventualities. For example, if I were to die this year, it would place a very heavy burden (in the form of lost earnings) on my family, such that it would be very difficult for me to save enough money to be prepared for the possibility. However, for a few hundred dollars a year, I can buy life insurance that would, if I should die in the next ten years, pay out a lump sum equivalent to 5-10 years of my current earnings.

The insurance company can afford to offer this to me because they believe they have an accurate model of how likely a healthy 30-year-old man is to die before 40. However, if a horrific plague were to strike the US, killing 10% of the population, and I was among its victems, could I console myself on my deathbed with the knowledge the insurance company would be taking care of my family?

No. Their models do not account for the possibility that 10% of the healthy, adult population would die in a single year. They would undoubtedly go bankrupt and my family would get little or nothing.

This is something we don't normally think of in regards to insurance, because insurance companies have good enough models that it would take something very unexpected to make them unable to meet their obligations. Still, because insurance companies base their prices on what they believe to be the probability of having to pay out, if something unforseen by their models occurs, they may well find themselves on the rocks. One should always understand insurance to come with a "all other things being equal" sort of proviso.

But shouldn't AIG have been able to forecast the possibility of a global real estate downturn and take that into account in their pricing? To my knowledge, there's never been a real estate downturn of the scale of the current one on such a wide scale. Sure, plenty of people were predicting the housing market would go down. That was one of my own reasons for getting out of California back in 2003 -- but those who bought a house there when I left would have seen 50% appreciation over the next four years before things went south. And even though many people were going around saying, "It has to go down," there was no historical precident for such a widespread downturn. In the past, it looked like so long as the economy and population were growing, real estate would continue going up.

Now, if you'd asked me in 2006 to insure mortgages against real estate values going down (and the resulting mortgage defaults), I would have refused. And some cautious companies doubtless did. But since there was a decent historical/statistical case to be made that they wouldn't go down, it's not exactly a surprise that someone agreed to meet the need.

Shouldn't there have been regulations in place to prevent people from offering insurance products that they wouldn't actually be able to pay out on in an emergency? Well, here we run into the probability problem again. Is it in fact likely that something will occur which will bankrupt an insurance firm? There's no reason to believe that regulators will be all that much better at understanding what could happen in the future than insurance actuaries are.

In the end, we probably just need to accept that sometimes bad and unexpected things happen. We can do our best to prevent them, but the funny thing about unexpected events is that it's hard to know they'll happen until they do.

So no, I don't think there was necessarily fraud involved here. Though there was an excessive faith in our ability to forecast the future. We'd do well to avoid such excessive faith in analytics in the future, and yet most of the suggestions for "fixing the problem" simply involve having government regulatores forecast the future instead of private insurance writers. I rather doubt that ends us up in a better place.