When A Middling Amount of Knowledge is a Dangerous Thing

In a long post which is mostly a review of Jim Manzi's Uncontrolled and Nate Silver's Signal and the Noise (of the two, he recommends Uncontrolled, and being three quarters of the way through it I'd certainly recommend it) Razib makes an interesting side point about the much discussed questions on polling and forecasting the recent election:

But after soft-pedaling my confidence in polling averages, why did I think the pro-Romney people were delusional? The simple answer is 2004 and 2008. When the polling runs against you consistently and persistently motivated reasoning comes out of the wood-work. There’s a particularly desperate stink to it, and I smelled it with the “polls-are-skewed” promoters of 2012. In 2004 there were many plausible arguments for why the polls underestimated John F. Kerry’s final tally. And in 2008 there were even weirder arguments for why McCain might win. In 2012 it went up to a whole new level, with a lot of the politically conservative pundit class signing on board because of desperation.
...
After the election was over I actually started reading some of the arguments about why the polls were skewed, and I find that they are extremely plausible to me. And not just me, John Hawks owes me a drink because he simply didn’t believe the turnout models which suggested a demographic more like 2008. The reality is that my instinct was to go with John. I too was very skeptical of the proposition that Obama could turnout the same voters as he did in 2008. And yet he did turnout those voters!

What struck me as interesting here is that Razib essentially made a choice to stick with one basic but fairly well established thing that he knew (averages of large numbers of polls are pretty good at predicting elections) and knowing that he wasn't going to be a true expert in all the mitigating factors which might cause one to question that rule, chose simply not to look at all the argumentation and to stick with his one basic piece of knowledge. Looking at the situation after the fact, he found that indeed the arguments put forward against the basic rule were in fact fairly convincing, but this just served to reinforce his judgement that he had not been in a position to correctly weigh the merits of counterarguments to the basic rule.

This is really interesting from a more general point of view, and I think it underscores some of the pitfalls of having a middling amount of knowledge about a subject. If our knowledge of a topic is basic, and we know it's basic, and so all we do is apply and well agreed upon rules, it's likely that at the very least we won't embarrass ourselves by coming up with something way outside the mainstream. The point when we often get into trouble is when we study a topic a bit more and start to think we know enough to get creative and know when things won't behave in the way that the most basic rules would suggest. It's at this middling stage of knowledge that we're likely to make big mistakes and make them with excessive confidence.