Thoughts on McIntyre and McKitrick 05

The following is a comment published at And Then There's Physics.  I am posting it here as a ready access source for me for the relevant links and commentary on M&M05:



I have been looking further at our hosts rampant campaign of unwarranted accusations of a "lack of integrity" suggested, by apophasis, to exist by Mosher and was coming up empty. Fortunately Brandon Shollenberger came to the rescue at Climate Etc with this comment:

"I disagree andrew adams. Anders falsely accused Steve McIntyre of cherry-picking runs of a simulation to exaggerate his results when he hadn’t even looked at the paper in question. That’s iffy on its own, but when I confronted him on the issue, he adamantly refused to look at the paper. When I pressed this issue, Anders banned me because he didn’t like what I said on a different site. I think banning a person despite them having never behaved poorly at your site, because they say you are leveling baseless accusations against a person based upon a paper you refuse to even look at, calls a blogger’s moral compass, ethics and integrity into question."

The accusation has been endorsed by Carrick and Mosher. Carrick in fact admits that the two accusations made by Shollenberger are the only examples he knows of that have lead him to claim that Anders has never "...demonstrated that [he] have a moral compass", and that he is an example of "Scientific sociopathic behavior at its finest". The accusation, it turns out, is related to this comment by Anders at Shollenberger's blogsite, where he writes:

As far as your criticism of MBH98 is concerned, I don’t dispute the issues. It may also be true the using the MBH98 data to produce the red noise is largely irrelevant. It does, however, seem odd – as a physicist – to see people claim to produce independent random red noise, but to do so using the data they’re trying to compare to. Maybe that illustrates my ignorance with respect to what actually happens here, but it still seems a little odd. What seems indisputable, though, is that the 10 hockey sticks presented in MM05 (one of the papers, you probably know which one) were not selected randomly from their sample of 10000. They were chosen to be most hockey-stick like. People, however, clearly interpret the results of MM05 as implying that random red noise typically produces hockey sticks, rather than random red noise sometimes (probably quite rarely) produces hockey sticks.

The actual facts of the case are that after running his program to generate 10,000 pseudo reconstructions from red noise, McIntyre and McKitrick, 2005 selected the 100 pseudo reconstructions with the highest Hockey Stick Index (HSI, =(1902-80 mean minus 1400-1980 mean)/1400-1980 standard deviation). That group of 100 pseudo reconstructions were mentioned in the paper, and included in the supplementary information. All choices of pseudo reconstructions to graph by M&M05 were chosen from that cherry picked selection. That includes the single graph shown together with the MBH98 reconstruction as Fig 1 in M&M05, and the panel of twelve pseudo reconstructions generated by the code included in supplementary material of M&M05, and published in the Wegman Report. At no point in the paper or in supplementary material do M&M05 draw attention to the fact that the selection is a biased selection. Now, it is clear that Anders made a couple of mistakes. To start with, although the pseudo reconstruction that was published in M&M05 "... were not randomly selected from their sample of 10000", being in fact selected from their sample of the 1% with the highest HSI; and though the 10 pseudo proxies graphed by the code provided with the supplementary information of M&M05 "... were not randomly selected from their sample of 10000", the later 10 were not shown in the paper. Further, the eleven pseudo reconstructions graphed either in figure 1 or by the code provided with the paper were not the eleven (or 10) with the highest HSI, they were merely selected from the 1% of pseudo reconstructions with the highest HSI. It is clear, however, that while these are mistakes, they are trivial mistakes. They do not effect the substance of the issue. A cherry pick of the top 1% of pseudo reconstructions in terms of the HSI is still a cherry pick, and not informing readers either in the paper or notes on the supplementary information that the graphs generated by the program in the SI are a biased selection is a serious breach of normal standards of publication. This is particularly the case as the only tests conducted by M&M05 to show the ability of red noise to generate MBH98 like pseudo reconstructions is by visual comparisons with the cherry picked selection. That last may seem an odd claim, in that surely a comparison was made using the HSI itself, but in fact M&M05 never publish the HSI of either the MBH98 reconstruction (1.13), the MBH98 580 year PC (0.94), or the MBH 99 reconstruction (1.13). These are 22.7 (MBH98 and 99) and 27.9 (MBH98 PC1) standard deviations less than the mean HSI of the selected 1% of pseudo reconstructions. The are also in the bottom one percentile (MBH98 PC1) or, probably, 2.5%ile (MBH98 & 99) of results for all 10,000 pseudo reconstructions. I say "probably" for the later because M&M05 do not give full statistics. The do state that less than 1% have a HSI less than 1, and that only 27% have a HSI less than 1.5. From the histogram (figure 2), it is possible to determine that about a fifth of that 25% have a HSI less than 1.25. You can see, therefore, why McIntyre and McKitrick were loath to do more than visual comparisons. Had they done a statistical comparison using their chosen measure of similarity (the HSI), the paper would have (apparently) reported that the MBH98 method applied to red noise generates hockey stick like shapes but that statistical tests show at the 90% (certainly) or 95% (probably) confidence levels, the MBH 98 was not a random outcome from red noise. This is even clearer using more conventional tests. The peak r-squared comparing the 100 selected hockey sticks to the instrumental record over the period 1902-1980 was 0.35, compared to 0.76 for MBH98. That is seven standard deviations from the mean of the r-squared statistics for the selected pseudo reconstructions. Again, actual numerical statistical tests, as opposed to eyeballing cherry picked graphs, show the MBH98 reconstruction not to have been a chance outcome from red noise. This is something Michael Mann has also demonstrated by other means. Further, certainly McKitrick (McKitrick 05) and probably McIntyre were using the selected set of pseudo reconstructions in public commentary in 2005. Going back to Ander's "perfidy", it is clear that in this episode his claims were essentially correct. Further, he did not make an accusation of cherry picking as such. He did claimed a selection bias in the graph, but made no claim as to how that would effect the result or as to the motives of McIntyre and McKitrick in imposing the selection bias. His most strenuous criticism is to say some of M&M05 methods "seem odd", and then to qualify that by indicating that that may merely illustrate his lack of specialist knowledge on the topic. A damnation of M&M05 on the basis of lack of research integrity his comment is not. The second basis of Anders purported "lack of moral compass" is that he banned Shollenberger from this site because Shollenberger repeatedly claimed that he (Anders) had lied (Shollenberger uses the word "fabricating" which definitely implies intent, and in context implies the claims are false). After a discussion with Shollenberger on his site, I heartily endorse Anders sentiment of never wanting to discuss anything with Shollenberger again. It is not a moral or a personality flaw to dislike discussing things with people who do not discuss in good faith, show a lack of personal integrity, and behave like complete pricks.