We Need A Census On Consensus

Here we go again. Read the tale of how inexpert researchers incorrectly claim that expert researchers are more reliable, both proving and disproving their thesis in a convoluted construction that resembles an Escher print.

Richard Tol, former lead author of a section for the Intergovernmental Panel on Climate Change’s Assessment Review, recently published a critique of John Cook’s literature review that came up with a 97% consensus of climate scientists supporting the view that more than half of the recent warming in global temperatures is caused by human emissions of greenhouse gases.

Yesterday the Empire struck back, with a letter co-authored by, among others, Naomi Oreskes, John Cook, William Anderegg, Dana Nuccitelli, J. Stuart Carlton and Ken Rice (who we know better as operator of the weblog And Then There’s Physics).

In the abstract of their letter they write, “Tol (2016 Environ. Res. Lett. 11 048001) comes to a different conclusion using results from surveys of non-experts such as economic geologists and a self-selected group of those who reject the consensus. We demonstrate that this outcome is not unexpected because the level of consensus correlates with expertise in climate science.”

In their introduction they continue, “Tol’s erroneous conclusions stem from conflating the opinions of non-experts with experts and assuming that lack of affirmation equals dissent.”

This is somewhat inconsistent with their dismissals of senior skeptical scientists with hundreds of papers to their credit, such as Richard Lindzen or Judith Curry, as well as their over-use of the term ‘Gone Emeritus’ to describe other, older scientists who disagree with them.

Perhaps we should pause for a moment and use their own logic about their own papers. Most of the letter’s authors neither claim nor show expertise in climate science or conducting social research of the type they presented in their original papers and the letter just published.

These people are clearly interested in the consensus regarding climate change, but they just as clearly show no background or expertise that would automatically confer credibility on the subject.

John Cook is a cartoonist. “He studied physics at the University of Queensland, Australia. After graduating, he majored in solar physics in his post-grad honours year. He is not a climate scientist.”

Naomi Oreskes is a historian, not a climate scientist, not a social scientist.

William Anderegg was until 2015 a post doctoral candidate at Princeton who taught a course on ‘The Life and Literature of John Steinbeck.’ To be fair, his focus has been on climate change.

J. Stuart Carlton writes of himself “Hi there. I’m Stuart Carlton, and I’m a Healthy Coast Ecosystems & Social Science Specialist at Texas Sea Grant. I study the human dimensions of natural resource issues, including climate change, stakeholder controversies, and the conservation of rare and imperiled species.”

“Dana Nuccitelli is an environmental scientist at a private environmental consulting firm in the Sacramento, California area. He has a Bachelor’s Degree in astrophysics from the University of California at Berkeley, and a Master’s Degree in physics from the University of California at Davis.” That private consulting firm has at least one large fossil fuel company as its client.

Ken Rice is not a climate scientist, nor a social scientist. “I currently focus mainly on understanding the formation and evolution of planets and planetary systems.  In particular I am interested in the formation of directly imaged, massive planets that have large orbital radii.”

Bart Verheggen is one exception, being an aerosol scientist and someone I think very highly of. Sadly, I disagree with what he writes, as you will see.

Considering that many of these same people have dismissed luminaries such as Freeman Dyson and Ivar  Giaevar because of the lack of a supposed requisite background in climate science, it is curious that they consider themselves qualified to pronounce upon subjects like expertise or the analysis of this type of research.

But, as it happens, I disagree with their insistence on formal credentials. I think their work should be judged on its merits. Indeed, I have written frequently in judgment of their work. On Oreskes here, On Cook here and here, On Anderegg, Prall et al here and on Verheggen et al here and here.

And I know, as does almost everybody following the subject, that there is a consensus among climate scientists that human contributions have caused half or more of recent warming. Two surveys conducted by climate scientists of climate scientists found that 66% of climate scientists support this narrowly constructed consensus statement. Bart Verheggen, who I respect highly, conducted one of the surveys. However, he buried the headline finding of a 66% consensus, subtracted those who answered ‘I don’t know’ or ‘it is unknown’ from his respondent pool and reported a higher percentage. He then spent the remainder of his analysis looking at the higher percentage found in scientists with more publications. He said that having more publications made these scientists more expert.

Some of the other studies mentioned in this letter responding to Tol are literature searches. They were not conducted properly. Oreskes wrote a short essay on the scientific consensus titled ‘Beyond the Ivory Tower.’ She has never published her methodology, but she missed 117 skeptic papers found on the Skeptical Science website while reporting there were no skeptical papers found during her search.

Anderegg, Prall et al, PNAS 2010 conducted their search using Google Scholar in English only, misreported details on job specializations and numbers of publications, botched the mathematical analysis of their ‘consensus’ and created a blacklist of skeptical scientists, leading Joe Romm to call for the withdrawal of funding and a media blackout on scientists named as ‘deniers’, the metatag for the paper’s publication.

John Cook’s paper is a nightmare of shoddy workmanship. He recruited regular visitors to his website to rate abstracts of 12,000 plus papers. One was so enthusiastic that he rated 675 papers in 3 days. They were supposed to be isolated from each other, but were not. They were supposed to be unaware of the authors of the paper, but they looked them up as they read the transcript. After the fact, many scientists said that their papers had been misconstrued by the researchers.

This letter contains many of the same flaws. Brandon Schollenberg has highlighted the usual difficulties with basic mathematics that has afflicted these consensus searchers. I want to focus two other issues here.

1.The first is their statement that ‘lack of affirmation does not equal dissent.’ This refers to the fact that when climate scientists said they didn’t know if humans were the cause of half or more of the climate change, Verheggen et al removed their answers from the total. Verheggen feels that this is justified. I do not.

A consensus is a positive agreement on an overarching theme. It doesn’t mean you have to agree with everything put forward and it doesn’t mean you have to support the conclusions others reach. But it is a positive affirmation.

It would be very fair to say that those who answer ‘I don’t know’ or ‘It is unknown’ to such a question are not necessarily skeptics or opponents, and I would support an explicit statement to that effect.

But it certainly does not warrant excluding their data because they don’t know. It skews the results. They are quite clearly not sharing the affirmative nature of a consensus view on climate science and that should be reported.

2.I have corresponded with Bart Verheggen on the use of the numbers of publications as a proxy for expertise in the past.

I wrote, “The weaknesses of publication records are:

1. Very capable younger scientists have not had time to establish a record of publications. Dismissing their opinions leads to loss of useful information.

2. As ‘alarmists’ like to point out whenever an older scientist expresses a skeptical viewpoint, at some point in the natural cycle of a person’s career, ongoing education becomes less important. One can make the case that someone reaching the end of their career actually knows less than a freshly minted scientist.

3. The tools and techniques used in tertiary education are different than they were when many older scientists were educated. In addition, new knowledge is incorporated into texts available to younger scientists. This again may advantage the young at the expense of the old.

4. Some scientists are co-authors of numerous papers for reasons other than their ability to contribute to the main body of the scientific arguments advanced in the paper. Their publication count may be more impressive than their actual command of the field.

5. Some very good scientists work outside the academic world and publication may not be a priority for them. Using publications as a proxy for expertise again may devalue their opinions.

I see no good reason to accept your statement that “Number of publications relates logically to how much scientific research somebody has been engaged in.” Are there publications supporting this point of view?

It seems in the way you refer to it that it is an assumption on your part. I think the 5 points I list constitute an argument that the assumption, if that is what it is, is unwarranted.

Because your analysis framework emerged after viewing the results of the survey, as you mention above, it seems that you are in the position of trying to compensate for unwanted results.

And I know you better than that, Bart. I don’t think you operate that way. But you haven’t adequately (for me, at any rate) explained your analytical choices.

You might find this of interest: http://higheredstrategy.com/wp-content/uploads/2012/06/2012-Bibliometrics-and-Publication-Culture-HESA.pdf

This paper suggests that citation counts might have been more productive in determining expertise: http://link.springer.com/chapter/10.1007%2F978-3-658-03348-4_4#page-1

Several publications that discipline normalization would have been valuable to your research efforts, eg. https://books.google.com.tw/books?id=olu3k6CiBuUC&pg=PA262&lpg=PA262&dq=problems+with+publication+counts&source=bl&ots=00hoPJRWoB&sig=ZTT3T0EEsWvYpeDABIx9FcRRdbg&hl=en&sa=X&ved=0CEQQ6AEwBmoVChMIzNb5g6mvyAIVQx2UCh1_mg2a#v=onepage&q=problems%20with%20publication%20counts&f=false

In fact, a cursory overview of publications regarding publication counts shows that the problems pub counts face in providing valid information are formidable.”

It is obviously easier to attract attention, funding and political support for a slow-moving problem like climate change if you can claim 97% of the scientists agree with you. But the fact is the consensus is (a still healthy) 66%.

Whitewashing the higher level of uncertainty felt by younger, better educated and more aware scientists is bad enough when it happens in academia. When it is done by inexpert enthusiasts pushing a political program and labeled ‘science’ it is far worse.