Tuesday, August 29, 2006

The role of consensus in science

In the policy debate over global warming, anti-AGW advocates often disparage "consensus" as a refuge for scoundrels and bad science. However, my experience is that consensus is in reality the cornerstone of science.

Science is a multi-layered, collective, and impersonal process consisting of three parts. First is the individual scientist testing hypotheses according to the norms of his or her field. Second, the results of the individual scientist undergo peer-review and are published for the community to evaluate. At this point a result may be considered preliminary, but not proven. Third, important claims are then re-tested in the "crucible of science" -- they are either reproduced by independent scientific groups or they have their implications tested to insure consistency with the existing body of scientific knowledge. After enough tests/reproductions, a CONSENUS emerges that the idea is correct.

In the end, claims that are repeatedly verified by the scientific community (e.g., the Earth is warming, DNA is a double-helix, CFCs destroy ozone) eventually come to be accepted as true.

A good example is Einstein and his theory of general relativity. When he published his theory, it was not immediately accepted. However, it was rigorously tested by other scientists, most famously by Eddington's observations of star positions during eclipses, and eventually it was accepted as being "true" by the community. In other words, a consensus emerged that it was correct. At that point, people moved on to the next question, using Einstein's theory as a building block to the next interesting scientific question.

The key point here is the importance of consensus. After an idea is sufficiently well tested, everyone simply accepts the idea and people move on. While a scientific consensus might turn out to be wrong, for important and well-tested ideas (e.g., smoking causes cancer, the Earth is warming), it's exceedingly unlikely.

26 comments:

Bill F said...

"they have their implications tested to insure consistency with the existing body of scientific knowledge."

But what to do when the "existing body of scientific knowledge" may be insufficient to determine if consistency exists or not? Ask a geologist educated in the 1950s about continental drift. He will tell you that there was great scientific consensus that the theory was consistent with the existing body of scientific knowledge. And then he will tell you how it was dead wrong because we simply didn't know enough to know how inconsistent it was. So is our body of knowledge sufficient to justify consensus that man is "significantly" responsible for global warming? You believe yes, and I believe no...so where will we find that consensus we all want?

Anonymous said...

Bill,

I understand where you are coming from. There will never be enough evidence to convince somebody like you. If you want to wait for a 100% certain answer to the question of life, you better make yourself comfortable, sit back and have a cigar. In the mean time the rest of us can get on with things.

Mark UK said...

The previous comment should have stated my name, not anynomous. Apologies.

chris weaver said...

hey, i posted this comment on the previous thread, but it seems to fit better here ...

*******

dr. j: you said, "Chris, you will be hard pressed to find "consensus" as part of the scientific method and principles too, that is an invention of the societies and media for this topic alone, I have never heard of it in my 40 years in science before."

i agree with dr. j 100% on this. consensus is not part of the scientific method at all (i'm unsure if dr. j is claiming i said it was, or what, but no matter ...).

the way it works, actually, is that application of the scientific method by members of the scientific community may produce the social construct that we call "consensus" around certain specific scientific issues, if the scientific arguments are compelling. or it may not.

this sort of consensus, being a potential by-product of the scientific method and not an element of it, is, however, an important element of scientific advisory activities. in the specific case of scientific assessments like the IPCC, the goal is to accurately portray the consensus of the peer-reviewed scientific literature as an informational tool for policy makers.

now, there are many things to examine in that approach - for example, should an assessment of the peer-reviewed literature try to reflect some kind of consensus view, or should it instead focus on the differences and breadth of opinions existing in the literature? which is more useful to policy makers, and does it depend on the particular problem under consideration? i'd like to see science policy types spend more time on that question.

bill f: i think i understand what you're trying to say, but i would still argue that saying, "i might not have any better theories, but i'm pretty sure we don't know enough about how this complex system works to accept your theory either," is a pretty weak position. you're talking about a very passive kind of scepticism that to me seems out of character with the true spirit of scientific inquiry. it is this sense of our imperfect understanding that drives the discovery process - to whit: "you don't like my theory? you think it's too neat and tidy and ignores all those things we know we don't know? fair enough - now go do something about it and get back to me when YOU'VE put something on the table for ME to criticize, like i've done. until then i'll keep saying what i believe the scientific evidence tells me."

Andrew Dessler said...

All-

I think that part of the problem is in our definition of "scientific method". I define it to be the entire process, soup to nuts (whatever that means!). Viewed that way, I think consensus does play an important role. However, if one defines it to be the process employed by an indificual scientist to run an experiment (hypothesis-experiment-conclusion), then I agree that consensus doesn't enter into it there.

Chris-

I like your point about "passive skepticism." That's what Bill Gray does, and it's not helpful because he makes no specific testable criticisms. Without that, then the criticism does not sharpen our knowledge.

Regards

Dr. J said...

Dr. Dessler, you can't just dream up a definition of something and say it proves you are right, very unusual tactic in an argument, for a scientist, but not a politician. The scientific method and principles have a distinct long lived definition, maybe you should petition the NAS to change it to fit their political leanings today.

But really, on climate change, give me the defintion of what the consensus says, about what is there a consensus? And by whom? do all the scientific goups (collectively and individually) agree on this defintion of climate change and when was this consensus reached and by what process? If you reference me to the IPCC dogma, fine, but then that becomes your definition of the consensus, and so you should read it thoroughly, I think many on this and other liberal blogs will disagree and claim it doesn't go far enough.

Bill F said...

Mark,

I am not waiting for a 100% certain answer. There really is no such animal in science. There are just a number of variables that have the potential to be VERY important in how the climate system responds to forcings that I don't believe we understand well enough to say whether they are significant or not. Today's news story on New Scientist about mis-estimated CO2 uptake by plankton is a good example. For many years, we thought we understood that aspect of the system, but now, new data has shown that we were off by as much as several billion tons per year of CO2. That we created models based on that previous understanding that matched the climate trends tells me that we don't understand the system well enough to model it for any significant length of time. It may be that some other process is taking in the CO2 not taken in by the plankton, but without knowing what that process is, it leaves a gap in our understanding of the system that is large enough to fit all the Kyoto cuts through.

Is my skepticism passive? Sure...I am a geologist with a day job, and studying the science around climate change is a hobby for me, not an occupation. So if I am not convinced by the science presented to me, I have a right to express my skepticism, and am not under any obligation to take a 6 month sabbatical from my job just to publish a paper expressing my doubts so that they can be considered "non-passive" and "legitimate".

Bill F said...

Here is another question for you Mark. Read this paper:

http://www.sciencemag.org/cgi/content/full/312/5775/900?ijkey=zIUpgFJE2fXf.&keytype=ref&siteid=sci

Do you not find it the least bit interesting that a process tied to the heat flux from the earth's core (formation and strength of the earth's magnetic field) showed a marked change in trend beginning in the mid 1800s? Does it not interest you to know that recent previous glacial periods have been tied to periods of high intensity in the earth's magnetic field and that low intensity has been tentatively correlated to warmer periods? Does seeing something like that involving a process where very little in the way of real measurement has been performed (measurement of non-radiation heat output from the core) not cause you to question even ever so slightly whether or not our lack of knowledge about that process might somehow cloud our ability to determine exactly what affect it has on our climate?

That is just one of several examples where our current understanding of factors potentially affecting climate is lacking. I don't know if it has a significant effect at all or if the effect is in fact reversed (climate change at the surface causes increased heat storage and magnetic changes). But I do know that with that and other uncertainties out there, I am not convinced that our current understanding justifies the building of a consensus around it just yet.

Andrew Dessler said...

Dr. J-

I'd be interested to know where your definition of science disagrees with mine.

As far as the scientific consensus, here's my reading of it:
1) Earth is warming
2) Humans are likely responsible for most of the recent warming
3) Warming over the next 100 years will be a few degrees C

I do think the IPCC reports neatly summarize the consensus. These reports (WG I in particular) has been supported by policy statements by the AGU, AMS, and AAAS (although not the society for petroleum geologists --- I can't remember its exact name --- which I found to be really interesting). It was also reviewed and endorsed by a National Academy committee. Overall, the IPCC reports are the most thoroughly reviewed documents in the history of science.

Regards

Mark UK said...

Dr J.

We will have to simply agree to disagree. I think Dr Dessler has nicely listed what the consensus is today. A consensus that has come to exist not overnight but as the result of thirty odd years of science.

Within that consensus there are many, many debates and disagreements. Will the warm water circulation in the Northern Atlantic switch off and cause a cooling in Europe? Or will changes in climate in the Amazon have a much greater effect?

You can write books full of questions not yet answered. Point is that we are conducting a major experiment with the only place suitabe for us to live in.

Models have predicted more droughts in Europe and this is exactly what has happened over the last ten years. Just one small example.

For some reason climate change discussions often lead to very intense responses, not seen in other environmental discussions and issues. Could that be because people are scared that to admit climate change is to accept part of the responsibility? Having to make lifestyle changes?

For me, it is really strange for the US not to want to be part of an emissions trading scheme. No country on the planet is better at constantly increasing the efficiency with which it uses energy. Building energy efficient houses leads t lower energy bills. Having more fuel efficient cars leads to lower gas consumption.

Not to mention the fact the most of the rest of the planet has decided that the evidence does show that climate change is caused by humans. That is a very large market of industries working on improving their efficiency. The US can either join in or be left behind.

The good news is that the majority of people in the US now realize that human actions are contributing to climate change. I am positive there will be a major shift in US policy in the next five years... It makes to much sense.

Mark UK said...

This is just a really great animation of the earth and its environmental systems by NASA... Must see for all... A tour of the crysphere...

http://www.youtube.com/watch?v=OKRhy-oeqtM

Dr. J said...

Dr. Dessler,our definition of the scientific method differs a great deal , when you say "I think that part of the problem is in our definition of "scientific method". I define it to be the entire process, soup to nuts (whatever that means!). "

My definition and the rest of the scientific world's, is quite different, perhaps you could google it and see if you don't know, you can also find it in any textbook on intro science.

As for the IPCC, yes they have a statement, but it is full of qualifers, and if you are happy with that as your "consensus" definition on man's role in cimate change so be it, it has enough waffle words and loopholes to drive a Mack truck through.

Andrew Dessler said...

Dr. J-

Rather than simply telling me I'm wrong, it would be more useful if you could tell me specifically what part of my definition you disagree with.

Thanks!

Dr. J said...

All of it, the scientific method is not the entire process from soup to nuts, whatever that means. It is a standard and time tested and accepted (more than consensus, but unamimous, except for you I guess)method by which science forms and tests and modifys hypotheses based on experimentation and consideration of new data and approaches. It does not include votes, polls, or consensus building sessions at scientific societies and their administrative boards issuing press releases and "position policy" statements. It seems to me your vague, general definition of the scientific method is not even close to mine and all scientists I know.

Dr. J said...

Try this one Dr.D.
http://teacher.pas.rochester.edu/phy_labs/AppendixE/AppendixE.html

Dr. J said...

Oops to long, try this, but I'm sure you know the definition.
http://tinyurl.com/mp3xk

Andrew Dessler said...

Dr. J-

I agree completely with the write-up in the link, and I think it's the same process as I described.

Again, I ask you where my description is wrong. You are just telling me that "consensus" is not a part of the science that you practice ... so how do you determine whether a scientific claim is true or false?

Regards

Dano said...

science forms and tests and modifys hypotheses based on experimentation and consideration of new data and approaches.

I ask again what specific evidence Dr J has to back his implicit claim.

Evidence would include a conclusion that was changed under peer pressure.

Best,

D

EliRabett said...

Dr. J. Science speak. is always full of qualifiers. Drives my wife nuts.

Dr. J said...

Dr. D, let me phrase it this way, what theory (or theories)other than AGW theory has included a "consensus" position statement from the NAS, NRC, AGU, IPCC, etc.? I recall the Wegener plate tectonics theory that was derided and ridiculed for decades, until evidence and research slowly mounted and it was accepted by most geoscientists and was universally taught with the old theories slowly fading away. But this was due to the scientific method beng applied, not a vote, poll, or consensus process undertaken by politically inspired and oriented scientists. AGW has a taint and smell that is NOT science, even you can recognize this I hope.

Andrew Dessler said...

Dr. J-

If you have to go back 80 years to find an example of where the "consensus" was wrong, then I think you've just proven how robust a consensus position really is. While there are well-known examples of consensuses being wrong, a strongly held consensus almost always turn out to be right.

As a policymaker, I think you have to start with the assumption that a strongly held consensus is correct.

I would also note that a consensus can be weak, in which case it's a lot more likely to be wrong.

Regards

Dr. J said...

Dr. D, you obviously have never taken an earth science course (or forgot it), as I suspected from your comments and attitudes here. Dr. Wegener died in 1930, and developed his theory in 1915 or so, but it was not generally accepted (but not a consensus process even then) until the late 1960s and early 1970s. Hardly 80 years ago. And I am not saying consensus is wrong, I'm just saying it is not part of the scientific method or scientific principles. Again, point out where I am wrong and show me that many scientific theories are developed and proven by consensus processes by NAS, NRC, IPCC, UN, AGU, etc., rather than by the scientific method. AGW theory is the only one I have ever seen done this way, doesn't that seem odd to you? Guess not.

Andrew Dessler said...

Dr. J-

I think we talk about different things when we talk about consensus. I think you consider the IPCC, etc. to be what I mean by consensus ... but that's not true. The IPCC is an assessment, and that's not a part of the scientific process at. Rather, assessments are written for policymakers by scientists.

When I talk about consensus, I talk about the informal process whereby scientists evaluate research and determine what claims are correct.

Regards.

Ian Forrester said...

Dr. J, where do you get your idea that consensus "include(s) votes, polls, or consensus building sessions at scientific societies and their administrative boards issuing press releases and "position policy" statements"?

Consensus is simply an agreement of opinion.

In the case of AGW all of the peer-reviewed science, yes all of it, supports the case for AGW. There is NO dissenting evidence for the contrary in the peer reviewed scientific literature. Thus there is agreement in principle that qualifies as a consensus. No votes polls etc. are needed.

If any scientist had genuine proof to the contrary he would be eligible for a very lucrative contract from the Industrial naysayers. No scientist has stepped up to the plate with such evidence.

Ian Forrester

Ian Forrester said...

Here is my view on how the term "scientific consensus" came into being.

The reason that “scientific consensus” is being used over the past 20 or so years and not before is that previously science was an honorable profession and scientists sought the “truth”, whatever it might be. Starting recently, some scientists resorted to dishonorable techniques. These scientists were funded by groups who were opposed to what the honest scientists were saying about their products (tobacco, asbestos, CFC’s etc.). These groups of rogue scientists used a variety of deceitful techniques to muddy the water on what was fact or fiction. Scientists working in those fields could see through this strategy but the rogue scientists were not trying to convert them but were focusing their untruths on the lay public and on politicians who they hoped would act favorably when it came to law making with regard to their products.

They were very successful in this regard, for example, tougher laws on tobacco were delayed for approximately 20 years. There are still ongoing arguments in the lay press (not the scientific literature) about VCM, even though VCM was accepted as a carcinogen over 30 years ago.

Thus it was necessary to show that scientists as a group disagreed with the industry funded charlatans and thus the term “consensus” was adopted to show that true scientists were in complete disagreement with the unscrupulous ones.

These unscrupulous scientists used a number of techniques to try and con politicians and the lay public. In the case of AGW they employed false petitions to show that large numbers of scientists agreed with them (thus disproving the so-called consensus). Examples of these methods include the Heidelberg Appeal, the Oregon Petition, the Leipzig declarations, the letter signed by 60 “climate scientists” sent to Prime Minister Harper (at least one of the signers has stated publicly that he was misinformed as to what he was signing).

The number of scientists opposing the “consensus view” is very small. However, they are all related and appear together in a number of places, see Friends of Science” and Exxon Secrets. The AGW deniers (please do not call them “skeptics” since all good scientists should be skeptical) trace their roots back to Fred singer and Fred Seitz who, if the US Statute of Limitations had not intervened would have been found guilty of fraud in relation to their tobacco lobbying techniques. They have used similar techniques in their anti AGW lobbying.

In my opinion, anyone hitching their wagon to that group is guilty by association of scientific malfeasance.

Ian Forrester

Ian Forrester said...

bill f said:

"Ask a geologist educated in the 1950s about continental drift. He will tell you that there was great scientific consensus that the theory was consistent with the existing body of scientific knowledge. And then he will tell you how it was dead wrong."

bill pleaase tell me how "continental drift" is dead wrong and "plate tectonics" is correct?

The work on plate tectonics did not disprove continental drift it just explained in better terms and more detail the underlying processes involved.

Please do not try and confuse people on basic science understanding by introducing specious arguements, it may work in high school debates but not in the world of science.

Such bogus arguements shed a lot of light on your understanding of AGW and your motives behind those false beliefs.

Ian Forrester