Stupid Science Tricks
I'll warn you right now, this is likely to be a series, because there are a lot of stupid science tricks to talk about.
What, you may ask, is a stupid science trick? It's when someone is using the façade of Science to pass off something that is, well, less than science. Of course, that means we need to talk about what "science" really is, and that can be a little bit tricky, because there's no one who can finally define it. It's not a thing, it's a system of beliefs, and as with other belief systems, it can be a little hard to define. (Consider, for example, the arguments over what constitutes a good Christian.)
Still, there are some common characteristics we can identify. Science is an attempt to understand and explain the world based on some assumptions: that there is a real world outside of ourselves; that this real world can be understood and explained; and that those explanations are true for everyone, so they can be tested and confirmed, or fail the test and be discarded.
We've built up a bunch of social processes around these assumptions, something I've called the "social contract of science", that establish some basic rules: when you are doing science, you publish your results so that others can see them and criticize them, and you make this easier by including in the publication full details of your methods, and by keeping your data and making it available to others.
Like other social processes, real-world science isn't being done by saints, and the social processes can be messy, but over time science has proven to be self-correcting. Sometimes, especially as people get grants and build up reputations in the scientific world, that self-correction can be a little slow. Add in politics, and the self-correction can be even slower, the stakes for refutation higher, and the discussions can get just a little bit ugly.
Of course, regular readers of this column seeing politics and ugly discussions in the context of science immediately think of the climate change debate; sure enough, that is the topic of today's stupid science trick.
Undoubtedly, you've heard quoted the result from Cook and others that "97 percent of scientists agree that global warming is caused by humans". (If not, have a look at this Google search.) This has been a subject of a succession of stupid science tricks.
The Cook et al study basically worked like this: a collection of papers were selected by computer, and rated by readers on a 1 to 7 scale from "Explicit endorsement with quantification" to "Explicit rejection with quantification."
Richard Tol examined the paper statistically and found:
The claim that 97% of the scientific literature endorses anthropogenic climate change (Cook et 15 al., 2013, Environmental Research Letters) does not stand. Numbers are padded with irrelevant papers. A trend in composition is mistaken for a trend in endorsement. Reported results are inconsistent. The sample is not representative. Data quality is low. Key results cannot be reproduced or tested as data disclosure is incomplete.
Stupid Science Trick #1: Don't Understand the Results
This one is a favorite of journalists. If you read the headlines in, say, the Washington Post
97 percent of scientific studies agree on manmade global warming, so what now?
they read as if 97 percent of all studies agree. Unfortunately, that's not even what Cook et al say: the real result was that 97 percent of studies that they classified as taking a position on climate change agreed -- but 66 percent of the papers they examined took no position at all.
Stupid Science Trick #2: Be Caught Misclassifying Data
A number of authors whose papers were rated as taking no position actually were well-known climate skeptics, and objected firmly that their papers took an explicitly negative position.
Stupid Science Trick #3: Silence Implies Consent
It gets better, though, because in the paper, the authors argue that taking no position constitutes an endorsement of the pro-AGW position:
Of note is the large proportion of abstracts that state no position on AGW. This result is expected in consensus situations where scientists '...generally focus their discussions on questions that are still disputed or unanswered rather than on matters about which everyone agrees' (Oreskes 2007, p 72). This explanation is also consistent with a description of consensus as a 'spiral trajectory' in which 'initially intense contestation generates rapid settlement and induces a spiral of new questions' (Shwed and Bearman 2010); the fundamental science of AGW is no longer controversial among the publishing science community and the remaining debate in the field has moved to other topics. This is supported by the fact that more than half of the self-rated endorsement papers did not express a position on AGW in their abstracts.
Since the range of time over which the papers were published is pretty long, this is basically arguing that there is a consensus and always had been so no one felt the need to express an opinion.
"Silence implies consent" may be good law, but it's bad science. If a paper takes no position, then you can't impute a position from that.
Stupid Science Trick #4: Science Is Not Determined by Polling
Let's just imagine someone doing a similar study about Einstein's Special Relativity (SR) in 1910. The original paper was published in 1905, and was very controversial. If you had examined papers at the time, most of the papers that took an explicit position on SR would have been against it, and most Physics papers would not have taken a position on it at all. There would have been substantial "consensus" against SR.
Of course, over time more and more attempts to falsify SR failed; now it's very widely accepted.
Watch for stupid science tricks
The point here is really that what you read about science has to be taken very skeptically, especially if you agree with it. I could have written about nutrition, genetically-engineered foods, vaccines, or a dozen other things as easily. I have a Facebook friend who is absolutely committed to the notion of human-caused global warming; she quotes this result regularly, and no criticism of it can be considered.
Of course it's just as true from the other side: I'm confident I'm going to have comments telling me that the whole notion of CO2 causing a greenhouse effect is false. (Folks, here's a hint: both the physics and the experimental evidence is strong the other way.)
The real answer -- and real science -- isto think for yourself. What are the ideas, and what are the consequences? And if someone criticizes an idea, think again about it.
A New Feature
I've set up an email address firstname.lastname@example.org for readers of this column. If you have science questions, or examples of stupid science tricks, send them to me, and I'll try to answer them in this column. I can't promise I'll get to all of them, but I'll try. Don't expect me to get into debates with you through that email, though, as I just don't have the time.