Tag Archives: critical thinking

A Rough Guide to Spotting Bad Science

Over at Compound Interest they’ve put together a handy Rough Guide to critically reading science reporting.

Image

 

Might have to invest in a poster for the wall of my office!

Do sharks love ice cream? Scientific inquiry and critical thinking

Anybody can make a ‘scientifically proven’ claim, but what does that really mean? In its essence, the scientific method aims to prove or disprove a hypothesis by providing rigorously-gathered and reproducible evidence. Though this provides a loose framework under which scientists can go about addressing a multitude of questions we must be careful to ensure that the results have come about as a result of good practice. Over the next few blog posts I will be exploring some of the key areas of scientific inquiry and highlighting the possible downfalls.

Image

(image from http://sharksicecream.com)

Causal links vs coincidence

Just because an experiment gives a certain set of results that doesn’t mean it’s been proven without a doubt; often times people will try and ‘spin’ a weak study to suggest outcomes that either aren’t strongly supported by the evidence, or have been justified by bad science. It’s key to make sure that the study is carried out in a way that goes beyond coincidence and proves the link between the treatment and the results. This means ensuring there is only one dependent variable being measured, and that the other variables are carefully controlled.

Look at the claims below. Have they been reliably proven, or are there other explanations you can think of?

  • In the 1970s a correlation was drawn between people who drink high volumes of coffee and heart problems. Hence it was said that caffeine is bad for your heart! Highlight for possible explanations: (Other explanations: people who drank large amounts of coffee were also generally highly stressed, smoked cigarettes, had poor diets and did not exercise. Unfiltered coffee also contains oils which have been shown to raise cholesterol levels.)
  • Social drinking and earnings – drinkers earn more money according to Bethany L. Peters & Edward Stringham (2006. “No Booze? You May Lose: Why Drinkers Earn More Money Than Nondrinkers,” Journal of Labor Research, Transaction Publishers, vol. 27(3), pages 411-421, June). (Other explanations: people who earn more money drink more either because they have a greater disposable income or due to stress?)
  • Image What do you think is the reasoning behind this correlation?
  • One of the most famous examples is the stunning correlation between ice cream sales and shark attacks – the latter rises sharply when the former increases. Is it because it’s harder to swim away from a shark when you’re holding an ice cream cone? Are sharks particularly fond of mint chocolate chip? (In what is perhaps a boring revelation, both ice cream sales and shark attacks tend to go up during the summer months, when more people are enjoying the surf and sun at the beach.)

In addition to the fact that these correlations are spurious at best and silly at worst, they assume that any result can be overwhelmingly attributed to one single cause. While scientific studies certainly can (and do) support a ‘one cause, one effect’ relationship in some circumstances it would be naive to assume that all changes can be traced back to one single cause. It may be reassuring to think that there’s one thing you can cut out of your diet that will help your heart health, but it’s rarely the case. More often there are multiple causes that go into effecting a change, and it’s here that a properly conducted study will begin to reveal interesting results.

Stay tuned for more scientific inquiry principles! Next up: Study size and population representation!

Facebook is evil or; Meme-sharing can be hazardous to your brain

Facebook is evil.

Now, hear me out. Some of the things Facebook does are great – allowing communication and connection across great physical distances foremost among them. However, Facebook also allows memes – “an idea, behavior or style that spreads from person to person within a culture” – to spread faster than ever before. The meme I am bemoaning today is the sharing of information without any review or fact-checking, hence allowing false and potentially harmful information to propogate across vast groups of people like wildfire.

The internet makes it very easy for sources to be lost – ask any artist or photographer who has come across their work being shared without any credit. Without embedded attribution or an easy way to trace where something came from a picture or statistic or quote can be repurposed to almost anything, and if we don’t question its veracity then all we’re doing is blindly swallowing down whatever the nameless, faceless denizens of the internet want us to believe. While it sounds rather dramatic to state it like that, I do think the inclination to take purported ‘news stories’ and other such information without questioning the source behind it to be unscientific and actively harmful to our critical thinking skills.

Sometimes all it takes is a few seconds to search for ‘horse shelter hoax’ on snopes.com to find the truth – a lot of the time people have already done the work for you! Sometimes it takes a bit more digging to uncover that what seems like a brilliant idea might not be but either way, wouldn’t you rather know the truth, rather than what a snappy headline or wittily-captioned picture tells you?

Think, people. Ask questions. Do your research. You don’t have to have a PhD to check sources, and you don’t need to call yourself a scientist to seek out the truth.

Don’t believe everything Facebook (and its millions of hangers-on) tells you. The internet has no obligation to tell you the truth, but anybody with a dedication to learning (and a snappy poster) can confirm that the truth is out there, if you look hard enough.