These days studies are a dime a dozen. Everyone has one to support every cause. They are small, like the informal “What should we eat?” poll to those who compile studies about studies. For better or worse these are the best method we have right now expressing what we learn about our world. Every topic we try to find the truth about will depend on how dedicated the guy who interviewed subjects was to getting the most unbiased information.
Research is a tough business. You may have an important question that needs specific research to get answered, but you won’t get anywhere without funds. Unfortunately, those who possess the funding often have the most to gain from research that slants in their favor. We are left to figure out which research gives us findings that are the closest to the truth. In light of this search for truth I though I would throw out a few tips I use to find relatively accurate information
- Be skeptical of headlines: Overall it is great that journalists are bringing scientific discoveries to the general public. The average Joe should know more about the newest near earth, and the latest news on heart disease. What you need to keep in mind is that journalists and scientists have inherently different goals. Scientists are after very specific measurements. These measurements may tell us something about our daily lives, but they may just lead us to the necessity of another study, or they may indicate the scientist was on the wrong track. Journalists are after meaning, and if that meaningful headline can include the word “sex”, even better. They need views, subscribers, Attention! They need to make those black and white measurements into something that will give you better sex, or, you know, something else awesome. Somewhere in that transition, accuracy can be lost. I think the best practice is to take that headline as a starting point. Follow the links, glance at the study referenced, and look at a couple more articles about the topic. The truth will probably be somewhere in the middle.
- Follow the Links: I have read articles that I like. They resonate with my ideas, but they reference “the study” that claims “the thing” and they won’t lead me to the actual study. I’m not saying that I will wade through all the jargon of a real scientific study, but I do like to know exactly how many of what kind of rats were pressing how many levers. It matters. I have to let those ego-boosting, but unsubstantiated articles go. Sometimes I have to let whole warm and fuzzy ways of viewing the world go because when I do get to count the rats, I find there were only four, and there weren’t any levers. Not only that, but it takes repetition by others for an idea to really be tested. If there aren’t multiple experiments that agree, it doesn’t mean an idea is wrong, just that you can only take so much comfort in the result.
- The studies themselves matter: If you interview thirty college co-eds about, say, SEX, it could be a fairly entertaining afternoon for you, but you have not done a scientific study that can tell you much about the sexual exploits of college co-eds. In order to start gaining real knowledge about sex, or even college age sex, you need large numbers, preferably from many areas of the country, preferably interviewed by many scientists in different studies. Then you need to take the information at face value without drawing too many conclusions that aren’t given to you directly in the information.
- Who is the author?: Who is the person, business or college behind the information you are reading? What is their motivation? Obviously if enough money comes from a large corporation the study’s objectivity should be called into question. But there are lots of cases that are not so obvious. Not everyone who has a specific agenda has money, but strong biases can affect outcomes. Everyone has biases, but some people don’t try hard enough to set them aside. What website is the author associated with? Are they connected with climate change websites that have a strong opinion on climate change? Then it would be prudent to check her facts with reputable articles and studies. The truth will probably be somewhere in between. Education and experience can also affect accuracy. Is a psychologist measuring climate temperatures? They may not fully understand all the nuances of the numbers they are seeing.
These are just a handful of tips I like to use. If you want more ideas of how to decipher the information you come across look at these sources:
Here is a warning about science and headlines, along with a Ted Talk. The article is called “Science and journalism: studies have findings, not facts.”Posted on February 25, 2013 on Dragonflyeye.net by Tommy Belknap
This is a wonderful movement called “Alltrials”. Their goal is to make all clinical trials accessible, not just the ones that make good headlines, or succeed at their goals. What do you think of their ideas?
- What are your biases? What issues do you hear and just yell “Yes!”? Now go find an article that seems reputable but makes you yell “NO!”. Read it. Find a fact or two you can mull over. Don’t mock it.
- Find an author you resonate with. Research the author. Follow the links, then look for similar articles about the topic. Do the facts ring true?
- What are your best tips for finding accurate information? What is the best fact check app you know about? Where do you find authoritative science and health news?
- What would you do to make data collection and research more accessible and accurate? Would you find a way to distribute funding more fairly? Would you organize those doing the research in a different way? Would you work with the press to distribute the information differently? Would you rely more on computers and less on people? What are your practical ideas as well as your crazy ones?