One simple vitamin can reduce  your risk of heart disease. Eating chocolate reduces  stress in students. New drug prolongs lives of  patients with rare disease. Health headlines like these  are published every day, sometimes making opposite claims  from each other. There can be a disconnect between broad, attention-grabbing headlines  and the often specific, incremental results of the medical  research they cover. So how can you avoid being  misled by grabby headlines? The best way to assess a headline’s  credibility is to look at the original  research it reports on. We’ve come up with a hypothetical research scenario for each of these three headlines. Keep watching for the explanation of the first example; then pause at the headline to  answer the question. These are simplified scenarios. A real study would detail many more  factors and how it accounted for them, but for the purposes of this exercise, assume all the information  you need is included. Let’s start by considering the  cardiovascular effects of a certain vitamin, Healthium. The study finds that participants taking Healthium had a higher level of healthy cholesterol  than those taking a placebo. Their levels became similar to those of  people with naturally high levels of this kind of cholesterol. Previous research has shown that people  with naturally high levels of healthy cholesterol have lower  rates of heart disease. So what makes this headline misleading: "Healthium reduces risk of heart disease." The problem with this headline is that the research didn’t actually investigate whether Healthium reduces heart disease. It only measured Healthium’s impact on levels of a particular  kind of cholesterol. The fact that people with naturally high levels of that cholesterol have lower risk of heart attacks doesn’t mean that the same  will be true of people who elevate their cholesterol levels using Healthium. Now that you’ve cracked the  case of Healthium, try your hand at a particularly alluring  mystery: the relationship between eating chocolate and stress. This hypothetical study  recruits ten students. Half begin consuming a  daily dose of chocolate, while half abstain. As classmates, they all follow  the same schedule. By the end of the study, the chocolate  eaters are less stressed than their chocolate-free counterparts. What’s wrong with this headline: "Eating chocolate reduces  stress in students" It’s a stretch to draw a conclusion about  students in general from a sample of ten. That’s because the fewer participants are in a random sample, the less likely it is that the sample will closely represent the target population as a whole. For example, if the broader population of  students is half male and half female, the chance of drawing a sample of 10 that’s skewed 70% male and  30% is about 12%. In a sample of 100 that would be less than a .0025% chance, and for a sample of 1000, the odds are less than 6 x 10^-36. Similarly, with fewer participants, each individual’s outcome has a larger  impact on the overall results— and can therefore skew big-picture trends. Still, there are a lot of good reasons for scientists to run small studies. By starting with a small sample, they can evaluate whether the results are  promising enough to run a more comprehensive,  expensive study. And some research requires very specific participants that may be impossible to  recruit in large numbers. The key is reproducibility— if an article draws a conclusion  from one small study, that conclusion may be suspect— but if it’s based on many studies  that have found similar results, it’s more credible. We’ve still got one more puzzle. In this scenario, a study tests a new drug for a rare, fatal disease. In a sample of 2,000 patients, the ones who start taking the drug upon  diagnosis live longer than those who  take the placebo. This time, the question  is slightly different. What’s one more thing you’d like to know  before deciding if the headline, "New drug prolongs lives of patients with rare disease", is justified? Before making this call, you’d want to know how much the drug  prolonged the patients’ lives. Sometimes, a study can have results that, while scientifically valid, don’t have  much bearing on real world outcomes. For example, one real-life clinical trial  of a pancreatic cancer drug found an increase in life expectancy— of ten days. The next time you see a surprising medical headline, take a look at the science  it’s reporting on. Even when full papers aren’t  available without a fee, you can often find summaries of  experimental design and results in freely available abstracts, or even within the text  of a news article. It’s exciting to see scientific research  covered in the news, and important to understand  the studies’ findings.