pkp.sfu.ca

The fastest way to stop drowning in science headlines is to sort every story by its evidence level before you read it. A peer-reviewed study in a reputable journal is stronger than a preprint, which is stronger than a university press release, which is stronger than a headline. Most confusion comes from reading the weakest link in that chain and treating it as the strongest.

This guide gives you a repeatable routine: trace a claim to its source, judge the study behind it, spot common framing tricks, and scan a small set of outlets instead of chasing every alert.

The evidence ladder: peer review, preprint, press release

When a science story reaches you, it has usually passed through several hands. Each step can add accuracy or distortion.

The order matters because each step down removes caveats. A paper might say "associated with" while the headline says "causes." A preprint might show a promising result in mice while the coverage implies it applies to humans.

Practical rule: Before you accept a claim, find the original paper. If you can't find it, treat the story as unconfirmed.

How to trace a story back to its source

Most good science coverage links to the paper. If it doesn't, you can still find it.

1. Search the study title or authors. Copy a distinctive phrase from the article and search it alongside "study" or "paper."
2. Check the journal name. Look for it in the article or the paper itself. A journal's reputation matters, but even strong journals publish weak papers.
3. Look for a DOI. The Digital Object Identifier is a permanent link to the paper. Paste it into a search engine or doi.org.
4. Read the abstract first. It summarizes the question, method, and main finding. If the abstract doesn't support the headline, the headline is wrong.
5. Check the publication date. Old studies get recycled as "new" surprisingly often.

If the paper is behind a paywall, the abstract and any available supplementary material still tell you a lot. Many authors also post free copies on personal sites or repositories.

Judging whether the study supports the headline

Once you have the paper, ask a few blunt questions.

Sample size and population

• How many participants or subjects? Small samples (often under 30) produce noisy results.
• Who or what was studied? Mice, rats, cells, or humans? A mouse result is not a human result.
• Was it a representative group, or a narrow slice (e.g., healthy college students)?

Method and design

• Was there a control group?
• Was it randomized, or observational?
• Was it blinded? For subjective outcomes, blinding matters.
• Did the study measure what the headline claims, or a proxy?

Replication status

• Is this a single study, or does it fit a body of evidence?
• Has anyone else found the same thing?
• Is it a meta-analysis or systematic review? Those weigh multiple studies and are generally stronger.

Effect size and significance

• Statistical significance is not the same as practical importance. A tiny effect can be "significant" with a large enough sample.
• Look for confidence intervals. Wide intervals mean uncertain estimates.

Example (hypothetical): A headline says "New compound reverses aging." The paper shows improved markers in mice, n=12, no human data, no replication. That's a preliminary signal, not a reason to change your behavior.

Common framing tricks to watch for

• Single-study claims: "Science says" often means "one study found." One study is a data point, not a consensus.
• Animal versus human: Mouse and cell studies dominate early research. They matter for science but rarely justify human advice.
• Correlation as causation: Two things rising together doesn't mean one causes the other. Look for the word "associated" and treat it carefully.
• Relative versus absolute risk: "Doubles the risk" sounds alarming until you learn the baseline was 1 in 100,000.
• Preliminary presented as settled: Words like "may," "could," and "linked to" are signals of uncertainty. Headlines often drop them.
• Press release inflation: Compare the release to the abstract. If the release claims more, trust the abstract.

A lightweight routine for staying informed

You don't need to read everything. You need a small, trustworthy set of sources and a quick filter.

1. Pick 3–5 outlets known for linking to primary sources and explaining caveats. Mix general science coverage with one or two journals or aggregators.
2. Scan headlines once a day or every few days. Don't chase alerts.
3. For any story that matters to you, run the trace. Find the paper, read the abstract, check the sample and method.
4. Save the paper, not the article. The paper is the durable source.
5. Note the evidence level. Peer-reviewed human study? Preprint mouse study? Press release only? Label it in your head.
6. Wait for replication on big claims. If a finding would change your behavior, look for follow-up studies before acting.

This routine takes a few minutes per story and prevents most hype-driven mistakes.

When a finding is genuinely preliminary

Some results are early by nature: new drug targets, exotic physics, first-of-their-kind observations. That's fine. The key is to change what you do with them.

• Don't change behavior based on a single preliminary study, especially for health, diet, or supplements.
• Do note it as an interesting signal worth watching.
• Look for the next step: larger sample, human trial, independent replication.
• Be skeptical of urgency. Real science rarely requires you to act today.

If a story says "more research is needed," believe it. That's not a dodge; it's how science works.

Bottom line

Follow science news by following the evidence, not the headline. Trace claims to the paper, check the study design and sample, and treat preprints and press releases as early signals rather than conclusions. A small, consistent routine beats an endless feed of alerts.