chris-said.io

What is Chris Said?

Chris Said is the personal website and blog of a data scientist who writes about statistics, technology, and science. The site is primarily an archive of essays and technical posts, organized from newest to oldest, rather than a company, product, or news publication.

What you’ll find there

• Applied statistics and experimentation: posts on A/B testing sample sizes, variance after scaling and summing, empirical Bayes methods, cross-entropy, and instrumental variables.
• Science and public-health analysis: essays on respiratory infection studies, air quality metrics, pandemic virus prediction, antigen versus PCR testing, and teen depression correlations.
• Technology and risk: topics such as AI containment, solar storm and EMP risk, data pipelines, and MathJax upgrades.
• Occasional policy and ethics commentary: for example, scientific misconduct, whistleblower compensation, and charity investment.

Who it’s for

The strongest fit is a reader with some quantitative background—data scientists, analysts, researchers, or statistically literate readers who want concise explanations of methods and evidence. The writing assumes comfort with concepts like correlation, Bayesian updating, and experimental design, though the posts often aim to make those ideas accessible.

How to use it

Start with the Selected Blog Posts list if you want the author’s own highlights, then move to the Full Blog Archive for chronological browsing. For a practical next step, pick one selected post that matches your work—for example, the A/B testing series if you run experiments—and read it alongside the related parts to see how the argument develops.

A useful comparison

The main trade-off is depth versus breadth: the site rewards readers who already have a quantitative frame, but it is not a structured course or a general-interest blog. It is best treated as a personal archive—useful for specific topics, less so for comprehensive coverage of any one field.

How can I improve my A/B testing sample size using Chris Said's methods?

Chris Said's blog is a working data scientist's notebook, not a course. The relevant post for your question is "Optimizing sample sizes in A/B testing," a three-part series from January 2020. The approach is worth reading if you are deciding how long to run a test, what effect size to power for, or whether a smaller-than-textbook sample can still answer your question.

What the method is about

The series treats sample size as an optimization problem rather than a fixed convention. Instead of asking "how many users do I need for a conventional 80% power calculation," it asks what you actually gain from each additional observation and what that costs you in time, traffic and missed opportunities. The general summary post lays out the framework; the later parts work through the details.

How to apply it

• Start from the decision, not the formula. Write down what you will do differently if the test wins, loses, or comes back inconclusive. If the inconclusive branch leads to the same action as the losing branch, you may not need the sample a standard power calculation demands.
• Model the cost of waiting. Every extra day of a test is a day you are not shipping the better variant, and a day of exposure to the worse one. Said's framing makes that trade-off explicit rather than treating runtime as free.
• Accept wider uncertainty when the stakes are low. If a wrong call is cheap to reverse, a smaller sample with a wider interval can be the rational choice.
• Reuse information across tests. His related post on empirical Bayes for multiple sample sizes is the natural companion: when you run many similar experiments, earlier results inform how much you should trust a small new one.

Where it fits and where it doesn't

This suits product teams running frequent, reversible experiments on a steady traffic stream, where the main cost is time and the decisions are incremental. It is a poor fit for one-off, high-stakes or regulated tests where a pre-registered conventional power analysis is the point—there, the value is in the defensible procedure, not the optimization.

A concrete reader scenario: a growth team has 4,000 daily signups, wants to test a checkout change, and can only reach the standard sample size in five weeks. Said's approach would have them ask whether a two-week read with a wider interval still supports the ship-or-kill decision, and whether the extra three weeks buy enough certainty to justify the delay.

Next step

Read the general summary first, then the two follow-up parts, and check the full archive for the empirical Bayes post if you run many small tests. For a second, more conventional reference on power and sequential testing, see Evan Miller.

What are the practical applications of empirical Bayes for multiple sample sizes?

Empirical Bayes for multiple sample sizes is most useful when you have many small groups and want to estimate each group's true rate without overreacting to noise.

The practical pattern: estimate a prior from all groups together, then shrink each group's observed estimate toward that prior. Groups with little data move more; groups with lots of data barely move. This is the core idea in Chris Said's post "Empirical Bayes for multiple sample sizes."

Where it pays off

• A/B testing across many segments. You rarely run one test; you run dozens of variants, countries, or device types. Empirical Bayes keeps you from declaring victory on a segment that happens to have a lucky small sample.
• Rankings and leaderboards. When you rank users, products, or schools by a noisy metric, raw rankings are dominated by small-sample extremes. Shrinkage produces a more stable order.
• Small-area estimation. Counties, clinics, or classrooms with few observations can borrow strength from the overall distribution.
• Sports and performance stats. Batting averages, conversion rates, and per-player metrics improve when early-season noise is shrunk toward a league baseline.
• Quality and reliability monitoring. Defect rates per production line, with uneven volumes, benefit from the same logic.

The trade-off to weigh

Shrinkage trades a little bias for a large reduction in variance. If you genuinely expect groups to differ a lot, a strong prior will pull real outliers toward the middle and hide them. The fix is to model the prior's spread rather than assume it, which is exactly what the multiple-sample-size approach does.

A concrete decision rule

Ask: "Would I trust this group's estimate if it had ten times less data?" If not, empirical Bayes is probably appropriate. Start by plotting the group estimates against their sample sizes; a funnel shape is the signal that shrinkage will help.

For a worked example and the estimation details, the original post is the place to start: Chris Said.

Why does Chris Said argue that small correlations can drive large increases in teen depression?

Chris Said's argument is about how a small statistical relationship, measured across a whole population, can still produce a large change in outcomes when it shifts the average. If social media use has only a small correlation with teen depression, that does not mean the effect is trivial: a modest correlation applied to millions of teenagers can move a meaningful number of people across the threshold into depression, and small average shifts can look dramatic in aggregate rates. The post is part of his broader writing on statistics and science at Chris Said.

How to read the claim

The key distinction is between individual-level and population-level effects. A correlation of, say, 0.1 tells you little about any one teenager, but if it applies broadly, the cumulative change in the number of depressed teens can be substantial. This is why "small correlation" and "large increase" are not contradictory.

A practical way to test the argument is to ask:

• What is the base rate? A small shift near a diagnostic threshold can change many diagnoses.
• How many people are exposed? Large populations amplify small average effects.
• Is the correlation causal? Correlations can reflect reverse causation, confounding, or shared trends.
• What is the counterfactual? Compare the observed rise with what would have happened without the change.

For a reader trying to evaluate claims about teen depression and social media, the useful next step is to look for the underlying effect size and the population it applies to, not just the headline correlation. A small coefficient and a large real-world change can coexist, but only if the exposure is widespread and the outcome is sensitive near the relevant threshold.

What is the instrumental variables analysis approach for non-economists?

Instrumental variables (IV) is a statistical technique for estimating a cause-and-effect relationship when a simple comparison won't work because something unobserved is tangled up with both the cause and the effect. Chris Said's post "Instrumental variables analysis for non-economists" (13 Mar 2021), listed on Chris Said, is aimed at readers without an economics background—so it introduces the idea in plain terms and works through the intuition rather than assuming econometrics training.

The core idea

Suppose you want to know whether X causes Y, but people who differ in X also differ in other ways you can't measure—motivation, income, health, whatever. A direct comparison of X and Y will conflate the effect of X with those hidden differences. An instrument is a third variable, Z, that shifts X but has no direct path to Y except through X. When such a variable exists, it lets you isolate the part of X's variation that is effectively random with respect to the confounders, and estimate the causal effect from that slice.

The classic example is distance to a college as an instrument for years of schooling when studying earnings: distance nudges how much education someone gets, but plausibly doesn't affect wages except through education. The post is a practical introduction to this logic, including why the instrument must be relevant (it actually moves X) and why the exclusion restriction (it affects Y only through X) is the assumption that carries all the weight—and the one that's hardest to defend.

What the non-economist version changes

Standard IV treatments lean on two-stage least squares, weak-instrument diagnostics and asymptotic theory. A non-economist framing tends to:

• Lead with the intuition and a causal diagram before any algebra.
• Explain why ordinary regression fails here, using a concrete confounding story.
• Show the mechanics of the two-stage procedure as "predict X from Z, then use the predicted X in place of X."
• Be honest that a bad instrument can be worse than no instrument at all.

When to reach for it

A practical next step

Before adopting IV, write down your proposed instrument and ask two questions: does it demonstrably move the treatment, and can you tell a story for why it has no other route to the outcome? If the second answer is vague, the estimate will be fragile no matter how clean the code looks. For the full walkthrough, read the original post on Chris Said. Readers who want the formal treatment can turn to standard econometrics texts, but the blog entry is a good first pass at the reasoning.

How does Chris Said explain cross-entropy and its common confusions?

Chris Said’s post “Things that confused me about cross-entropy” is aimed at readers who already know the basic machine-learning definition but keep getting tripped up when they see cross-entropy used in different contexts. The post is a clarification piece rather than a from-scratch tutorial: it works through the points where the formula, the information-theory story and the practical loss function seem to disagree.

What the post is for

• Audience: data scientists, students and engineers who can compute cross-entropy but want to understand why it is written in several apparently different ways.
• Use case: debugging your intuition before you use cross-entropy as a classification loss, a model-comparison metric or a way to measure uncertainty.
• Trade-off: it assumes familiarity with probability and logarithms. If you need a first introduction, start with a general statistics or machine-learning resource and return to this post for the confusing parts.

Common confusions it addresses

The recurring theme is that cross-entropy is one idea wearing several costumes. Readers often meet it first as a loss function for classifiers, then later as an information-theoretic quantity, and the notation changes between the two settings. Said’s post is useful because it separates those views instead of pretending they are obviously the same thing.

Typical sticking points include:

• Sign and direction: whether you are minimizing a loss or measuring the cost of encoding data with the wrong distribution.
• True labels versus predicted probabilities: in classification, the “true” distribution is often a one-hot vector, which makes the formula look simpler than the general case.
• Log base: changing the base rescales the value, so comparisons only make sense when the base is consistent.
• Averaging versus summing: whether the reported number is a total or a per-example mean changes how you interpret it.
• Relationship to entropy and KL divergence: cross-entropy is not the same as entropy, and the difference is the KL divergence between the true and predicted distributions.

A practical way to keep them straight

Ask what role each distribution plays. If one distribution is fixed and represents reality, and the other is your model’s guess, you are looking at cross-entropy. If both distributions are your model’s and you are comparing them, you are probably looking at KL divergence. If you are measuring the cost of the true distribution under itself, that is entropy.

For a concrete reader scenario: you train a classifier, see a loss value of 0.35, and wonder whether that is good. The number is only meaningful relative to the number of classes, the log base and whether it is averaged. Said’s post is a good companion when you need to explain that number to a colleague without hand-waving.

Next step

Read the post alongside a worked example of your own: take a three-class prediction, compute the cross-entropy by hand, then compare it with the entropy of the true labels and the KL divergence between the two distributions. Seeing the three numbers side by side is usually what makes the distinctions stick. For a broader treatment of the underlying statistics, Chris Said collects his related posts on the same site.