The same survey says 11.29% or 8.44%
The National Survey on Drug Use and Health asked 134,897 adults across 2022 to 2024 about depression, among much else.1 Counted as they answered, 11.29% reported a past-year major depressive episode. Weighted to represent the adult population, the figure is 8.44%.1 Both numbers come from the same people, and only the weighted one describes the country.
Why the weights move the number
The survey is a stratified multistage probability sample. Some groups are over-sampled on purpose, so that estimates for them are precise enough to publish, and every respondent carries a weight saying how many people they represent.2 Young adults are among the over-sampled groups, and they report depression at much higher rates, so the unweighted figure overstates the national rate by about a third.2 A weight undoes the over-sampling. Each young adult counts for fewer people in the population than an older respondent does, so the young adults together count for their true share.
| Measure | Unweighted | Weighted to the population |
|---|---|---|
| Major depressive episode, past year | 11.29% | 8.44% |
| Marijuana, past month | 19.32% | 16.22% |
| Alcohol, past month | 52.65% | 51.80% |
| Cigarettes, past month | 14.34% | 14.86% |
Depression and marijuana move most, and both are concentrated in younger adults, who are over-represented among respondents.1 Cigarette use moves slightly the other way. The direction and the size of the shift depend on who was over-sampled and on how the measure varies across those groups, which is why no rule of thumb replaces computing both.
Who is in the sample matters as much as how each person counts. The three survey years hold 174,407 respondents, and 34,875 of them are aged 12 to 17, who answer a different set of questions and were never asked the adult depression question.1 They are a restriction on the population, so every figure in the analysis describes adults 18 and over.1
The margin of error moves too
The design changes the precision of an estimate as well as the estimate itself. In the same analysis, standard errors that respect the survey’s strata and replicates were 1.74 to 1.96 times larger than naive ones, so a confidence interval built without the design would be roughly half as wide as it should be.1 With a sample this large, no conclusion reversed, but the reported precision would have been wrong.
A margin of error measures precision and nothing more. It can be small on a sample that describes the wrong population, because it says how much an estimate would vary across repeated samples drawn the same way.3 Weighting fixes who the estimate describes, and the design-based margin says how sure you can be.
Survey numbers often lose this context on the way to a slide. A press release quotes one figure, a vendor deck repeats it, and by the time it supports a decision nobody can say whether it was weighted or how wide its interval was.
What to do next
- Analyst. Use the survey’s weights and design variables, meaning its strata and its clusters or replicates, for every estimate you publish, and report the design-based margin of error with it.
- Analyst. When you show an unweighted figure while exploring the data, label it as describing respondents, and keep it out of any headline.
- CIO. Before a survey figure goes into a board paper, ask whether it was weighted and how wide its margin of error is.
Go deeper
The survey analysis computes every descriptive figure both ways, with the R pipeline and a Python notebook behind it. Chapter 74 of the Visual Handbook explains the margin of error and the three things that drive it, and chapter 70 designs and analyzes a survey from start to finish.
Try it
To weight a survey yourself and check the result against a known answer, open Quote a survey figure with its weights and margin of error, whose notebook runs in Colab with nothing to install.
Found this useful? Follow me on LinkedIn for the next one.
Working on this? Connect and tell me what you are seeing.
Read the survey analysis and run its notebook yourself.