Depression and Substance Use in the NSDUH

Summary

1. Who is in the analysis

Pooling 2022, 2023 and 2024 gives 174,407 records. The analysis sample is 134,897. The 34,875 difference is worth stating precisely, because it is not what it first looks like.

Nearly nine in ten of the dropped records are adolescents who were never eligible to answer. Treating that as missing data would be a mistake: it is a restriction on the population. Every result here describes adults, and any statement about "Americans" that includes teenagers would be wrong on this evidence.

Pooling three years also requires dividing each respondent's weight by three. Without that, the weighted totals describe a country three times too large.

Where else data is missing

Almost everything is complete. The two exceptions are the depression and mental-illness measures, and as set out above, most of that is adolescents who were never asked rather than adults who declined. Self-rated health is missing for a few dozen people. The substance and demographic variables have no missing values at all, which is unusual and reflects how much editing and imputation SAMHSA does before release.

2. Looking around first

Before any model, the simple percentages. These are weighted, so they estimate the adult population rather than describing the respondents.

The striking thing is how little moves. Across three years there is no meaningful trend in any of the four measures. That is worth knowing before reading anything into year-to-year differences, and it is a good reason to pool the years rather than compare them.

Women report depression at roughly 1.6 times the rate of men. Income shows a gradient, with the lowest bracket highest, but it is shallow compared with age or self-rated health, and section 6 shows it disappears once the other variables are known.

The outcome itself

119,664 adults (88.7%) reported no depressive episode; 15,233 (11.3%) did. This picture is the single most important thing for understanding section 7. Any classifier can score 88.7% accuracy on this outcome by never predicting the minority class, and one of ours does exactly that.

3. The survey design, and what ignoring it costs

NSDUH over-samples some groups by design. A respondent's weight says how many people they stand for, and the stratum and replicate identifiers describe how the sample was drawn. All three ship with the public-use files.

It moves the estimates

Depression and marijuana move most, both downward: 11.29% to 8.44% and 19.32% to 16.22%. Both are behaviours concentrated in younger adults, and younger adults are over-represented among respondents.

The mechanism is visible here. Within an age band the two figures almost agree. It is the mix of ages that differs between the sample and the country, so only the overall figure moves. That is what weighting is for, and it is why an unweighted national rate can be wrong even when every subgroup rate is nearly right.

It changes how precise the estimates are

Respondents are drawn in clusters rather than independently, so they carry less information than the same number of independent draws would. A model that ignores this reports standard errors that are too small.

Every standard error grows by between 1.74 and 1.96 times. A confidence interval built on the naive figures is roughly half the width it should be, and a p-value computed from them is correspondingly optimistic. Nothing here reverses a conclusion, because the effects are large and the sample is enormous, but the reported precision would be wrong.

The design-adjusted figures use the stratum and replicate identifiers as the clustering unit. This captures the clustering and the weighting but not the gain from stratification, so it is approximate and tends to be conservative. Read the ratio as the rough scale of the error, not an exact design effect.

4. Who reports depression

The age gradient is the strongest pattern in the data, and self-rated health is the strongest single correlate of all. Both are far more informative than any individual substance measure. Income shows a gradient too, but a shallow one, and it turns out to add nothing once the other variables are known.

5. Squeezing seven variables into three

Seven measures is not many, but they overlap: age, income and health move together, and so do the three substances. Principal components analysis rewrites them as a smaller set of combined scores that carry most of the same information, which makes the grouping in the next step tractable.

The variables are on different scales, so they are standardised first. Without that, a variable coded 1 to 6 would dominate one coded 0 or 1 purely through its range.

The first component carries 20.9%, the second 18.4%, and three together 55.6%. That is a modest reduction rather than a dramatic one. These seven measures genuinely do carry distinct information, so the components should be read as a convenient summary rather than as hidden traits.

What the components actually mean

This is the figure that makes PCA interpretable. Each arrow is one original variable, placed by how strongly it loads on the first two components. Arrows pointing the same way are variables that rise and fall together. Arrows pointing opposite ways move in opposition. A long arrow is well represented by these two components; a short one is not.

Two things fall out of it. The first component contrasts cigarette and marijuana use, together with poorer self-rated health, against higher income. It is close to a disadvantage-and-heavy-use axis: those three rise together and run opposite to income. The second component picks out alcohol and income moving together, against being female and in poorer health, which separates social drinking from everything else.

Note also which arrow is short. CATAG6, the age band, barely registers on either component, so age is poorly represented by this two-dimensional summary. That is a useful warning: age is the second strongest predictor of depression in section 7, and it is nearly invisible here. A variable can matter enormously for an outcome and still contribute little to the main axes of variation among the predictors.

Choosing how many groups

K-means needs to be told how many groups to find, and there is no correct answer, only a reasonable one. The elbow plot shows how tightly the groups fit as k rises. Tightness always improves with more groups, so the question is where it stops improving quickly. The bend here is around four, which is where extra groups stop buying much.

6. The four groups, and what defines them

Cluster numbers are arbitrary, so each group is named from its own profile. These are the "natural types" the analysis set out to look for, and it is worth being precise about what each one means rather than leaving the labels to do the work.

The separation is mostly about marijuana and about how many substances travel together. Drinking on its own is common and, on this evidence, unremarkable: the largest group drinks at 79.9% and has the lowest depression rate of the four, at 6.2%. It is co-use that marks out the polysubstance group, whose depression rate is more than three times as high.

7. Predicting depression, and the trap in accuracy

Which predictors matter, and what the two penalties do

Logistic regression with a penalty term shrinks coefficients toward zero to stop the model chasing noise. The two standard penalties behave differently: LASSO can shrink a coefficient all the way to zero and so drops variables entirely, while ridge shrinks everything but keeps all of them.

The two agree closely on both order and size, which is reassuring: the result is not an artefact of the penalty chosen. CATAG6 and HEALTH dominate, both far ahead of any substance measure. Among the substances, marijuana (+0.220) is roughly three times alcohol (+0.062), which is the basis for the claim that marijuana is the substance most associated with depression.

Income is the interesting case, at -0.002: close to nothing once age, sex, health and substance use are known. The gradient in Figure 3 is real, but it appears to run through those other variables rather than adding anything of its own.

A tree tells the same story differently

A classification tree asks a sequence of yes-or-no questions. Its first is HEALTH <= 2.5: is self-rated health at least "good"? That the tree independently picks health as the single most useful question, when the regression also ranked it top, is a stronger result than either method alone.

Reading down the branches, the highest-risk path is poor health combined with younger age, and the lowest is good health with older age. The tree finds no interaction the regression missed, which is itself informative: the relationships here are close to additive.

Then the model is asked to make decisions

A penalised logistic regression on the seven predictors, evaluated by cross-validation, reaches an AUC of 0.7406. AUC is the probability that a randomly chosen person with depression is ranked above a randomly chosen person without it, so 0.7406 means the model carries real signal.

Now the same model, judged by accuracy. It is 88.7% accurate. It also flags 805 people out of 134,897 as depressed, when 11.29% of the sample actually report an episode. Cohen's kappa is 0.0401, effectively zero.

The failure is in the decision rule, not the model. The ranking is good; the default 0.50 cut-off is simply far above where the probabilities live for an outcome this rare.

At 0.50 the model finds 2.7% of people with depression. Lowering the cut-off to 0.20 finds 39% of them, at the cost of flagging 15% of everyone for a closer look. Which trade-off is right depends entirely on what happens to a flagged person: a cheap follow-up question justifies a low threshold, a costly intervention does not.

The imbalance is 7.86 to 1. Class weighting and resampling are the other standard responses, and they move the same trade-off along the same curve rather than creating new information.

8. Bottom-line findings

What the groups look like

Adults fall into four recognisable substance-use profiles. A third are higher-income drinkers who drink and do little else, and who have the lowest depression rate of the four. A quarter are younger low-use adults, low on everything. About a fifth are older cigarette users with the poorest self-rated health. And about a fifth are polysubstance users, combining alcohol, marijuana and cigarettes, with a depression rate more than three times the drinkers'. If any group is a public-health priority on this evidence, it is that last one.

What predicts depression

Self-rated health and age are the strongest predictors by a wide margin, and both a penalised regression and a decision tree agree on that independently. Women report substantially higher rates than men. Among substances, marijuana is the strongest signal, roughly three times alcohol or cigarettes. Income adds nothing once the rest is known, and LASSO shrinks it to near zero on its own.

What the survey design changes

Weighting moves the national depression estimate from 11.29% to 8.44%, because the survey over-samples the young and the young report more depression. Accounting for the clustering widens every standard error by 1.74 to 1.96 times. Neither changes a conclusion here, but both change what may honestly be claimed about precision.

What accuracy hides

The clearest methodological result on this page: a model can be 88.7% accurate, carry genuine signal at AUC 0.7406, and still be useless as a classifier, because it answers "no" to almost everyone. Accuracy is the metric that conceals this, and kappa of 0.0401 is the one that reveals it. Whether the model is useful depends entirely on where the decision threshold is set, and that is a question about consequences, not about statistics.

Honest limitations

Self-report. People under-report behaviour they expect to be judged for, and both the substance and the depression measures rest on what respondents said.

No causation. Cross-sectional data. Marijuana use and depression travel together; nothing here shows which came first, or whether something else drives both.

Coarse measurement. "Any use in the past month" treats one drink and daily heavy use identically. Severity, frequency and treatment are not in these variables.

Adults only. Everyone aged 12 to 17 is excluded by construction.

Most of the story is missing. Genetics, life events, social support and care history are not recorded here, which is why AUC 0.7406 is the ceiling rather than a disappointment, and why this model should never be read as diagnostic.

Scope choices. Seven of more than 2,600 available variables were used. Race, education and other substances were available and left out, and a different selection would give a different model.

9. How this was computed

All figures are computed from the SAMHSA public-use files with pandas, scikit-learn and statsmodels, using only the columns needed rather than all 2,650. Random seeds are fixed at 1; changing them moves the cluster labels and the last digits of the cross-validated figures.

The data is not in the repository. Aggregate results are written to tools/derived/, which is committed, so every page rebuilds without it. See the code page.