Psychosocial Treatments for Cocaine Dependence

Summary

1. Who is in each analysis

This is a six-month trial in cocaine dependence, so retention is the first thing to establish. Roughly a third of randomized patients are missing from each cross-section.

Retention does not differ significantly by arm at either timepoint (chi-squared p = .52 at month 3, p = .48 at month 6). That is the reassuring version: the arms are not losing patients at different rates, so the comparison is not obviously distorted.

It is not the whole story. Patients who stay in an addiction trial differ from those who leave in ways a balance test cannot see, and the month 6 group is not the month 3 group. A treatment effect that appears at one timepoint and not the other could reflect the treatment, or it could reflect who was still there to measure. These data cannot separate those.

2. What the data looks like before modelling

Before any model, it is worth seeing the distributions. Scores are wide and heavily overlapping in every arm, which is the honest starting picture: whatever treatment does here, it does not separate the groups cleanly.

Two things stand out. IDC+GDC has the highest mean at month 3 and the ordering is visible, but the boxes overlap almost entirely, so the effect that follows is a shift in the middle of wide distributions rather than a separation of groups. And by month 6 even that ordering has largely gone.

3. The primary comparison

Each timepoint is analysed by ANCOVA: outcome on treatment arm, adjusted for the patient's own baseline score. Differences below are against group drug counselling alone.

At month 3 the model explains 0.343 of the variance and treatment matters (p = .0110). At month 6 it explains 0.252 and treatment does not (p = .2580). Baseline is overwhelmingly significant at both. Note what that means: most of what the model explains is the patient, not the therapy.

4. Assumptions, and a check that was run on the wrong model

The original analysis tested residual normality and found a violation at month 6, then investigated a Box-Cox transformation and concluded none was needed. Reproducing that turned up something worth reporting.

The normality check was run on residuals from a model containing treatment only. The model actually reported includes the baseline covariate. Those are different models with different residuals, and the assumption belongs to the model you report.

At month 6 the residuals of the treatment-only model reject normality (p = .0047), while the residuals of the reported ANCOVA do not (p = .4285). At month 3 neither rejects (.2668 and .1238). So the violation that prompted the transformation work was a property of a model that was not being reported. The reported models satisfy the assumption.

Variances are homogeneous across arms at both timepoints (Levene p = .30 and p = .37).

5. Subgroups, and how much weight they hold

The analysis also tested whether treatment effects depend on gender and race, through two-way and three-way interaction models. A striking difference appears at month 6, and it needs its denominators attached.

The highest adjusted mean is IDC+GDC at 55.19, the lowest is CT+GDC at 43.43, both among non-Caucasian women. That is a gap of 11.8 points, which would be a large treatment effect if it were real.

It rests on 14 and 10 patients. Splitting 329 patients across four arms, two genders and two race categories leaves 16 cells, the smallest holding 8. At month 3 the picture is worse: 6 of 16 cells hold fewer than ten patients, the smallest 7.

Read the intervals rather than the ordering. The cells at the extremes have intervals five points wide in each direction, and they overlap most of the middle of the table. The ordering is not stable information.

There is also a multiplicity problem that no single p-value reflects. Two timepoints, three model specifications, sixteen cells and a set of pairwise contrasts is a large family of tests, and the subgroup result is the most extreme cell in that family. Selecting the largest gap from sixteen cells and then quoting its p-value overstates the evidence by an amount the p-value does not show.

6. The recommendation, and how it was reached

The original analysis closed with a specific recommendation: assign non-Caucasian women to IDC+GDC. Because that is a clinical recommendation about a named group of patients, the chain of reasoning behind it deserves to be laid out in full rather than summarised.

The chain

The evidence, in full

Here is every arm for that subgroup, which is the comparison a treatment decision actually needs.

The IDC+GDC versus CT+GDC difference is real within this subgroup. Those two intervals do not overlap, so the comparison that generated the recommendation is not noise.

The comparison a treatment decision turns on is different. Nobody chooses between IDC+GDC and CT+GDC in a vacuum; the question is whether adding individual drug counselling beats the standard care every patient already receives. Against group drug counselling alone, IDC+GDC gives 55.19 against 49.39, and those intervals overlap substantially. On this evidence, IDC+GDC is not distinguishable from standard care for this subgroup.

So the finding supports a narrower statement than the recommendation makes: cognitive therapy appears to do worse than the alternatives for non-Caucasian women in this sample. That is worth investigating. It is not the same as establishing that individual drug counselling should be assigned to them.

Three further reasons for caution

The numbers behind it are small. 14 patients in the IDC+GDC cell, 10 in CT+GDC, 11 in the reference. Losing or gaining two or three patients would move these means noticeably.

The cells were chosen after seeing them. The highest and lowest of sixteen cells will differ substantially even when nothing is going on, simply because they are the extremes. A p-value computed on a comparison selected for being extreme does not mean what it appears to mean, and no correction was applied.

It did not replicate across timepoints. The same subgroup at month 3 had the highest adjusted mean too, and the original analysis records that it was not statistically significant there. A subgroup effect that appears at one timepoint and not the other, inside a null overall result, is the pattern one expects from chance.

7. What this analysis supports

Adding individual drug counselling to group counselling was associated with better behaviour scores at three months. That association is not present at six months. Baseline severity predicts outcome more strongly than treatment assignment at both timepoints.

What it does not support: assigning treatment by gender or race. The subgroup signal in section 6 is worth following up and is not strong enough to act on.

What it cannot address: whether the month 3 advantage faded, or whether the patients measured at month 6 were simply different. Answering that needs the longitudinal model applied to a defined analysis population with the missingness handled explicitly, rather than two independent cross-sections.

8. How this was computed

Every figure is computed from the trial data with pandas and statsmodels, and the published models reproduce exactly: R² of 0.343 and 0.252, F of 37.45 and 27.34, treatment p of .0110 and .2580. The adjusted means in section 4 reproduce the values in the original write-up to two decimal places.

The trial data is restricted and is not in this repository. Aggregate results are written to tools/derived/, which is committed, so every page rebuilds without it. See the code page.