Suppose you want the average time a subscriber stays before canceling. The obvious move, average the tenure of everyone who has canceled, is badly biased: it ignores every customer who is still subscribed, who are precisely the loyal ones. Their tenure is not missing, it is censored: you know it is at least this long, just not the final value. Survival analysis is the branch of statistics built to squeeze information out of these partial observations, and it powers churn models, reliability engineering, and clinical trials alike.
S(t) = P(T > t),
the probability of lasting beyond time t.
Dropping still-active customers keeps only those who already churned, guaranteeing an underestimate of survival. Treating censored times as if the event happened then is just as wrong in the other direction. Censoring is information, not missingness: “survived at least 24 months” is a real fact, and survival methods are designed to use exactly that.
Time-to-Event Data and Censoring
Every subject contributes a length of follow-up and a verdict at the end of it: the event happened, or it did not (yet). Because subjects enter at different times and the study has a fixed end, most datasets are a mix of complete and right-censored observations.
An X marks a subject whose event we saw; an arrow marks a censored subject, still going when we stopped looking. Ben reached the study cutoff without churning; Dan simply dropped out of view. Both carry the same message, “lasted at least this long,” and a good method credits them for the time they survived without pretending to know when, or whether, they will eventually churn. The standard assumption is that censoring is non-informative: leaving the study is unrelated to being about to have the event.
The Survival Curve and the Kaplan-Meier Estimator
Two functions describe a time-to-event distribution. The survival function S(t) = P(T > t)
starts at 1 and steps down toward 0. The hazard h(t) is the instantaneous risk of the
event right now, given survival so far, the engine that drives the curve down.
The Kaplan-Meier estimator builds S(t) straight from the data without assuming any
distribution. At each moment an event occurs, it multiplies the running survival by the fraction who made it through
that instant, (1 − events / at-risk). Censored subjects quietly leave the “at-risk” count
at their censoring time, so they shorten the denominator without ever counting as an event, which is precisely how
their partial information gets used. The result is the familiar staircase: flat between events,
dropping at each one, with little ticks marking censored points. Where the curve crosses 0.5 is the
median survival time, a far more honest summary than a mean when the longest-lived subjects are still
censored.
Comparing Groups: Log-Rank and Cox Regression
Two questions follow immediately. Do two groups survive differently? And how does survival depend on several variables at once? Two tools answer them, both censoring-aware.
- ●The log-rank test compares two or more survival curves across all event times at once. Its null hypothesis is that the groups share one survival function; a small p-value says they genuinely differ.
- ●Cox proportional-hazards regression is the survival analog of multiple regression. It models the hazard as a baseline times
exp(β₁x₁ + β₂x₂ + ...), so each coefficient becomes a hazard ratio: the multiplicative effect on the instantaneous risk. It is semiparametric, it never assumes a shape for the baseline hazard, which is why it dominates applied work.
An HR of 2.0 means twice the risk of the event at any given instant, not twice the total or half the lifetime. Cox rests on the proportional-hazards assumption: the ratio between groups stays constant over time (the curves do not cross). When it fails, you check it (with scaled residuals) and reach for time-varying effects or stratification. As always, the model is only as good as the assumption you can defend.
Real-World Example: Subscription Churn
A streaming service wants to know how long subscribers stay and what shortens their tenure. Half the customers are still active, so their tenure is censored, exactly the setting survival analysis was built for. The companion notebook works it end to end with statsmodels.
1,000 subscribers followed for up to 24 months. Columns:
customer_id, contract (month-to-month or annual), monthly_charges,
tenure_months (the duration), and churned (1 = churned, 0 = still active and therefore
censored). Just under half churned during the window; the rest are censored.
- ●Censoring: 487 customers churned (49%) and 513 are censored, still subscribed at the 24-month cutoff.
- ●Kaplan-Meier: month-to-month subscribers have a median survival of about 16 months and a 1-year survival of 0.58; annual subscribers never reach the median in the window (1-year survival 0.77).
- ●Log-rank and Cox: the gap is real (p < 0.0001); adjusting for price, month-to-month carries 2.24 times the churn hazard of annual (95% CI 1.86 to 2.71), and each standard-deviation increase in monthly charges multiplies the hazard by 1.48.
The business reading is immediate: month-to-month customers are not a little riskier, they churn at more than double the instantaneous rate, and pricier plans churn faster still. A retention team can turn that into action, nudging high-charge month-to-month customers toward annual contracts, and the median-survival and hazard-ratio numbers put a defensible figure on the expected payoff. None of it would be trustworthy if the 513 censored customers had been ignored.
Survival Analysis in Machine Learning & AI
Time-to-event is a first-class prediction target in industry, and modern machine learning has grown censoring-aware versions of its favorite models. The prize is predicting not just whether an event happens but when, with the uncertainty intact.
| Where it shows up | What it does | Examples |
|---|---|---|
| Survival ML models | Tree and boosting methods that respect censoring instead of dropping it | Random survival forests, gradient-boosted Cox, scikit-survival, XGBoost survival |
| Deep survival models | Neural networks with censoring-aware losses that learn an individual survival curve | DeepSurv, DeepHit, Nnet-survival for churn and clinical risk |
| Predictive maintenance | Remaining useful life of equipment, a Weibull or hazard model over sensor data | Reliability engineering, RUL estimation (the Predictive Maintenance case study) |
| Customer analytics | Time-to-churn and lifetime-value models that time interventions, not just flag risk | Retention, subscription and lifetime-value modeling |
| Competing risks | Several mutually exclusive events modeled together | Death from cause A vs B, churn vs upgrade vs downgrade |
| Evaluation | The concordance index (C-index), a survival-aware analog of AUC for ranking risk | Harrell's C, time-dependent AUC |
Survival analysis is having a deep-learning renaissance. Active threads include individualized survival curves from neural networks, transformer models over electronic health records that predict time-to-diagnosis, and discrete-time and competing-risk formulations that free deep models from the proportional-hazards straitjacket. The unifying goal is calibrated, personalized time-to-event prediction, a risk score that is honest about when and about how sure it is, which the classic ideas in this chapter, censoring, the survival function, and the hazard, still anchor.
Do survival analysis in Python
The companion notebook loads the churn data, plots a Kaplan-Meier survival curve with censoring marks, splits it by contract type and compares the two curves with a log-rank test, then fits a Cox proportional-hazards model for adjusted hazard ratios on contract and price, all with statsmodels, every step with a plot.
View opens the rendered notebook instantly.
Open in Colab runs it live. To run locally, install numpy, pandas,
matplotlib, statsmodels, and openpyxl (the popular lifelines library
is a fine alternative and mirrors the same API).
🎓 Key Takeaways
- ✓Survival data is duration plus an event flag: the flag says whether the event was observed or the row is censored.
- ✓Censoring is information: “lasted at least this long” is a real fact, so you never delete censored rows or treat them as events.
- ✓S(t) and h(t): the survival function is the probability of lasting past
t; the hazard is the instantaneous risk given survival so far. - ✓Kaplan-Meier estimates S(t) as a staircase directly from censored data; the median is where it crosses 0.5.
- ✓Log-rank compares curves; Cox regression gives adjusted hazard ratios, an HR of 2 means twice the instantaneous risk.
- ✓Cox assumes proportional hazards (a constant ratio over time); check it, and use time-varying effects when it fails.
Practice Challenges
Five exercises on the churn data. Full solutions are in the companion solutions notebook.
Naive vs survival
Average the tenure of churned customers only, then of everyone. Why does each mislead, and how does the KM median fix it?
SurvfuncRight median.Kaplan-Meier curve
Plot the overall KM survival curve with its confidence band and read off S(6), S(12), and S(24).
SurvfuncRight(t, event).plot().Compare contracts
Overlay the KM curves for month-to-month and annual, and run the log-rank test. Report the p-value.
survdiff(t, event, group).Cox hazard ratios
Fit a Cox model on contract and monthly charges and interpret each hazard ratio in plain language.
PHReg.from_formula(..., status=event).fit(); HR = exp(coef).Check proportional hazards
Discuss how you would check the proportional-hazards assumption for the contract variable, and what to do if it fails.
Solutions notebook
All five challenges worked in code, naive versus survival estimates, the Kaplan-Meier curve, the contract comparison with a log-rank test, the Cox hazard ratios, and how to check proportional hazards, each with a short explanation.
Quiz: Test Yourself
Eight questions on censoring, the survival and hazard functions, Kaplan-Meier, log-rank, and Cox regression. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved.
Survival analysis handled one specialized data structure. Next, Advanced & Applied Topics takes on layered and multi-equation models. Structural Equation Modeling & Mixed Models fits systems of relationships and data with nested structure, students within schools, measurements within patients, and latent variables you can only observe indirectly.