Contents/ Part XXII · Time Series & Forecasting/ Chapter 134

Volatility & Advanced Models

Forecasting the level is only half the story. Financial returns are unpredictable in the mean yet cluster into calm and stormy stretches, memory that lives in the variance. This chapter adds the professional toolkit: reading correlograms, cross-checking stationarity, modeling volatility, and relating one series to another.

⏱️ ~26 min read
🐍 Notebook included
📊 Chapter 134

Some series defy the models of the last chapter. Daily stock returns are essentially unforecastable in the mean, their ACF is flat, so no ARIMA will beat a coin flip on tomorrow's direction. Yet they are far from random: big moves cluster together into volatile and calm regimes. The signal is in the variance, and capturing it needs a different family of models, plus tools to cross-check stationarity and to relate one series to another.

σ
This chapter covers the advanced toolkit: ACF/PACF for order selection, unit-root tests (ADF, PP, KPSS) that cross-check stationarity, ARCH/GARCH for volatility clustering, and Granger causality and ARDL for relationships between series.
🌊
The key idea: memory in the variance

A series can be white noise in its level (unpredictable returns) while its squared values are strongly autocorrelated (predictable volatility). ARIMA models the first and finds nothing; GARCH models the second and forecasts the size of moves, the basis of financial risk management.

1

Reading ACF & PACF

The autocorrelation and partial-autocorrelation functions are the classic tools for picking model orders, and for spotting where the predictable structure lives.

Correlograms tell you the order, and where the signal hides processACFPACF AR(p)tails offcuts off at p MA(q)cuts off at qtails off ARMA(p,q)tails offtails off “cuts off” = drops inside the band after that lag returns: ACF flat mean is unpredictable returns²: ACF spikes variance HAS memory Flat ACF on returns, tall ACF on squared returns → the story is volatility, not direction. (in practice, tools like AIC and auto-ARIMA search orders for you, but the correlogram builds the intuition)

The reading rules are a useful shorthand: an AR(p) process shows a PACF that cuts off after lag p while its ACF tails away; an MA(q) is the mirror image. But the deeper use here is diagnostic. Our returns have a flat ACF, no linear memory in the level, yet the ACF of their squared values is full of spikes. That single contrast is the entire motivation for the volatility models below.

2

Unit-Root Tests: ADF, PP & KPSS

The Components of a Time Series chapter introduced the ADF test. In practice you cross-check stationarity with more than one test, and the trick is that they do not all share the same null hypothesis.

Three tests, two opposite nulls, use them together ADF H₀: unit root (non-stationary) small p → stationary Phillips-Perron (PP) H₀: unit root (same as ADF) small p → stationary KPSS (flipped) H₀: STATIONARY LARGE p → stationary ADF says stationary AND KPSS says stationary → trust it our returns pass all three; a price LEVEL would fail (difference it first). If the two disagree, be cautious.

The ADF and Phillips-Perron (PP) tests share the null “the series has a unit root” (non-stationary), so a small p-value lets you reject it and conclude stationary. The KPSS test reverses the null to “the series is stationary”, so there a large p-value is the good news. Pairing ADF with KPSS is the standard confirmatory move: when both point the same way, the verdict is solid. Our returns are stationary on all three tests; a wandering price level would fail them, which is why you model returns, not prices.

3

Volatility Clustering: ARCH & GARCH

Now the marquee model. When variance itself has memory, you stop modeling the level and start modeling the variance.

GARCH models the variance: big moves cluster together turbulent calm turbulent GARCH conditional volatility (the band) varₜ = ω + α·shock²ₜ₋₁ + β·varₜ₋₁ today's variance leans on yesterday's α + β near 1 → shocks are long-lived

ARCH and its generalization GARCH let today's variance depend on recent squared shocks and recent variance. The workhorse GARCH(1,1) is var_t = omega + alpha · shock²_(t-1) + beta · var_(t-1). Fitting it to our stock gives a persistence of alpha + beta ≈ 0.96: a shock to volatility decays slowly, so a stormy day raises expected volatility for many days. The model's conditional volatility rises and falls with the clustering, and forecasting it, the size of future moves, is exactly what risk measures like Value-at-Risk need.

4

Relationships: Granger Causality & ARDL

So far, one series at a time. The last tools ask whether one series helps explain another.

Does one series help predict another? marketₜ₋₁yesterday's market stockₜtoday's stock Granger: past of market improves stock's forecast (p < 0.05) ARDL stockₜ = a·stockₜ₋₁ + b·marketₜ + c·marketₜ₋₁ Caveat: Granger causality is predictive precedence, NOT proof of cause a common driver or a faster-reacting third series can produce the same pattern, ARDL then quantifies the effect

Granger causality asks a precise, testable question: does adding the past of series X improve the forecast of Y beyond Y's own past? For our data the market Granger-causes the stock (p far below 0.05) but not the reverse, matching the built-in lag. The essential caveat: this is predictive precedence, not true causation. An ARDL (AutoRegressive Distributed Lag) model then quantifies the link, regressing the stock on its own lag plus current and lagged market returns (coefficients about 0.33 today and 0.29 yesterday). ARDL also powers the bounds test for cointegration, whether two trending series share a long-run equilibrium.

5

Real-World Example: Daily Market & Stock Returns

The companion notebook runs the whole advanced toolkit on three years of daily returns, using statsmodels and the arch library.

📂 Dataset · volatility-and-advanced-models--daily_returns.xlsx

756 trading days (2021 to 2023) of daily percent returns for a broad market index and a single stock. The stock reacts to the market both same-day and with a one-day lag, and its shocks follow a GARCH process. Columns: date, market_return, stock_return.

  • Stationarity: returns pass all three tests (ADF and PP p ≈ 0, KPSS p large); the price level would not.
  • Signal in the variance: returns' ACF is flat, but squared returns are autocorrelated, volatility clustering.
  • GARCH(1,1): persistence alpha + beta ≈ 0.96, so volatility shocks are long-lived.
  • Granger: market → stock is significant (p ≈ 0); stock → market is not (p ≈ 0.56).
6

Advanced Time Series in Machine Learning & AI

The classical tools here have modern, learned counterparts, and the questions they pose, forecasting volatility and untangling relationships, are active machine-learning research areas.

Classical toolModern / ML counterpartIdea
GARCH volatilityDeep / neural volatility models, realized-vol MLLearn the variance process from richer inputs and high-frequency data
Granger causalityNeural Granger, PCMCI, causal discoveryFind lead-lag and causal structure among many series with nonlinear models
ARDL / VARMultivariate deep models, graph neural networksModel many interacting series jointly, sharing structure across them
Regime detectionHidden Markov models, change-point & regime-switchingDetect shifts between calm and turbulent states automatically
Risk / tailsQuantile & distributional forecasting (DeepAR, quantile nets)Forecast intervals and tail risk, not just a point
🔬 Research frontier

Two threads are hot: causal discovery in time series, learning the directed graph of who-drives-whom among hundreds of series with nonlinear, neural methods, and probabilistic deep forecasting, which models the full predictive distribution (and thus volatility and tail risk) rather than a single number. Both push past what GARCH and Granger can do, yet they are validated with the very ideas in this chapter: stationarity, autocorrelation of residuals, and honest out-of-sample testing.

🐍

Run the advanced toolkit in Python

The companion notebook reads the daily returns, plots the ACF and PACF (flat for returns, spiky for squared returns), runs ADF, Phillips-Perron, and KPSS, fits a GARCH(1,1) and plots its conditional volatility, tests Granger causality in both directions, and fits an ARDL model, using statsmodels and the arch library (installed automatically if missing).

📓 View Notebook (code & outputs) ▶ Open in Colab ⬇ View / Download on GitHub

View opens the rendered notebook instantly. Open in Colab runs it live. To run locally, install numpy, pandas, matplotlib, statsmodels, arch, and openpyxl.

🎓 Key Takeaways

  • ACF/PACF pick orders: AR cuts off in the PACF, MA in the ACF; and the ACF of squared returns reveals memory in the variance.
  • Cross-check stationarity: ADF and PP share the unit-root null; KPSS flips it, use ADF and KPSS together.
  • GARCH models volatility: when big moves cluster, forecast the variance; high persistence (alpha + beta near 1) means shocks last.
  • Granger tests predictive precedence: does X's past improve Y's forecast? It is not proof of causation.
  • ARDL quantifies the link: regress a series on its own and another's lags; it also underpins cointegration testing.
7

Practice Challenges

Five exercises on the daily-returns series. Full solutions are in the companion solutions notebook.

1

Mean or variance?

Use the Ljung-Box test on the returns and on the squared returns. Where is the predictable structure?

Hint: acorr_ljungbox on each.
2

Levels vs returns

Build a price level (cumulative returns) and show it is non-stationary while the returns are.

Hint: run ADF and KPSS on both.
3

Forecast the volatility

Fit GARCH(1,1) and forecast the next five days' volatility. Does it mean-revert?

Hint: res.forecast(horizon=5).variance.
4

Granger both ways

Test causality in each direction at two lags. State the direction and the caveat.

Hint: grangercausalitytests with the columns swapped.
5

ARDL vs Granger

Fit an ARDL and check whether its coefficients agree with the Granger conclusion.

Hint: compare the market lag coefficients to the Granger p-values.
📓

Solutions notebook

All five challenges worked in code, Ljung-Box on returns versus squared returns, levels versus returns stationarity, a five-day GARCH volatility forecast, Granger causality both ways, and ARDL versus Granger, each with a short explanation.

📓 View Solutions ▶ Open in Colab ⬇ GitHub
8

Quiz: Test Yourself

Eight questions on correlograms, unit-root tests, GARCH, and Granger/ARDL. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved.

➡️
Up next

You have built many models; now learn to grade them fairly. Forecast Accuracy defines ME, MAE, MSE, RMSE, MAPE, and Theil's U, and shows how to compare forecasts, closing the loop before the forecasting case study.