Hierarchical Forecasting: Store, Region, National

Twenty-one series, one arithmetic constraint, and four ways of satisfying it. The disagreement between the independent forecasts turns out to be under one percent, which is exactly what makes this a hard problem to take seriously and an easy one to get wrong.

Each team forecasts what it is responsible for. Holt-Winters on the chain series for the board, on each region for the regional managers, on each store for the store managers. Six years of history, twelve months ahead, twenty-one separate fits. Then somebody adds them up.

Ten thousand nine hundred and twenty-four units separate the highest from the lowest. On a chain doing 1.3 million a year that is under one percent, and there is no error anywhere to find. Each of the three numbers is a competent forecast of the series it was fitted to. They disagree because nothing in the procedure ever asked them to agree, and three separate fits to three different series have no reason to.

Sixteen stores, four regions, eighty-four months from January 2019 to December 2025. The export arrived with August 2022 run twice, one store's region spelled three ways, a stocktake booked as minus 412 units and two months where the till export wrote a zero. Those last three are repaired by interpolating within each store's own series, because a till failure read as a month of no sales drags that store's trend down for the rest of the history.

Two stores are worth naming now, because they decide the outcome. WE2 grows at 18.3 percent a year while the rest of the West edges down at 0.8. SO3 falls at 13.3 percent while the rest of the South grows at 6.1. Neither is anomalous as a business. Both are impossible to express as a fixed share of a national number.

Forecast the sixteen stores and add them up. Coherent by construction, since every level is now a sum of the same sixteen numbers, and it keeps each store's own history including the two that are going their own way. The chain fit is simply discarded.

The objection to bottom-up is that the chain forecast becomes an accumulation of sixteen individual errors. Whether that matters depends entirely on whether those errors cancel, which is the question the bottom-right panel above was answering. Here they largely do, and that is not a general fact about retail chains. It is a fact about this one, and it is measurable.

Forecast the chain, which is the smoothest series in the file, then split it down by each store's historical share. Also coherent, and it is the method that fails hardest here. The failure is not a matter of degree.

A share is a constant and these two stores are not. WE2 has climbed from a tenth of the chain to a sixth while SO3 has fallen away, and a six-year average splits the difference on both. The plan cuts a growing store by more than a third and hands a shrinking one an increase of nearly eighty percent. Neither number could survive a conversation with the store manager.

The other fourteen stores are well enough behaved that the method still looks defensible in aggregate, which is exactly how it survives in practice. Nobody checks the two stores that the average is wrong about, because the chain total it produces is the best-forecast series in the building.

Both methods so far throw information away. Bottom-up ignores the chain fit; top-down ignores the store fits. Reconciliation keeps all twenty-one forecasts and looks for the coherent set closest to them, which is a projection and therefore a single line of linear algebra once the structure is written as a matrix.

The last column is the honest summary of what reconciliation does. It is a nudge, not a rebuild. MinT moves the original forecasts least of the four, because it is looking for the nearest coherent set rather than imposing a structure on the answer. Top-down moves them by more than twelve times as much, which is another way of stating what the previous section showed.

The difference between OLS and MinT is the covariance. OLS treats a store doing 2,700 units a month and the chain doing 108,000 as equally informative. MinT uses the in-sample errors to work out which forecasts have been reliable and which errors tend to move together, and weights accordingly. That is the only thing separating them, and it shows up in every row of the backtest.

Five origins, twelve months ahead from each, every method refitted at every origin. Scored with MASE, which divides by the seasonal naive error on that series' own training data, so a store doing four thousand units a month and a chain doing a hundred thousand can appear in the same table. Below 1 beats a seasonal naive rule.

Read the top-down column first. It is the only one that ever loses to a seasonal naive rule, and at store level it is half again as bad as everything else. For an individual store you would have done better assuming next December equals last December than using a plan built by splitting the national number down.

Then read across the other four. They are close. Bottom-up is the best coherent choice at every level, MinT is a step behind it, and OLS trails MinT in every row, which is the argument for using the error covariance rather than assuming it away.

The worst five series under top-down are WE2 at 4.3 times the bottom-up error, SO3 at 3.8, SO4 at 2.3, and then the West region and WE1. Every one of them is a place where a share has been drifting. This is what it looks like when a diagnostic run before any modeling tells you exactly where the model will fail.

Bottom-up won, and that could not have been known in advance. It won because these sixteen stores are individually forecastable and their errors largely cancel, which is the fact the first look measured and the backtest confirmed. On a chain of two hundred small stores, or on weekly data where each store is far noisier, the same procedure routinely goes the other way and the aggregate forecast carries the day.

That is the case for reconciliation and it is a modest one. MinT was never the best method at any level here and never more than 0.055 MASE behind the one that was, while requiring no judgment about which level to trust. It is insurance rather than an improvement, and it is cheap. What the backtest does establish beyond argument is which method not to use.

Coherence is not accuracy. Every method after the first produces numbers that add up, including the worst one in the table. Adding up is a property of the output and says nothing about whether the output is any good.

The winner is a property of this chain. Sixteen stores, monthly, with errors that mostly cancel. Change any of those and the ranking can invert, which is why the backtest is the deliverable and not the model.

One structure, one moment. The hierarchy here is geographic and fixed. Real chains open and close stores, move them between regions, and sell through channels that cut across geography, so a store can belong to two groupings at once. Reconciling across several groupings at the same time is a harder problem than the one worked here.

Nothing above produced an interval. Every number in this chapter is a point, and a plan is built on a range. The interval around a reconciled forecast is not the interval around the base forecasts, because the projection mixes them, and its coverage has to be measured rather than assumed. That is the subject of the last capstone in this part.

• ✓Add the levels up before anyone presents them. Independent forecasts have no reason to agree, and the gap is usually small enough to go unnoticed until three teams are holding three different targets.
• ✓Plot each unit's share of its parent over time before anyone proposes top-down. A share that is trending cannot be averaged, and the stores where it is trending are exactly the ones a planner will be asked about.
• ✓Test whether bottom-up will work before you rely on it. Difference out the season, compare the aggregate's variation against what independent series would give, and see how close they are.
• ✓Score by level, not overall. A method can win at the top and lose at the bottom, and only one of those is the number a store manager is held to.
• ✓If you reconcile at all, use the error covariance. MinT and OLS are the same projection with one difference, and it decided every row of the backtest.
• ✓Re-run the comparison when the portfolio changes. The ranking rests on how far store errors cancel, and that moves when stores open, close or change size.

Almost every forecast that reaches a decision is part of a hierarchy. Demand by product rolls into category and into total; capacity by machine rolls into line and plant; traffic by page rolls into section and site. The constraint is arithmetic and unavoidable, and most production forecasting stacks handle it badly or not at all.