Skip to content
Shyenova

Supply Chain & Analytics

Forecasting

If a model cannot beat last week's actual, it should not go to production. A surprising number cannot.

Forecasting attracts more sophistication than almost any other analytical problem and rewards it less reliably. Teams reach for gradient boosting and neural architectures on series where a seasonal naive method is within a couple of points, and the extra accuracy is consumed entirely by the maintenance burden of a model nobody fully understands.

The discipline that separates useful forecasting from expensive forecasting is unglamorous: benchmark against naive methods, backtest with proper temporal separation, measure accuracy at the level the decision is actually made, and be willing to conclude that the simple method wins. That conclusion is a good outcome and it is reported far less often than it occurs.

How the work runs

Four phases, and what comes out of each.

Durations are indicative and assume reasonable data availability. Where data is poor we say so in the first fortnight rather than modelling on sand.
01

1–2 weeks

Establish the benchmark first

Before building anything we establish what naive methods achieve (last period, seasonal naive, moving average) at the level the decision is made. This is the bar. Roughly a third of forecasting engagements find the incumbent sophisticated model is not clearing it, which is worth knowing in the first fortnight rather than after a build.

Outputs

  • Naive benchmark accuracy at the decision's level of granularity
  • Current model performance measured against that benchmark
  • Accuracy metric chosen to match the decision's cost asymmetry
02

2–3 weeks

Segment by what is actually forecastable

A portfolio contains series with entirely different characteristics (smooth and seasonal, intermittent, lumpy, new with no history) and one method applied across all of them is wrong for most. Intermittent demand in particular breaks standard methods badly and needs approaches built for it. We segment on demand pattern and set the achievable accuracy floor for each.

Outputs

  • Segmentation by demand pattern with method assigned per segment
  • Achievable accuracy floor per segment, the irreducible noise
  • Series flagged as genuinely unforecastable, with the implication stated
03

3–5 weeks

Build and backtest without fooling yourself

Models are built per segment and evaluated on rolling-origin backtests that respect time order, with no information available at evaluation that would not have been available at the moment of forecasting. Leakage is the single most common reason a model that looked strong in development disappoints in production, and it is usually subtle.

Outputs

  • Segment-appropriate models with rolling-origin backtests
  • Explicit leakage audit of every feature
  • Honest comparison against the naive benchmark, per segment
04

3–4 weeks

Deploy only what earns its place

Segments where the model beats the benchmark materially go to production, with monitoring for drift. Segments where it does not keep the simple method, and we say so plainly. We also measure forecast value-add on human overrides, which frequently shows that a meaningful share of manual adjustment makes the forecast worse and can simply be stopped.

Outputs

  • Production deployment limited to segments where it is justified
  • Drift monitoring with defined retraining triggers
  • Forecast value-add measurement on human overrides

What good looks like

The ranges we design toward.

These are the improvement ranges these levers are known to produce where the underlying practice has not been examined recently. They are targets to test against your baseline, not promises.

5–15 pts

Accuracy improvement

Typical MAPE gain where no segmented statistical baseline previously existed.

~⅓

Engagements where simple wins

Share in which a naive or classical method matches the sophisticated incumbent. Reported rather than buried.

20–40%

Harmful overrides

Share of manual adjustments that commonly degrade the statistical forecast, and can simply be stopped.

Deliverables

What you are left holding.

All of it handed over, documented, and with a session for whoever inherits it.
  1. 01Naive benchmark and current-model performance at the level decisions are made
  2. 02Segmentation with an appropriate method assigned to each demand pattern
  3. 03Rolling-origin backtests and a documented leakage audit
  4. 04Production models only where they beat the benchmark, with drift monitoring
  5. 05Forecast value-add measurement so override discipline becomes evidence-based

What an engagement looks like

A representative shape.

Illustrative. This is the shape of work in supply chain & analytics, not an account of a specific client. We do not publish named results, and we would be wary of a firm that publishes them without the client in the room.

12 weeks

  1. 01Situation

    A consumer goods manufacturer was opening a third distribution centre on the strength of a regional sales forecast, with freight cost per unit rising and no cost-to-serve view below channel level.

  2. 02What we did

    Reconstructed eighteen months of shipment data into a true origin–destination model, built cost-to-serve to customer level, and tested the proposed footprint against demand growth, rate shock and node-loss scenarios.

  3. 03Where it landed

    The third site turned out to be robust only under the most optimistic growth case. Redrawing catchments and re-tendering two lanes captured most of the projected saving without the capital, and the site decision was deferred rather than cancelled.

Where this connects

Rarely a self-contained problem.

These are the adjacencies that come up most often on this kind of work. Naming them is usually cheaper than discovering them halfway through.

Questions

The ones we get asked.

Will machine learning beat our current statistical forecast?

Sometimes, in specific places. Machine learning genuinely outperforms classical methods where there is rich related history, promotional and price response to learn, or many series that can share strength. It underperforms on sparse data, short history, structural breaks and anything requiring causal reasoning about a situation with no precedent. The honest answer requires a backtest, and we would rather run one for a fortnight than assert either way.

Our demand is genuinely volatile. Is forecasting even worth it?

Partly, and this deserves a straight answer. Irreducible volatility is real and no method removes it. But it is routinely used to excuse avoidable error, wrong aggregation level, ignored promotions, unmodelled cannibalisation, methods unsuited to intermittent demand. We measure the achievable floor first. If your accuracy is already near it, the answer is not better forecasting but an operation designed to need less of it, which is a supply chain engagement rather than an analytics one.

What accuracy should we be targeting?

There is no universal number and any consultant quoting one without seeing your data should be treated with suspicion. Achievable accuracy is set by the underlying variability of your series. The right target is the measured floor for your portfolio plus a realistic improvement, which we establish in the first fortnight, not a benchmark borrowed from a different business.

Can you forecast for a product with no sales history?

Only by analogy, and the uncertainty is large. New product forecasting works by finding structurally similar historical launches and borrowing their shape, which is legitimate but should be presented as a range with explicit assumptions rather than a point estimate. The bigger win is usually designing the launch so it can respond quickly to early signal rather than committing hard to a forecast nobody can make well.

Start a conversation

Talk to us about forecasting.

Most useful first conversations are thirty minutes and involve no proposal. Describe the problem you are looking at and we will tell you honestly whether it is one we can help with.

Or write directly to hello@shyenova.com