Supply Chain & Analytics
Applied AI
Language models made a class of previously intractable problems tractable almost overnight, extracting structure from unstructured documents, classifying free text at scale, drafting where a human then edits. That is genuinely new and genuinely valuable, and it sits alongside a great deal of activity that is not.
The distinguishing test is unglamorous and it is the same one we apply to forecasting: can you demonstrate, on held-out cases, that the system beats what it replaces? Most AI projects that disappoint were never held to it. They were evaluated on demonstrations, which select for the cases that work.
How the work runs
Four phases, and what comes out of each.
2 weeks
Select on value at stake, not on feasibility
We inventory candidate use cases and score them on the value of the decision or task, the volume, the cost of an error, and whether a materially simpler method already solves it. That last filter removes a surprising share. What survives is a short list where the technique is warranted rather than merely applicable.
Outputs
- Use case inventory scored on value, volume and error cost
- Cases eliminated because a simpler method already suffices
- Shortlist with the success measure defined per case
2–3 weeks
Build the evaluation harness first
Before any system is built we construct the evaluation: representative cases with known correct outputs, including the hard and adversarial ones, plus the baseline that the incumbent process achieves. Building this first is what prevents a project from being judged on cherry-picked demonstrations, and it is the step most often skipped.
Outputs
- Evaluation set covering ordinary, hard and adversarial cases
- Baseline performance of the current process, measured
- Acceptance threshold agreed before development starts
4–6 weeks
Build the simplest thing that clears the bar
Prompting and retrieval before fine-tuning, fine-tuning before anything bespoke, and a rules engine before any of it where rules suffice. Each increment is measured against the harness, and we stop at the first architecture that clears the threshold, because every additional layer is something your team maintains forever.
Outputs
- Working system measured against the harness at each increment
- Architecture decisions recorded with what each layer bought
- Cost per task and latency measured under realistic load
3–4 weeks
Ship it with its failure modes visible
Deployment with confidence signalling, escalation to a human where confidence is low, monitoring for drift, and logging that makes wrong outputs diagnosable rather than merely regrettable. A system that fails invisibly erodes trust far faster than one that fails and says so.
Outputs
- Deployment with confidence thresholds and human escalation
- Drift monitoring against the evaluation set on a cadence
- Logging and review process that makes failures diagnosable
What good looks like
The ranges we design toward.
40–70%
Use cases eliminated
Share of a candidate list typically removed because a simpler method already solves it.
Before build
Evaluation built
The harness precedes the system. Judging on demonstrations selects for the cases that work.
Per task
Cost measured
Unit economics computed under realistic load, not from a pilot with a hundred requests.
Deliverables
What you are left holding.
- 01Use case shortlist scored on value at stake, with the rejected ones and why
- 02Evaluation harness including baseline performance of the current process
- 03Working system, built no more complex than the threshold required
- 04Measured cost per task and latency under realistic load
- 05Monitoring, escalation and a review process that surfaces failures rather than hiding them
What an engagement looks like
A representative shape.
Illustrative. This is the shape of work in supply chain & analytics, not an account of a specific client. We do not publish named results, and we would be wary of a firm that publishes them without the client in the room.
12 weeks
- 01Situation
A consumer goods manufacturer was opening a third distribution centre on the strength of a regional sales forecast, with freight cost per unit rising and no cost-to-serve view below channel level.
- 02What we did
Reconstructed eighteen months of shipment data into a true origin–destination model, built cost-to-serve to customer level, and tested the proposed footprint against demand growth, rate shock and node-loss scenarios.
- 03Where it landed
The third site turned out to be robust only under the most optimistic growth case. Redrawing catchments and re-tendering two lanes captured most of the projected saving without the capital, and the site decision was deferred rather than cancelled.
Where this connects
Rarely a self-contained problem.
Questions
The ones we get asked.
Should we build or buy?
Buy for anything close to a commodity (transcription, translation, standard document extraction) because vendors will outrun an internal build and the switching cost is low. Build where the task depends on your own data, your own taxonomy or a workflow nobody else has. The expensive error is building the commodity and buying the differentiated thing.
How do we stop it making things up?
You constrain and you verify rather than hoping. Ground answers in retrieved source material, require citation to that source, constrain outputs to a schema where the task allows, and route low-confidence cases to a person. Then measure the residual rate on your evaluation set rather than asserting it is low. Anyone promising elimination is not describing the technology accurately.
Everyone is selling AI agents. Are they real?
For narrow, well-instrumented tasks with reliable tools and a bounded action space, increasingly yes. For open-ended multi-step work in a messy environment, reliability compounds badly, a chain of ten steps at ninety-five per cent each is a system that fails four times in ten. The useful question is how many steps must succeed in sequence and what happens when one does not.
Our data is not ready. Should we wait?
Not necessarily, and this is one place where the usual advice is too conservative. Language models are unusually tolerant of unstructured and inconsistent input, which is precisely what makes them useful on the messy corpora that defeated earlier methods. What they are not tolerant of is data that is wrong, because they will present it fluently. Correctness matters far more than tidiness here.
Insights
Related reading.
The bullwhip effect: why your demand signal lies
Factory schedules swing by forty per cent while end-consumer demand moves by five. Nobody is behaving irrationally. That is precisely the problem.
Multi-echelon inventory: beyond the safety stock formula
The standard safety stock formula is correct and it is answering a question nobody in a multi-tier network is actually asking.
Cost-to-serve: the analysis most companies skip
Most businesses know their gross margin by customer. Rather fewer know which customers are profitable, and the two lists differ more than anyone expects.
S&OP versus IBP: what actually changes
Most descriptions of the difference are vendor taxonomy. The distinction that matters is whether a trade-off gets settled by someone with the authority to settle it.
Why people systematically under-order high-margin products
The mathematics of a single-period stocking decision is a hundred years old and unambiguous. Experienced buyers still get it wrong in the same direction, every time.
Closely related
More on Supply Chain & Analytics →Forecasting
Models measured against a naive benchmark and backtested honestly, including the finding that some of them should not be deployed.
Supply Chain & AnalyticsAnalytics Operating Model
Where analytics should sit, who owns models in production, and how to stop building things nobody uses.
Start a conversation
Talk to us about applied ai.
Most useful first conversations are thirty minutes and involve no proposal. Describe the problem you are looking at and we will tell you honestly whether it is one we can help with.
Or write directly to hello@shyenova.com
