Skip to content
Shyenova

Supply Chain & Analytics

Applied AI

The question is never whether the technique is advanced. It is whether the decision is currently being made badly, and whether you can prove the model makes it better.

Language models made a class of previously intractable problems tractable almost overnight, extracting structure from unstructured documents, classifying free text at scale, drafting where a human then edits. That is genuinely new and genuinely valuable, and it sits alongside a great deal of activity that is not.

The distinguishing test is unglamorous and it is the same one we apply to forecasting: can you demonstrate, on held-out cases, that the system beats what it replaces? Most AI projects that disappoint were never held to it. They were evaluated on demonstrations, which select for the cases that work.

How the work runs

Four phases, and what comes out of each.

Durations are indicative and assume reasonable data availability. Where data is poor we say so in the first fortnight rather than modelling on sand.
01

2 weeks

Select on value at stake, not on feasibility

We inventory candidate use cases and score them on the value of the decision or task, the volume, the cost of an error, and whether a materially simpler method already solves it. That last filter removes a surprising share. What survives is a short list where the technique is warranted rather than merely applicable.

Outputs

  • Use case inventory scored on value, volume and error cost
  • Cases eliminated because a simpler method already suffices
  • Shortlist with the success measure defined per case
02

2–3 weeks

Build the evaluation harness first

Before any system is built we construct the evaluation: representative cases with known correct outputs, including the hard and adversarial ones, plus the baseline that the incumbent process achieves. Building this first is what prevents a project from being judged on cherry-picked demonstrations, and it is the step most often skipped.

Outputs

  • Evaluation set covering ordinary, hard and adversarial cases
  • Baseline performance of the current process, measured
  • Acceptance threshold agreed before development starts
03

4–6 weeks

Build the simplest thing that clears the bar

Prompting and retrieval before fine-tuning, fine-tuning before anything bespoke, and a rules engine before any of it where rules suffice. Each increment is measured against the harness, and we stop at the first architecture that clears the threshold, because every additional layer is something your team maintains forever.

Outputs

  • Working system measured against the harness at each increment
  • Architecture decisions recorded with what each layer bought
  • Cost per task and latency measured under realistic load
04

3–4 weeks

Ship it with its failure modes visible

Deployment with confidence signalling, escalation to a human where confidence is low, monitoring for drift, and logging that makes wrong outputs diagnosable rather than merely regrettable. A system that fails invisibly erodes trust far faster than one that fails and says so.

Outputs

  • Deployment with confidence thresholds and human escalation
  • Drift monitoring against the evaluation set on a cadence
  • Logging and review process that makes failures diagnosable

What good looks like

The ranges we design toward.

These are the improvement ranges these levers are known to produce where the underlying practice has not been examined recently. They are targets to test against your baseline, not promises.

40–70%

Use cases eliminated

Share of a candidate list typically removed because a simpler method already solves it.

Before build

Evaluation built

The harness precedes the system. Judging on demonstrations selects for the cases that work.

Per task

Cost measured

Unit economics computed under realistic load, not from a pilot with a hundred requests.

Deliverables

What you are left holding.

All of it handed over, documented, and with a session for whoever inherits it.
  1. 01Use case shortlist scored on value at stake, with the rejected ones and why
  2. 02Evaluation harness including baseline performance of the current process
  3. 03Working system, built no more complex than the threshold required
  4. 04Measured cost per task and latency under realistic load
  5. 05Monitoring, escalation and a review process that surfaces failures rather than hiding them

What an engagement looks like

A representative shape.

Illustrative. This is the shape of work in supply chain & analytics, not an account of a specific client. We do not publish named results, and we would be wary of a firm that publishes them without the client in the room.

12 weeks

  1. 01Situation

    A consumer goods manufacturer was opening a third distribution centre on the strength of a regional sales forecast, with freight cost per unit rising and no cost-to-serve view below channel level.

  2. 02What we did

    Reconstructed eighteen months of shipment data into a true origin–destination model, built cost-to-serve to customer level, and tested the proposed footprint against demand growth, rate shock and node-loss scenarios.

  3. 03Where it landed

    The third site turned out to be robust only under the most optimistic growth case. Redrawing catchments and re-tendering two lanes captured most of the projected saving without the capital, and the site decision was deferred rather than cancelled.

Where this connects

Rarely a self-contained problem.

These are the adjacencies that come up most often on this kind of work. Naming them is usually cheaper than discovering them halfway through.

Questions

The ones we get asked.

Should we build or buy?

Buy for anything close to a commodity (transcription, translation, standard document extraction) because vendors will outrun an internal build and the switching cost is low. Build where the task depends on your own data, your own taxonomy or a workflow nobody else has. The expensive error is building the commodity and buying the differentiated thing.

How do we stop it making things up?

You constrain and you verify rather than hoping. Ground answers in retrieved source material, require citation to that source, constrain outputs to a schema where the task allows, and route low-confidence cases to a person. Then measure the residual rate on your evaluation set rather than asserting it is low. Anyone promising elimination is not describing the technology accurately.

Everyone is selling AI agents. Are they real?

For narrow, well-instrumented tasks with reliable tools and a bounded action space, increasingly yes. For open-ended multi-step work in a messy environment, reliability compounds badly, a chain of ten steps at ninety-five per cent each is a system that fails four times in ten. The useful question is how many steps must succeed in sequence and what happens when one does not.

Our data is not ready. Should we wait?

Not necessarily, and this is one place where the usual advice is too conservative. Language models are unusually tolerant of unstructured and inconsistent input, which is precisely what makes them useful on the messy corpora that defeated earlier methods. What they are not tolerant of is data that is wrong, because they will present it fluently. Correctness matters far more than tidiness here.

Start a conversation

Talk to us about applied ai.

Most useful first conversations are thirty minutes and involve no proposal. Describe the problem you are looking at and we will tell you honestly whether it is one we can help with.

Or write directly to hello@shyenova.com