From AI pilot to results in the machine shop

Why do AI pilots in manufacturing stall?

Many pilots start with the technology. A trial set-up appears with a handful of examples, the demo impresses and then little happens. The pilot had no owner in daily work, no fixed job and no number to be judged by.

The pattern is well documented. In 2021 Fraunhofer IPT named three obstacles to AI in manufacturing: relevant data are rarely available in sufficient quantity and quality, decision makers lack an understanding of the technology, and few applications are known to judge the benefit by.

In a 2026 survey by FME and partners of small and mid-sized manufacturers in North Brabant, 83 per cent lacked a clear view of which parts of their production process suit automation with AI. Among EU enterprises with 10 or more employees that considered AI but did not use it, lack of relevant expertise was the most common reason in 2025, cited by almost 71 per cent, according to Eurostat.

None of these reasons is about computing power. They are about a job nobody chose, data that is not ready and people left out.

Start with one job, not the technology

Choose work that comes round often, takes time today and gives a result a colleague can check: carrying over the drawing requirements for each part, building estimate lines for milling and turning, comparing a repeat part with its last estimate.

A good pilot job ends in a decision someone already makes: the cost engineer on price, the manufacturing engineer on route. Software that puts a proposal in front of them fits a moment that already exists. Software that needs a new moment, such as a separate dashboard beside the ERP, stays a detour.

The FME research points the same way: smaller manufacturers increasingly choose small, achievable steps over large-scale programmes. That is not a lack of ambition: one job that runs every day changes more than a programme that overhauls the whole office at once.

What data should a pilot run on?

A general AI assistant does not have your machines, your hourly rates or the purchase price of your last batch of 6082. A cost price without those is an average of someone else's shop. A platform that prices parts by what a buyer will probably pay makes your shop an interchangeable supplier.

The OECD wrote in 2025 that small firms, especially in manufacturing, struggle to find AI vendors tailored to their needs and lack good-quality data. That is a reason to start from the data every shop already has.

That data is smaller than it sounds: the STEP model and PDF drawing in the request, your machines with envelope and axes, your rate per machine and your own purchase prices. If you build your hourly rate per machine, as in how to calculate a machine hour rate, you hold the most important part. A large archive of comparable orders is not needed; in one-off and small-batch work the same part rarely comes back.

Who confirms what the software proposes?

In a pilot that pays off, the human-in-the-loop sits where it counts. Software puts forward a proposal: a material from the drawing, a tolerance, an estimate line. The cost engineer or manufacturing engineer confirms or corrects it. Confirm and correct is not a check afterwards; it is the work itself, with the retyping done.

Bring those people in from day one; the FME research names involving employees from the start as a key. A US NBER working paper (2018, revised 2020) gives the background: a general purpose technology such as AI needs significant complementary investment, including new processes and skills. That investment is often intangible and poorly measured in the national accounts, so productivity growth is underestimated in the early years.

Every correction is also information. When the cost engineer changes a proposal and says why, shop knowledge leaves one head and is written down. A correction history with who, when and why makes it readable for a colleague; how to keep a cost engineer's know-how shows how to set that up.

How do you measure before and after?

Without a baseline, nobody can be proved right. The NIST AI risk management framework (2023) says that managing the risk of AI that augments or replaces human work requires some form of baseline metric for comparison, and that a system should be tested before deployment and regularly while in operation.

Speed alone is not a result: a faster quote with the wrong cost price is a faster mistake. Estimate versus actuals per operation shows how to compare estimate and actual.

1. Pick the number that belongs to the job: quote turnaround time, the minutes to carry over one drawing's requirements, or the number of questions sent back to the customer.
2. Measure a fixed run of real requests the way you work now.
3. Measure the same kind of requests the same way with the software, by the same people.
4. Count the corrections: proposals confirmed, proposals changed, and why.
5. Later, set the estimate beside the post-calculation per operation: a faster cost price only counts if it also holds.

When is a pilot finished?

A pilot is finished when the job runs in normal work, on every request it belongs to, or when the measurement shows it delivers nothing and you stop on purpose. A pilot still called a pilot months later is a decision nobody is taking.

This is where the roads part. Software suites are built for the engineering department of a large manufacturer and need a project before anything works. An ERP stores what happened, not why a price was set. A machine shop needs its own knowledge in one model, at the place where the price is made every day.

That is what we are building on. The next step is calibration: a cost price checked against actual hours, with times from CAM and NC beside the estimate. After that, the cost price will reach the designer while the part is still being drawn.