Technical memo: Cost-optimised inspection policy for semiconductor wafer screening

Analysis of whether a machine learning classifier should replace or supplement end-of-line inspection

Recommendation: Do not deploy this model as an automated inspection gate. At current cost assumptions, inspecting 100% of output is approximately seven times cheaper than any model-based policy that respects operational constraints.

Model-guided triage becomes economically preferable only under one of two conditions:

• Inspecting a good part costs more than approximately £290 — around 6% of the cost of a defect reaching a customer
• Inspection capacity physically prevents checking every part

Where capacity is the binding constraint, the model has real value: it identifies 2.8 times more defects per inspection performed than random sampling of the same volume.

Investment should be directed at improving recall — better sensors or better features — not at model tuning, which cannot address the limitation identified here.

This analysis required two days and prevented a pilot that would not have paid back.

Background

End-of-line testing of semiconductor wafers is currently performed on 100% of output. The question examined here is whether a classifier trained on in-process sensor data could predict failures accurately enough to reduce that inspection burden, and if so, at what decision threshold.

Data. 1,567 wafers, each with 590 in-process sensor readings and a pass/fail outcome. 104 failures (6.6%). Source: UCI SECOM dataset.

Preparation. Sensors with more than 50% missing values were removed, as were zero-variance sensors (disconnected or non-varying instruments). Remaining gaps were filled with the column median. This reduced the feature space to 446 sensors.

Model. Random forest classifier, 200 trees, evaluated by stratified 5-fold cross-validation. All performance figures are out-of-fold — no wafer contributes to the model that scores it.

Method. Rather than accepting the default 0.5 decision threshold, expected cost per 1,000 wafers was computed across the full range of thresholds under a defined cost matrix, subject to operational constraints.

Assumptions

Cost matrix

On the £5,000 estimate: sensitivity analysis shows the recommended operating point is unchanged for any escape cost above approximately £2,500. Precise costing is therefore unnecessary; the conclusion holds across all plausible values. This matters — it removes a common reason for business cases to stall while someone attempts to price reputational damage.

Operational constraints

Two constraints were imposed, without which the optimisation returns a degenerate result.

Alarm rate ≤ 15%. Inspection capacity is finite. An unconstrained cost optimisation recommends flagging essentially every wafer — a policy that is not expensive but impossible.

Precision ≥ 25%. At the base failure rate of 6.6%, an unconstrained optimum produces roughly fourteen false alarms for every genuine defect. Operator attention does not survive that ratio. Alarm fatigue is well documented in industrial inspection and clinical alarm systems, and its effect is to reduce effective recall toward zero while measured recall remains unchanged. A policy that is optimal on paper and ignored on the line is worse than no system.

These constraints are not conservatism. They describe the operating environment.

Findings

1. Cost-optimal policy is 100% inspection

Inspection itself is highly valuable — it removes roughly £294,000 per 1,000 wafers of escape cost. The model captures only a small fraction of that value.

The driver is the cost ratio. At £5,000 per escape against £40 per inspection, one escape costs as much as 125 inspections. With 104 failures in 1,567 wafers, checking everything is straightforwardly cheaper than any partial-coverage policy this model can support.

2. The crossover is approximately £290 per inspection

Expressed as a ratio: model-guided inspection becomes preferable when inspection cost exceeds roughly 6% of escape cost.

3. The model's cost is dominated by missed defects, not by inspection

This is the most consequential finding and the least obvious.

As inspection cost rises fortyfold (£40 → £1,600), the best model policy cost rises only 20% (£272,856 → £327,128). The reason: at a 5% alarm rate the model inspects fewer than 80 wafers, so the per-inspection cost barely enters the total.

The model's cost floor is set by the 85 defects it misses. Recall is 0.18 at the feasible optimum and does not improve materially across the constraint space.

The practical implication for investment:

• Improving precision would reduce false alarms — which cost almost nothing in this configuration. Near-zero value.
• Improving recall is the only change that moves the total.
• Hyperparameter tuning addresses neither at the scale required. The limitation is in the signal available from the sensors, not in how the model is configured.

4. Under a capacity constraint, the model has genuine value

Reframing from "should the model replace inspection?" to "if capacity is limited, which wafers should be inspected?" changes the answer.

At a 5% inspection budget:

• Random selection finds 6.6% of defects — the base rate
• Model-guided selection finds 18% of defects

That is a 2.8× improvement in defects found per inspection performed, at identical inspection cost. This benefit is independent of the escape-cost assumption, which makes it the more robust part of the case.

Where investment should be directed

1. Improve recall, via better sensor coverage or physics-informed feature engineering. This is the binding constraint.
2. Establish the true cost of an escape only if it might fall below £2,500. Above that, the conclusion is stable.
3. Do not fund hyperparameter tuning as a route to viability. Tuning typically delivers a few percent; the gap here is far larger.
4. Re-examine the inspection method itself. The crossover analysis shows that if inspection were slower, destructive, or capacity-limited, the model would already be justified. The economics are a property of the inspection method as much as of the model.

Screening rule for future use-case selection

The following generalises from this analysis and can be applied to any proposed inspection AI project before development begins:

Inspection ML earns its place when inspection cost per part exceeds roughly 5–6% of escape cost, or when capacity prevents 100% coverage.Cheap, fast, non-destructive inspection → inspect everything; a model adds littleSlow, destructive, or capacity-limited inspection → a model may payRare defects with modest model lift → the arithmetic rarely works; verify before committing

A note on applying this to vision systems. Automated visual inspection at line speed is cheap per part — a camera and a few milliseconds of compute. By this rule, it falls in "inspect everything" territory, which means a business case for machine vision should not rest on displacing inspection labour. The defensible arguments are capacity (100% coverage is physically impossible manually at line speed), consistency (human inspection varies by inspector, shift and hour), coverage (destructive tests cannot be applied to every part), and feedback latency (detecting drift within minutes rather than at end of shift).

Limitations

The dataset is anonymised. SECOM sensors are numbered, not named. No physical interpretation of features is possible, so the domain-informed feature engineering recommended above could not be attempted here. On real process data this would be the first avenue to pursue and could change the recall conclusion.

Cost figures are illustrative. They are structurally realistic but not derived from this operation's accounts. The crossover ratio (~6%) is more transferable than the absolute values.

Imputation was performed outside the cross-validation pipeline. Median values were computed across the full dataset before splitting, which is mildly optimistic. The effect is expected to be small, but the analysis should be re-run with imputation inside the pipeline before any decision is acted upon.

No temporal split was used. The dataset carries timestamps, and semiconductor processes are subject to drift, tool maintenance cycles, and lot effects. Random cross-validation may therefore overstate performance on future wafers. A time-ordered split would give a more realistic estimate and should be the next analytical step.

No replicates exist, so measurement uncertainty in the pass/fail determination cannot be estimated. Label noise would place a ceiling on achievable recall that is currently unknown.

Single model family. Only a random forest was evaluated. Given that the limitation appears to be signal availability rather than model capacity, alternative algorithms are unlikely to change the conclusion — but this has not been demonstrated.

Appendix: reproducing this analysis

Threshold sweep:      0.001 to 0.99, 120 points
Cross-validation:     StratifiedKFold, 5 splits, shuffle=True, random_state=42
Model:                RandomForestClassifier(n_estimators=200, random_state=42)
Constraints:          alarm_rate <= 0.15, precision >= 0.25
Feasible optimum:     threshold 0.234, recall 0.18, precision 0.25,
                      alarm rate 4.85%, £272,856 per 1,000 wafers

Method note. Three separate optimisation runs during this analysis produced results pinned to a boundary of the search space — an artefact, not a result. In each case the cause was the same: the objective function was missing a constraint that exists in reality. An optimum sitting on a boundary should always be treated as a diagnostic that the problem has been under-specified, not as an answer.