Objective
Determine whether model-guided experimental design reduces the number of trials required to reach a performance target, using a dataset with enough signal and samples to verify the pipeline behaves correctly.
Dataset
UCI Concrete Compressive Strength. 1,030 mix designs with measured compressive strength. Inputs: cement, slag, fly ash, water, superplasticiser, aggregates, age.
Structurally equivalent to a formulation-to-property problem: continuous composition variables plus a process variable, one scalar target.
Method
Surrogate. Gaussian process, Matérn (ν = 2.5) with per-dimension length scales, constant amplitude, white-noise term. Inputs and target standardised.
Small-data simulation. Fitted to a 40-sample subset, approximating a real development programme, with the full 1,030 retained for verification.
Validation. Repeated 5-fold cross-validation, 6 repeats, against a predict-the-mean baseline.
Campaign simulation. The full dataset treated as a simulated laboratory — each row a formulation that could be run, result hidden until selected. Random search compared against Expected Improvement, 15 seeds each, 8 initial samples, target 78 MPa (1.0% of candidates).
Results
Feature selection
| Features | MAE (MPa) | Fold σ |
|---|---|---|
| 6 | 6.28 | 2.01 |
| 4 | 6.32 | 1.98 |
| 3 | 5.92 | 1.46 |
| 2 | 5.83 | 1.32 |
Six features produced pinned length scales; the GP could not identify all dimensions at n = 40. The two surviving features — water-to-binder ratio and age — are those identified by Abrams' law and maturity theory.
Encoding
| Encoding | MAE (MPa) |
|---|---|
| Raw water mass | 7.07 ± 1.96 |
| Water/binder ratio | 6.28 ± 2.01 |
Raw water produced a length scale of 27,660 (pinned). As a ratio: 2.72, second most important variable.
Final model
1.47² × Matérn(length_scale=[1.75, 1.08], ν=2.5) + WhiteKernel(0.119)
| Metric | Value |
|---|---|
| Cross-validated MAE | 5.83 MPa |
| Baseline | 12.28 MPa |
| Improvement | 53% |
| Inferred noise | 5.22 MPa |
Model error sits 0.6 MPa above the inferred noise floor — little headroom remains for algorithmic improvement.
Optimisation
| Strategy | Median trials | Range | Failures |
|---|---|---|---|
| Random | 76 | 8–201 | 2 of 15 |
| Bayesian opt | 15 | 8–30 | 0 of 15 |
5× reduction at the median; worst case reduced from 201 to 30.
Constraints
Unconstrained optimum: w/b 0.235 (dataset minimum, below placeability) at 251 days. Constrained to w/b ≥ 0.32 and age ≤ 56 days: 61.3 MPa against 81.0 unconstrained — constraints cost 24% of achievable performance.
What went wrong along the way
Four corrections were required before the analysis was sound. Each is a reusable diagnostic.
| Symptom | Cause | Fix |
|---|---|---|
| Water length scale 27,660, pinned at bound | Variable acts through a ratio, not alone | Encode as water/binder |
| Slag pinned after adding w/b | Too many dimensions for n = 40 | Reduce to two features |
Both medians exactly 15 (= n_init) |
Target too easy; hit by the starting draw | Harder target, fewer initial samples |
| Ratio model apparently worse than raw | Single-split comparison | Repeated cross-validation |
Transferable conclusions
- Bayesian optimisation delivers a substantial reduction in trials to target where signal exists — indicatively 3–5×, with the elimination of the long tail an additional and separately valuable benefit.
- The benefit depends on the surrogate being able to learn the response. Where measurement noise dominates, the optimiser will propose candidates derived from noise with equal confidence.
- Constraints must be encoded explicitly. An unconstrained optimiser will propose the physically impossible, and will do so without any indication that it has.
- Physics-informed encoding of variables is not optional at small sample sizes. It determines what the model can learn at all.
Caveat on transfer
The simulation searches a fixed list of 1,030 known formulations. A real campaign searches a continuous space where the target may not exist, and each trial takes weeks rather than a function call.
The 5× figure is indicative of the mechanism, not a guaranteed return. What transfers reliably is the direction and the reason: using calibrated uncertainty to guide a search outperforms searching blind.
Figures generated this week, where each belongs, and what it demonstrates