Regression algorithms are tools used to predict continuous numerical values, like as prices, temperatures, or sales numbers, by learning the mathematical relationship between features and targets.
The purpose of a first ML model is not to prove that ML works. It is to determine whether the available data contain enough reliable predictive information to justify going further. Once we have a disciplined baseline and validation process, we could safely ask whether more sophisticated models or automated tuning actually added value, and sometimes we do not.
These some key principles to build our first ML model.
1. Start every ML project with a clear prediction question. Before choosing an algorithm, define what you want to predict, which inputs will be available, and what engineering decision the prediction will support.
2. Separate features from the target. X contains the information available to the model; y is the quantity you want to predict. For a regression exercise, formulation/process variables may be the features and mechanical strength the target.
3. Training and prediction are different steps. .fit() learns relationships from known examples; .predict() applies those learned relationships to new observations.
4. Never judge a model only on the data it trained on. A model must be evaluated on unseen data. Otherwise, you measure memorisation rather than generalisation.
5. Always establish a simple baseline. Our baseline predicted the training-set mean. If a linear regression does not beat it, it immediately tell us that the model has not demonstrated useful predictive value.
6. A disappointing model result is still useful evidence. If a model performs worse than the baseline, that can indicate weak predictive signal, missing variables, noise, too little data, or inappropriate model assumptions. It is not a failed experiment.
7. MAE should be interpreted in engineering terms, not just statistically. An MAE of 8 kPa is only meaningful once you ask whether an 8 kPa prediction error is acceptable relative to test variability, specifications, margins, and decision cost.
8. Regression coefficients describe the fitted model, not automatically the physical system. A large positive or negative coefficient does not prove that changing that variable will cause the target to change by the same amount.
9. Prediction and correlation do not establish causation. A model may identify useful associations without revealing the underlying materials mechanism. Causal conclusions require stronger experimental evidence.
10. More features do not necessarily mean more information. Component A%, Component B%, and binder % may sum to 100%, so one could be reconstructed from the other two. Including mathematically redundant variables can complicate interpretation without improving prediction.
11. Multicollinearity can destabilise interpretation even when predictions barely change. Dropping a redundant composition variable will not improve MAE, but it made the feature structure cleaner. Good modelling is not only about scores; it is also about representing the problem sensibly.
12. Validation strategy must match the real engineering use case. If the goal is to predict a completely new formulation, the same formulation should not appear in both training and testing. If the goal is to predict another specimen of a known formulation, a different split might be appropriate.
13. Data leakage can create excellent-looking but meaningless models. Duplicates, related specimens, future information, or grouped observations crossing train/test boundaries can make evaluation artificially optimistic.
14. Group-aware splitting should be enforced when grouping matters. If formulation separation is required, the code should guarantee it.
15. A random seed controls reproducibility, not model quality. random_state=42 simply lets us reproduce the same split. The number 42 has no special statistical value.
16. Never choose the seed that gives the best result. Testing many seeds and reporting only the most favourable one is effectively choosing the easiest test set. That produces biased performance claims.
17. Small datasets make performance estimates unstable. With only a few test specimens, moving one or two observations between training and testing can change MAE substantially.
18. Reproducibility and robustness are different. A fixed seed gives the same answer repeatedly. Robustness means the conclusion remains similar across several valid splits. A perfectly reproducible result can still be unreliable.
19. One train/test split is rarely enough for a small manufacturing dataset. Repeated splits or cross-validation give a better picture of average performance and variability.
20. Outliers should be investigated, not automatically deleted. A 980 kPa observation should be treated through sensitivity analysis. We ask how the model behaves with and without it rather than removing it simply because it was inconvenient.
21. Inspect individual predictions, not just headline metrics. MAE can hide whether the model is consistently slightly wrong or catastrophically wrong on one or two specimens. Actual-vs-predicted tables help reveal failure modes.
22. ML cannot manufacture information that is absent from the dataset. If the chosen features contain little signal about the target, increasing algorithm complexity will not create a meaningful relationship.
23. Treat model development like an engineering experiment. Form a hypothesis, test one change, inspect the outcome, and update your understanding. Avoid randomly changing parameters until a favourable score appears.
24. The test set is itself part of the experimental design. Ask whether it represents the kinds of formulations, batches, feedstocks, process conditions, and operating ranges the model would encounter in reality.
And perhaps the most important principle:
The purpose of model development is not to find the algorithm with the lowest number. It is to establish whether the available data contain enough reliable information to support the engineering decision.