Working with manufacturing data before you model it

The dataset is the deliverable that determines everything downstream. No model recovers from bad data, and almost every serious industrial ML failure is a data failure that was discovered too late. A model can learn a bad measurement just as easily as a good one. To evaluate how ready for modelling a dataset is, use this ML data scorecard.

The dataset must therefore be treated as an engineering asset requiring validation, traceability, documentation and judgement; not as a file to be tidied.

The goal of data cleaning is not to make the dataset look neat. It is to make sure the data faithfully represent what was actually manufactured, measured or observed.

The working rule is to interrogate the dataset before you model it, and write down what you found.

Part I - Frame the dataset before you open it

1. Define the engineering problem

What decision will the model support? Everything below, which rows count, which columns are permissible inputs, what "good enough" means, follows from the answer.

2. Define what one row represents

Identify the unit of observation before analysing anything:

• One specimen
• One panel
• One formulation
• One batch
• One machine cycle
• One image
• One hour of equipment operation

If two rows share the same formulation but represent two independently manufactured specimens, they are replicates, not duplicates. The unit of observation also determines how the data may later be divided into training and test sets.

3. Preserve traceability

Identifiers such as specimen_id, formulation_id, batch_id, raw-material lot, test date, operator and equipment may never be used as model input features, but they should normally remain in the master dataset. They support the chain:

Prediction → specimen → batch → formulation → material lot → test record

Traceability is what lets you identify data leakage, investigate unusual results, reproduce experiments, and understand changes in raw materials or manufacturing conditions.

4. Separate raw data from cleaned data

Never overwrite the original dataset during cleaning. Maintain four distinct assets:

A specimen with no mechanical-strength measurement may legitimately remain in the clean master dataset, yet cannot serve as a labelled example when training a strength model.

Part II - Inspect

5. The inspection ritual

Five commands, run on every new dataset, before anything else:

df.shape        # scale
df.head()       # shape of a record
df.info()       # types and completeness
df.describe()   # ranges and distribution
df.columns      # exact names

Then read them properly rather than glancing at them:

• From .info(), are types correct? A measurement column arriving as object means text contaminated it. This is the most common reason a model refuses to train. Also check row count, column count, unique IDs and missing values here.
• From .describe(), do the min and max make physical sense? Negative mass, zero strength, an age of 10,000 days. Sensor faults and data-entry errors announce themselves here in thirty seconds.
• Identify zero-variance columns while you are looking.

6. Plot everything - visualisation is a data-quality tool, not a presentation tool

Summary statistics can be identical across radically different distributions. Only a plot shows shape, clustering, gaps and boundaries. A histogram of every variable belongs in the ritual.

Useful early plots: histograms, scatter plots, box plots, time-series plots, batch-by-batch comparisons. They expose extreme observations, impossible ranges, strange clusters, unit problems, unexpected gaps, changes between batches and probable recording errors.

At this stage the question is:

"Does anything look inconsistent with how these data should have been generated?"

not yet "what scientific relationships are present?"

Part III - Estabilish provenance

7. Clean data is a red flag

Real experimental and production data have gaps: samples that failed before testing, instruments that were down, product a technician judged not worth testing.

Zero missing values in a few hundred records of real work is not a sign of quality. It is a sign that something happened to the data before it reached you. Three possibilities, in ascending order of danger:

1. Someone cleaned it, and their decisions are now invisible
2. Failures were dropped from the export rather than recorded as failures
3. It was generated

Number 2 is survivorship bias, and it is silent. If failed formulations were excluded rather than recorded, the model learns only from the region of the design space where samples survived to be tested, and will then confidently recommend formulations that fall apart in the mould, because it has never seen one do so.

Always ask what happened to the failures.

8. Detecting synthetic or generated data

A dataset can be described as production output and not be. Four independent tests:

The uniformity test

For a uniform distribution, std ≈ range / √12 (√12 ≈ 3.464), and the mean sits at the exact midpoint. If several independent variables all satisfy this to within ~1%, they were sampled, not measured. Physical processes do not produce flat distributions across every variable simultaneously.

The evenly-spaced quartile test

Quartiles at exact even intervals (4.885 / 5.250 / 5.625) indicate a generated design. Real data clusters.

The bin-count test: counter-intuitive and important

Real randomness is lumpier than people expect. With 200 points in 25 bins, the average count is 8, but the standard deviation of bin counts is ~2.8. Genuine random sampling typically produces at least one bin near 2–3 and one near 13–14.

If every bin falls between 5 and 9, too regular, that is the signature of stratified or Latin hypercube sampling, not randomness.

Most people check whether data is too messy. Fewer check whether it is too clean. Both are informative.

The internal-contradiction test

Cross-check the data against what you have been told about it. If the commercial gate is 1400 kPa and half the records fall below it, the data is not passing production output, regardless of what the label says.

Practical use: these tests are how you audit a dataset handed to you by a vendor, a partner or a consultant, when the claim about its provenance is the thing you cannot independently verify.

Part IV - Judge individual values

9. Missing values are not the same as zero

extraction_temperature = 0 means the value was measured or recorded as 0 °C. extraction_temperature = missing means we do not have a trustworthy value.

Never replace missing engineering data with zero to make a dataset complete. Automatic mean-imputation is equally dangerous: it hides information and manufactures artificial observations.

Treatment should depend on:

• Why the information is missing
• Whether it can be recovered from source records
• Whether the variable is required for the modelling task
• Whether an appropriate imputation method exists

Whatever the choice, quantify the missingness first.

10. Distinguish an error from an outlier

An error does not correctly represent what actually happened or was measured: negative thickness, an extrusion temperature entered as 1800 °C instead of 180 °C, a wrong unit, a transcription slip. An error need not look extreme: 128 entered instead of 118 is wrong while appearing entirely plausible.

An outlier is merely unusual relative to the rest of the dataset. It may be a real and important observation, a rare formulation response, a new material behaviour, a process excursion, a measurement problem, or a data-entry error.

An unusual value should be investigated, not automatically deleted.

A value of 980 PSI may be suspicious relative to its neighbours, but if it is physically possible, there must be evidence before declaring it erroneous.

11. Use engineering knowledge to check ranges

Statistical tools alone cannot determine whether manufacturing data are valid. Range checks should ask whether values are:

• Physically possible
• Within the equipment operating range
• Within the experimental design
• Consistent with the test method
• Consistent with known units
• Plausible for the material system

For example: negative thickness is physically impossible; 800 °C extraction is incompatible with the process; a zero water-to-solids ratio is inconsistent with a wet extraction process; very high strength is unusual but potentially real.

Data preparation therefore requires both data skills and domain expertise.

12. Standardise categorical data

Computers interpret text literally. Priya, priya and PRIYA read as one category to a person and three values to a pipeline. The same problem occurs with material names, suppliers, batch numbers, equipment IDs, test methods, product types and defect labels. Establish naming conventions before modelling.

13. Duplicates, replicates, and leakage

An exact duplicated record gives one observation more statistical weight than it deserves. Far more serious: duplicates, or near-identical samples, appearing in both training and test sets. The model then appears to perform extremely well because it has effectively already seen the test observation. This is data leakage:

Information reaches the model during training that would not genuinely be available when predicting future unseen cases.

In manufacturing, leakage also occurs whenever specimens from the same batch or formulation are randomly divided between training and test sets.

Replicates are not duplicates. Two independently manufactured specimens of the same formulation share formulation inputs but have different specimen IDs, slightly different densities and different measured strengths. They capture real manufacturing and testing variability and should normally be retained as separate observations, while being kept together on the same side of any train/test split.

14. Never silently repair or "correct" data

Suppose density is recorded as 0.176 kg/m³. It looks obvious that 0.176 g/cm³ = 176 kg/m³ was intended. Changing it to 176 without checking the source record is still an assumption.

The sequence is:

Detect → Investigate → Verify → Decide → Document

Where records violate a physical constraint: fractions summing above 100%, negative concentrations, the same discipline applies as three concrete steps:

1. Count the affected rows
2. Flag them with a boolean column that survives into every downstream analysis
3. Decide and document: correct, exclude, or retain

If the original measurement cannot be verified, leaving the value missing or excluding it from the relevant analysis is usually safer than inventing a correction.

Silent correction destroys the audit trail. Six months later nobody can tell which rows were altered or why, and the correction itself becomes an unexamined assumption embedded in every result. A violation too large to be rounding error is a data integrity finding, not a formatting inconvenience.

15. Keep a data-issue log

No manufacturing data should change without an audit trail. A useful log records, per issue:

• Specimen ID
• Variable
• Original value
• Action taken
• Reason
• Whether the source was verified
• Reviewer

Typical actions: correct · recover from source · set to missing · exclude from a particular analysis · keep and flag.

Part V - Structural traps

16. Design variables, intermediates, and outcomes

Not every column is an input. Sort them into three tiers before modelling:

The test that matters: would this value exist at the moment I need the prediction?

Using a measured intermediate to predict a final property produces an excellent-looking model that is useless for design, because you would have to make the sample to obtain the input. This is not textbook leakage; the intermediate is not caused by the target, but it fails the same practical test.

The fix is usually a two-stage chain, not a discarded column: predict the intermediate from the design variables, then the property from the intermediate. This mirrors the physics and is frequently both more accurate and more interpretable than modelling directly.

Optimisation can only act on design variables. An optimiser given a measured intermediate will return recommendations you cannot execute.

17. Compositional data has a hidden trap

If ingredient fractions sum to a constant, the variables are perfectly collinear: knowing all but one determines the last. Consequences:

• Regression coefficients become unstable and individually uninterpretable
• "The effect of increasing X" is undefined, because increasing X necessarily decreases something else, and what it decreases changes the answer
• SHAP values inherit the same ambiguity
• An unconstrained optimiser will propose formulations summing to 1.4

The trap within the trap: closure can be invisible. If a component is missing from the table, a filler, a balance, a carrier, the remaining fractions will not sum to a constant, and the problem looks absent. It is not. Compute the balance explicitly and add it as a column:

df["balance"] = 1 - df[component_columns].sum(axis=1)

An implicit component that varies substantially is often one of the strongest drivers in the system, and leaving it unnamed forces the model to infer it while making every coefficient harder to interpret.

18. Separate setpoint from measured actual

Where equipment overshoots or drifts, a recorded process value contains two distinct pieces of information:

• The setpoint: what was intended. This is the design variable, and the only thing optimisation can act on.
• The measured actual: what happened. This is a measured outcome, and the deviation may itself be predictive.

Keep both as separate columns. Rounding the actual to the nearest setpoint destroys the deviation permanently, and deviation from setpoint is frequently a signal worth having.

df["temp_setpoint"] = (df["temp_measured"] / 5).round() * 5

19. Merges lose rows silently

Joining process data to quality data is one of the most common operations in manufacturing analytics, and one of the most common sources of quiet, serious error. Unmatched keys are dropped without warning, leaving a biased subset that looks like a complete dataset.

Rule: record the row count before and after every merge.

result = left.merge(right, on="key", how="left", indicator=True)
print(result["_merge"].value_counts())

how="left" preserves every row from the left frame; indicator=True reports where each row came from. Missing matches become visible instead of silent.

20. Always report the group size

df.groupby("category")["value"].agg(["count", "mean", "std"])

A mean computed from three samples and a mean computed from three hundred look identical in a table. count is the cheapest defence against a class of embarrassing conclusions.

21. Check for censoring on target variables

Measured outcomes that terminate at exactly round numbers are suspect. Bounded design variables are expected; bounded responses usually indicate clipping at an instrument, scale or reporting limit.

(df["target"] >= upper_limit - tol).sum()
(df["target"] <= lower_limit + tol).sum()

One row at a boundary is coincidence. Several is censoring, the true value exceeded the limit and was truncated.

Why it matters: a censored target caps what the model can learn, and biases residuals precisely at the extremes. If you are optimising toward high values of that target, the bias sits exactly where you care most.

Part VI - Targets, features and interpretation

22. The target deserves special attention

In supervised learning, the target is the result the model is trying to learn. A missing target normally makes an observation unusable as a labelled example. Errors in the target are especially damaging, because the model is explicitly trained to reproduce them. A perfect set of input features cannot compensate for unreliable target measurements. Check the target distribution and class balance as a matter of course, alongside the censoring test.

23. Suitability is partly task-specific

A row does not have to be universally "good" or "bad". A specimen with valid formulation data, valid mechanical strength and missing density can serve a strength model that does not use density, and must be excluded once density becomes an essential input. Hence the separation of clean master dataset from task-specific modelling dataset.

24. Domain knowledge belongs in the features

Derived features encoding known physical relationships routinely outperform raw ingredient columns:

df["w_b_ratio"] = df["water"] / df["binder"]

A ratio can be more predictive than either component alone, because it encodes a physical law the model would otherwise have to rediscover from limited data.

This is the strongest available argument for keeping materials and process engineers central to ML projects, and it is demonstrable in four lines of code. Keep the demonstration; it is useful in a steering meeting. Domain expertise is part of the ML workflow, not something added after modelling.

25. Correlations mislead in both directions

A weak correlation does not mean no effect. A weak correlation between an additive and the property it is supposed to control has three common explanations:

1. Narrow range: an additive varied only between 1% and 2% has little leverage to demonstrate an effect
2. Confounding: another variable drives the same outcome in the opposite direction, masking the relationship
3. Nonlinearity or saturation: the effect flattens above a threshold, so a linear correlation understates it

Pairwise correlation cannot separate these. Partial dependence and SHAP can.

A strong correlation does not establish causation. If strength rises with density, the relationship may instead reflect a third process variable, formulation differences, batch effects, test-method differences or a confounded experimental design. ML identifies predictive relationships unless the experimental design supports causal conclusions. Engineering interpretation must remain separate from statistical association.

26. Understand the objective before optimising

Distinguish:

• Single objective: maximise one property
• Constrained single objective: maximise A subject to B ≥ threshold
• Multi-objective: properties trade off; no single optimum exists, only a Pareto front

Negatively correlated properties (thermal conductivity versus flexural strength, for example) mean there is no best formulation, only a curve of trade-offs.

A constraint threshold is usually a business assumption, not a law of nature. Mapping the Pareto front alongside the constrained optimisation lets you ask whether a threshold is a hard requirement or a round number that was written down once and never revisited.

Optimisation is often most valuable when it interrogates the constraint rather than obeying it.

Part VII - Synthetic data

27. Workign with data you know to be generated

Synthetic data is a legitimate and valuable teaching asset, provided its role is explicit.

What it is good for:

• Learning a pipeline where you can verify the result
• Validating that interpretability methods recover relationships you know to be present, an exercise that is impossible with real data, because with real data the truth is what you are trying to discover
• Stress-testing an approach before committing real samples

What it cannot do:

• Support a business case
• Tell you anything about the physical system it imitates

Protect the ground truth. If a colleague generated the data, ask them for the script and ask them not to share it until you have made your interpretation. Checking a model's explanation against a known generating function is the single most direct test of whether to trust that explanation, and it works only once.

Calibrate expectations accordingly. Synthetic data has no sensor drift, no batch effects, no operator variation, and near-perfect design coverage. Models will perform better than they will on real data. When performance drops on the real dataset, that is the simulation-to-laboratory gap, not a broken pipeline.

Part VIII - Documentation

28. A dataset card is part of the dataset

A CSV tells someone what values are stored. It does not explain why the dataset exists, what one row represents, where the data came from, how it was cleaned, what assumptions were made, which values were excluded, what the known limitations are, or which uses are and are not appropriate.

A Dataset Card supplies that context. Useful sections:

• Dataset purpose
• Unit of observation
• Variables and units
• Data source
• Data-quality processing
• Known limitations
• Intended use
• Out-of-scope use

Good practice treats documentation as part of the technical dataset, not as optional administration.

Part IX - The two habits underneath all of the above

Verify, do not assume. Configuration that produces no error has not necessarily taken effect. A model that produces a plausible number has not necessarily produced a correct one. Silence is not confirmation.

Test properties you can derive, not values you assume. Symmetric data must give Cp = Cpk. A shifted mean must reduce Cpk while leaving Cp unchanged. Fractions must sum to their stated total. Invariants catch bugs that eyeballing a number never will, and they are the same instinct used to check whether a model's behaviour is consistent with known physics.

Appendix A - The workflow, in order, with checkpoints

1. Define the engineering problem. What decision will the model support?

2. Define the unit of observation. What does one row represent?

3. Preserve the raw data. Never overwrite it.

4. Run the inspection ritual.

• [ ] shape, head, info, describe, columns
• [ ] Types correct: no numeric column arriving as object
• [ ] Min/max physically plausible
• [ ] Unique IDs verified
• [ ] Missing data quantified
• [ ] Zero-variance columns identified

5. Visualise.

• [ ] Histogram of every variable, inspected for shape, gaps and boundaries
• [ ] Scatter, box, time-series and batch-by-batch plots as appropriate

6. Establish provenance.

• [ ] Measured, designed, or generated?
• [ ] Asked what happened to the failures

7. Check duplicates and replicates. True duplicates or legitimate replicates?

8. Check categorical consistency. Standardise labels, names, units and conventions.

9. Perform engineering range checks. Physical and process knowledge, not statistics alone.

10. Investigate questionable records. Return to source data wherever possible.

11. Run the structural checks.

• [ ] Columns sorted into design variables / intermediates / outcomes
• [ ] Compositional closure checked, including any implicit balance component
• [ ] Setpoint and measured actual held separately
• [ ] Constraint violations counted, flagged and documented, never silently repaired
• [ ] Row counts recorded before and after every merge
• [ ] Group sizes reported alongside group means
• [ ] Targets checked for censoring at scale limits
• [ ] Class balance or target distribution understood

12. Record every cleaning decision. Maintain the issue log.

13. Create the clean master dataset.

14. Create the task-specific modelling dataset.

15. Define the objective structure. Single, constrained, or multi-objective.

16. Document the dataset. Create the Dataset Card.

17. Write the verdict. Is this data fit for the decision it is meant to support?

18. Only then begin modelling.

Appendix B - Principles in one line each

Garbage in, garbage out is incomplete. In manufacturing ML, undocumented assumptions in can be just as dangerous as obviously bad data.

Missing means unknown, not zero.

Never delete an outlier merely because it is inconvenient. Investigate it.

Never invent a correction because the answer appears obvious. Verify it.

A violation too large to be rounding error is a data integrity finding, not a formatting inconvenience.

Zero missing values in real work is a red flag, not a quality signal. Always ask what happened to the failures.

Data that is too clean is as informative as data that is too messy.

Replicates capture variability; duplicates artificially reinforce observations.

Identifiers can be essential for traceability and validation even when they are not model features.

Raw data, clean data and modelling data are separate assets.

Data cleaning should be reproducible and auditable.

Ask of every candidate input: would this value exist at the moment I need the prediction?

Optimisation can only act on what you can set.

Domain expertise is part of the ML workflow, not something added after modelling.

Silence is not confirmation. Verify, do not assume.

A model should only be as trusted as the data, measurements, assumptions and validation behind it.