How it works
Data often needs cleaning before analysis because real collections include missing, duplicate, inconsistent, or incorrect values.
Choose transformations that support the question being asked, such as grouping by category or calculating a rate.
Common error
Averages can hide differences between groups or be distorted by outliers.
Worked example
Prepare a survey data set for analysis without silently inventing values.
Data / pseudocode
Row 1: grade 10, minutes 30
Row 2: grade "10 ", minutes 45
Row 3: grade 10, minutes missing
Row 4: duplicate of row 1Standardize harmless formatting differences, investigate the missing value, and check whether the apparent duplicate represents one repeated record or two real responses. Document the cleaning rule before computing statistics. Replacing a missing value with zero would assert something not observed.
Exercises
2.2.1
For observed values 10, 20, and 90, calculate the mean and median. What does the comparison show?
Show solution
The mean is 120/3 = 40 and the median is 20. The large value pulls the mean upward; the two statistics describe the data differently.
2.2.2
Why should a cleaning log record removed rows and the removal rule?
Show solution
It makes the transformation reproducible and lets another analyst assess whether records were excluded unfairly or whether the cleaning changed the result.