Open Lab/Preprocessing · Method

Normalization & Scaling

Transformations that control overall sample magnitude or variable influence before multivariate analysis.

ChemometricsSpectroscopyNormalizationMean CenteringAutoscalingPareto ScalingPythonMATLAB

What are Normalization & Scaling?

Normalization and scaling modify the numerical representation of multivariate data before analysis. They are not interchangeable operations.

Sample-wise normalization acts on each sample independently. Each row is rescaled relative to a measure of its own magnitude, for example its Euclidean norm. Variable-wise centering and scaling act across samples. Each column is translated and, when scaling is applied, divided by a measure of its spread.

The distinction matters because these transformations change the geometry of the data. In component analysis, that geometry determines how samples and variables contribute to the extracted components. Bro and Smilde treat centering and scaling as modelling choices rather than as a single generic pretreatment.

Why use them?

Overall signal intensity can vary between measurements for reasons that are unrelated to the relative profile of interest. Under assumptions appropriate to the application, sample-wise normalization can reduce those magnitude differences.

Variables can also have very different variances. Methods that depend on variance or covariance are then dominated by high-variance variables. Column-wise scaling changes that relative contribution. Equal weighting is not automatically desirable, and scaling does not automatically improve a model.

Bro and Smilde treat centering and scaling as part of the analysis rather than as a single generic pretreatment: centering relates to the structural model, and scaling relates to how the model is fitted. van den Berg et al. likewise conclude that the pretreatment choice depends on the analytical question, the data, and the subsequent analysis.

How do they work?

The four transformations on this page operate in different directions. Vector normalization is row-wise. Mean centering, autoscaling, and Pareto scaling are column-wise.

Vector normalization. Treat one spectrum as a vector. Compute its Euclidean norm and divide every intensity by that norm. The resulting spectrum has unit Euclidean norm. A zero-norm spectrum cannot be normalized this way.

Mean centering. For each variable, compute the mean across samples and subtract that mean from every sample value. The centered variable has mean zero. Centering moves the origin. It does not by itself rescale the variance.

Autoscaling, also called unit variance scaling in this context, mean-centers each variable and then divides by its sample standard deviation. Each resulting nonconstant variable has mean zero and unit sample standard deviation under the convention used below.

Pareto scaling mean-centers each variable and then divides by the square root of its standard deviation. van den Berg et al. describe this as an intermediate scaling strength between leaving the variance unchanged and autoscaling.

Mathematics & algorithm

Let $X \in R^{n \times p}$ be a matrix of $n$ samples by $p$ variables. Row $i$ is a sample. Column $j$ is a variable.

Vector normalization x_{i} = [x_{i 1}, x_{i 2}, \dots, x_{i p}] (1) ∥ x_{i} ∥_{2} = j = 1 \sum p x_{ij}^{2} (2) x ~_{i} = \frac{x _{i}}{∥ x _{i} ∥ _{2}} (3) ∥ x ~_{i} ∥_{2} = 1 (4) i sample / spectrum index j variable index x_{i} row i of X ∥ x_{i} ∥_{2} Euclidean norm of sample i x ~_{i} vector-normalized sample Interpretation Equation 3 is applied independently to each nonzero row. Equation 4 holds after a successful normalization. A sample with Euclidean norm zero cannot be divided in this way.

Mean centering x ˉ_{j} = \frac{1}{n} i = 1 \sum n x_{ij} (5) x_{ij, c} = x_{ij} - x ˉ_{j} (6) n number of samples x ˉ_{j} mean of variable j across samples x_{ij, c} mean-centered value Interpretation Centering translates the origin of each variable. It does not by itself change the relative variances of the variables.

Autoscaling x_{ij, UV} = \frac{x _{ij} - x ˉ _{j}}{s _{j}} (7) s_{j} sample standard deviation of variable j, using denominator n minus 1 x_{ij, UV} autoscaled, or unit-variance scaled, value Interpretation The transformed variable has mean zero and unit sample standard deviation, subject to floating-point precision. Autoscaling and unit variance scaling are used as equivalent terms here. Bro and Smilde provide the primary chemometric discussion of this column-wise operation.

Pareto scaling x_{ij, Pareto} = \frac{x _{ij} - x ˉ _{j}}{s _{j}} (8) s_{j} square root of the standard deviation of variable j Interpretation Pareto scaling divides the centered variable by the square root of its standard deviation, not by the square root of its variance. Because s_{j} is already the square root of the variance, the divisor is the square root of s_{j} . The resulting variables do not in general have unit variance. This convention follows van den Berg et al.

Algorithm 1

Normalization and Variable Scaling

InputData matrix X ∈ ℝⁿˣᵖ, transformation method m

OutputTransformed matrix X̃

01if $m = vector_normalization$ then
02for $i \leftarrow 1$ to $n$ do
03 $norm \leftarrow ∥ x_{i} ∥_{2}$
04if $norm = 0$ then
05error("zero-norm sample")
06end if
07 $x ~_{i} \leftarrow x_{i} / norm$
08end for
09else if $m = mean_center$ then
10 $μ \leftarrow column means of X$
11 $X ~ \leftarrow X - μ$
12else if $m = autoscale$ then
13 $μ \leftarrow column means of X$
14 $s \leftarrow column sample standard deviations of X$
15if $any s_{j} = 0$ then
16error("zero-variance variable")
17end if
18 $X ~ \leftarrow (X - μ) / s$
19else if $m = pareto$ then
20 $μ \leftarrow column means of X$
21 $s \leftarrow column sample standard deviations of X$
22if $any s_{j} = 0$ then
23error("zero-variance variable")
24end if
25 $X ~ \leftarrow (X - μ) / s$
26end if
27return $X ~$

This pseudocode is a SPARKS representation of Equations 1 to 8. It is not a verbatim extract from the cited papers. Vector normalization is row-wise. Mean centering, autoscaling, and Pareto scaling are column-wise. std denotes the sample standard deviation. Autoscaling and Pareto scaling require at least two samples.

Code

The functions below are SPARKS implementations of Equations 1 to 8. They are not copied from the cited papers. Both languages use a matrix with rows as samples and columns as variables. The sample standard deviation uses denominator $n - 1$ .

Vector normalization uses only the current row. Mean centering, autoscaling, and Pareto scaling estimate column statistics across samples. In a predictive workflow those statistics must be fitted on the training matrix and then applied unchanged to validation or test observations.

Call fit_scaling on the training matrix, then apply transform_scaling to both the training matrix and the test matrix using the same parameters. Do not recompute test-set means or standard deviations when evaluating a trained model. preprocess_scale fits and applies on the same matrix and is appropriate only when no held-out application is required.

import numpy as np def fit_scaling(X, method):    X = np.asarray(X, dtype=float)     if X.ndim != 2:        raise ValueError(            "X must be a 2D array of samples by variables."        )     if not np.all(np.isfinite(X)):        raise ValueError(            "X must contain only finite values."        )     if method == "vector":        return {"method": method}     mean = X.mean(axis=0, keepdims=True)     if method == "center":        return {"method": method, "mean": mean}     if X.shape[0] < 2:        raise ValueError(            "Autoscaling and Pareto scaling require at least two samples."        )     std = X.std(axis=0, ddof=1, keepdims=True)     if np.any(~np.isfinite(std) | (std == 0)):        raise ValueError(            "Scaling cannot be applied to a zero variance variable."        )     if method in ("autoscale", "pareto"):        return {"method": method, "mean": mean, "std": std}     raise ValueError(        "method must be 'vector', 'center', 'autoscale', or 'pareto'."    )  def transform_scaling(X, params):    X = np.asarray(X, dtype=float)     if X.ndim != 2:        raise ValueError(            "X must be a 2D array of samples by variables."        )     if not np.all(np.isfinite(X)):        raise ValueError(            "X must contain only finite values."        )     method = params["method"]     if method == "vector":        norms = np.linalg.norm(X, axis=1, keepdims=True)         if np.any(~np.isfinite(norms) | (norms == 0)):            raise ValueError(                "Vector normalization cannot be applied to a zero norm sample."            )         return X / norms     mean = params["mean"]     if method == "center":        return X - mean     std = params["std"]     if method == "autoscale":        return (X - mean) / std     if method == "pareto":        return (X - mean) / np.sqrt(std)     raise ValueError(        "method must be 'vector', 'center', 'autoscale', or 'pareto'."    )  def preprocess_scale(X, method):    return transform_scaling(X, fit_scaling(X, method))

Practical notes

Sample-wise normalization and variable-wise centering or scaling are different operations. They should not be treated as interchangeable terms.
Vector normalization divides each sample by its Euclidean norm. It does not use statistics computed across samples.
SNV is also row-wise, but it subtracts that spectrum's mean and divides by that spectrum's standard deviation. It is not vector normalization.
Mean centering changes the origin of each variable. It does not by itself change relative variable variances.
Autoscaling gives each nonconstant variable unit sample standard deviation, and therefore unit sample variance. That changes its relative influence in covariance-based analyses.
Pareto scaling is less aggressive than autoscaling because it divides by the square root of the standard deviation rather than by the standard deviation itself. Pareto-scaled variables do not in general have unit variance.
The appropriate scaling strategy depends on the analytical objective, the data structure, and the subsequent analysis. Bro and Smilde treat centering and scaling as part of that analysis rather than as an automatic pretreatment.
Scaling can increase the influence of low-variance variables, including variables dominated by noise.
In a predictive workflow, column means and scaling factors must be estimated from training data and then applied unchanged to validation or test observations.
Common NIR scatter-correction methods such as MSC and EMSC address a different family of multiplicative and additive spectral effects. They are not substitutes for the column-wise scaling discussed here.

References

1.
Bro, R., & Smilde, A. K. (2003). Centering and scaling in component analysis. Journal of Chemometrics, 17(1), 16-33.
doi:10.1002/cem.773
2.
van den Berg, R. A., Hoefsloot, H. C. J., Westerhuis, J. A., Smilde, A. K., & van der Werf, M. J. (2006). Centering, scaling, and transformations: improving the biological information content of metabolomics data. BMC Genomics, 7, Article 142.
doi:10.1186/1471-2164-7-142
3.
Rinnan, Å., van den Berg, F. W. J., & Engelsen, S. B. (2009). Review of the most common pre-processing techniques for near-infrared spectra. TrAC Trends in Analytical Chemistry, 28(10), 1201-1222.
doi:10.1016/j.trac.2009.07.007

← Back to Open Lab