Open Lab/Preprocessing · Method

Standard Normal Variate

SNV

A spectrum-wise preprocessing method that centers and scales each spectrum to reduce intensity and scatter-related variation.

SpectroscopyNIRPythonMATLABTutorial

What is SNV?

Standard Normal Variate is a spectrum-wise preprocessing transformation used with spectroscopic data, in particular near-infrared diffuse reflectance spectra.

Each spectrum is treated independently. The transformation subtracts that spectrum's mean and divides by its standard deviation. The transformed spectrum then has mean $0$ and sample standard deviation $1$ under the convention used in Equations 1 to 3.

Barnes, Dhanoa and Lister introduced SNV for near-infrared diffuse reflectance spectroscopy in order to reduce variation associated with scatter and particle size effects.

Why use SNV?

Spectroscopic measurements can contain physical variation that is not the chemical signal of interest. In diffuse reflectance spectroscopy, scatter and particle size effects can change the offset and the overall amplitude of a spectrum even when the underlying absorption features are similar.

SNV is commonly used as a scatter-correction preprocessing method. By centering and scaling each spectrum with its own statistics, it can reduce those global intensity differences before exploratory analysis or calibration.

Preprocessing is not automatically useful for every dataset. Its value depends on the presence of unwanted physical variation and on the analysis that follows.

How does it work?

The operation is row-wise. For a matrix of samples by spectral variables, SNV uses only the statistics of the current spectrum.

Compute the mean of the current spectrum.
Subtract that mean from every variable in the spectrum.
Compute the sample standard deviation of the spectrum.
Divide the centered spectrum by that standard deviation.

The spectrum is then expressed relative to its own intensity distribution. Shared peak positions remain. Offsets and global amplitudes are reduced.

Mathematics & algorithm

For spectrum $i$ and spectral variable $j$ , SNV is the standardization of that row.

Mathematical foundation x ~_{ij} = \frac{x _{ij} - x ˉ _{i}}{s _{i}} (1) x ˉ_{i} = \frac{1}{p} j = 1 \sum p x_{ij} (2) s_{i} = \frac{1}{p - 1} j = 1 \sum p (x_{ij} - x ˉ_{i})^{2} (3) i spectrum / sample index j spectral variable / wavelength index p number of spectral variables x_{ij} original intensity x ˉ_{i} mean of spectrum i s_{i} sample standard deviation of spectrum i x ~_{ij} SNV-transformed value Interpretation Each spectrum is centered and scaled using only its own statistics. The denominator in Equation 3 is p minus 1, the sample standard deviation used in the code below.

Algorithm 1

Standard Normal Variate

InputSpectral matrix X ∈ ℝⁿˣᵖ, rows = spectra

OutputSNV-transformed matrix X̃

01for $i \leftarrow 1$ to $n$ do
02 $μ_{i} \leftarrow mean (x_{i})$
03 $s_{i} \leftarrow std (x_{i})$
04if $s_{i} = 0$ then
05error("zero-variance spectrum")
06end if
07for $j \leftarrow 1$ to $p$ do
08 $x ~_{ij} \leftarrow (x_{ij} - μ_{i}) / s_{i}$
09end for
10end for
11return $X ~$

This pseudocode is a SPARKS representation of Equations 1 to 3. It is not a verbatim extract from the cited papers. std denotes the sample standard deviation. A spectrum with sᵢ = 0 cannot be standardized.

Visual example

Four NIR-like traces share the same peak positions and differ by additive offset and a multiplicative scale. After SNV, each row is centered and scaled to unit sample standard deviation.

Controlled scatter model. Left: raw traces. Right: the same rows after SNV. Residual peak-shape differences remain.

Code

The functions below are SPARKS implementations of Equations 1 to 3. They are not copied from the cited papers. Both operate on a matrix with rows as samples and columns as spectral variables. They use the sample standard deviation and reject a spectrum whose standard deviation is zero.

import numpy as np def snv(X):    X = np.asarray(X, dtype=float)     mean = X.mean(axis=1, keepdims=True)    std = X.std(axis=1, ddof=1, keepdims=True)     if np.any(std == 0):        raise ValueError(            "SNV cannot be applied to a spectrum with zero standard deviation."        )     return (X - mean) / std

Practical notes

SNV is widely used in near-infrared and other vibrational spectroscopic preprocessing.
Each spectrum is processed independently. No reference spectrum is required.
The original motivation was to reduce scatter and particle size effects in diffuse reflectance NIR spectra. SNV is not a general correction for every baseline effect.
Preprocessing should be chosen with the dataset and the downstream analysis in mind. It is not required in every application.
SNV and MSC can give similar looking results, but they are not geometrically identical and should not be treated as strictly interchangeable.
Division by the spectral standard deviation is undefined when that standard deviation is zero.

References

1.
Barnes, R. J., Dhanoa, M. S., & Lister, S. J. (1989). Standard Normal Variate Transformation and De-Trending of Near-Infrared Diffuse Reflectance Spectra. Applied Spectroscopy, 43(5), 772-777.
doi:10.1366/0003702894202201
2.
Rinnan, Å., van den Berg, F. W. J., & Engelsen, S. B. (2009). Review of the most common pre-processing techniques for near-infrared spectra. TrAC Trends in Analytical Chemistry, 28(10), 1201-1222.
doi:10.1016/j.trac.2009.07.007
3.
Fearn, T., Riccioli, C., Garrido-Varo, A., & Guerrero-Ginel, J. E. (2009). On the geometry of SNV and MSC. Chemometrics and Intelligent Laboratory Systems, 96(1), 22-26.
doi:10.1016/j.chemolab.2008.11.006
4.
Rinnan, Å. (2014). Pre-processing in vibrational spectroscopy – when, why and how. Analytical Methods, 6(18), 7124-7129.
doi:10.1039/C3AY42270D

← Back to Open Lab