Soft Independent Modelling of Class Analogy

This page uses original residual-distance SIMCA. Each predefined class receives its own principal-component model. A new observation is asked, independently for every class, whether it is sufficiently compatible with that class model.

Wold introduced disjoint principal-component models for pattern recognition: each class is represented by its own PCA model rather than by one global mapping of all classes. Wold and Sjöström named the chemical-data method SIMCA, for analysis in terms of similarity and analogy. Vitale, Cocchi, Biancolillo, Ruckebusch, and Marini treat SIMCA as a class-modelling method: the operational question is class membership relative to a modelled target class, not only a forced comparison among the classes that happen to be present.

SIMCA is therefore not PCA with labels attached after the fact, and it is not a nearest-class discriminant rule. PCA supplies the class geometry. The class decision is a separate acceptance criterion. This page teaches one verified residual F-rule. Other SIMCA variants use different acceptance regions.

Discriminant classifiers such as LDA, PLS-DA, and k-NN generally assign an observation by comparing available classes under their respective algorithms. SIMCA instead constructs class models independently. Acceptance by one class does not automatically imply rejection by another class.

That distinction is conceptual and must not be overstated. Some discriminant implementations can refuse assignment. The point here is the modelling target: SIMCA estimates the accepted variation of each class and then tests compatibility with each model. Kucheryavskiy, Rodionova, and Pomerantsev contrast one-class / class-modelling methods with discriminant analysis on that basis. Vanden Branden and Hubert describe original SIMCA as a soft classifier: an object may be assigned to one class, several classes, or no class.

Class modelling is the natural question in authentication and identity testing, where the scientifically relevant alternative may be “not the target class” rather than “one of the other labelled training classes.” Vitale et al. present SIMCA in that why, when, and how setting.

Known class labels determine which training observations build each class model. For class $k$, the class matrix is column-centered, PCA is computed, and $A_k$ components are retained. Training orthogonal distances to that PCA subspace determine a class residual variance. A new observation is centered with the stored class mean, projected with the stored loadings, and compared with the class through a residual-variance F statistic.

De Maesschalck, Candolfi, Massart, and Heuerding describe the original residual criterion as Euclidean distance towards the space of selected principal components, compared with a confidence region around that space. Original SIMCA also used a second “closed box” in the selected score space. This page does not teach that score box. The taught decision is the residual F comparison reconstructed for original SIMCA by Vanden Branden and Hubert.

Model parameters come from class training data only. PCA is not recomputed for the new observation. Validation and test objects do not enter class means, loadings, residual variances, or critical values.

Independent means that each class is modelled from its own training subset. Class $k$ has its own mean, PCA decomposition, component count, residual variance, and F limit. The page does not require a different $A_k$ for every class. Separate selection is allowed. A shared component count is a convenience, not a scientific law.

The class data matrix is $X_k\in {\mathbb{R}}^{n_k\times p}$, with $n_k$ training objects of class $k$ and $p$ variables. Samples are rows. The class mean and centered matrix follow the Open Lab PCA convention. De Maesschalck et al. write the class SVD after column centering.

$A_k$ is a class hyperparameter. It is not a universal 95% explained-variance rule. Kucheryavskiy et al. reject explained-variance percentage as a SIMCA law. Vitale et al. treat model complexity as a quantity to be fine-tuned and validated. Select $A_k$ inside the model-development validation design, not on the final test set.

Notation matches the PCA page. Distances below are not interchangeable with DD-SIMCA score distance $h$, squared residual $q$, or combined-index SIMCA.

The algorithm is the residual F-rule above. It returns a Boolean membership matrix, not a forced integer label.

Soft, in the original SIMCA sense used here, means that class decisions are not mutually exclusive. An observation may be accepted by class A only, by class B only, by both, or by neither. Vanden Branden and Hubert state that original SIMCA assignment. Independent means that each class model is estimated from its own training objects and is queried separately.

Soft membership is not a posterior probability. The residual F-rule produces an accept/reject decision, not $P(k\mid x)$. Do not interpret $F_k(x)$ as a class probability.

If several class models accept the same observation, this page keeps all acceptances. It does not add a nearest-class tie-breaker. If every class rejects the observation, it remains unassigned. That no-class outcome is part of class modelling. It is not a defect of the algorithm.

$A_k$ and, if varied, $\alpha$ are model-development choices. Do not tune them on the final test set. Any data-dependent pretreatment, including SNV, MSC, derivatives, scaling, or variable selection, belongs inside the same validation design. The Cross-Validation page states that rule. Class means, loadings, and residual limits are estimated from the training portion of each split.

Class-modelling evaluation is not only multiclass accuracy. For a target class model, sensitivity is the ability to accept objects that belong to that class. It is estimated from target members. Specificity is the ability to reject objects that do not belong to the target class. It requires alternative or non-target objects. Kucheryavskiy et al. make that split explicit. Specificity cannot be estimated from target-class objects alone. The Classification Metrics page gives the formulas: sensitivity $\mathrm{TP}/(\mathrm{TP}+\mathrm{FN})$ and specificity $\mathrm{TN}/(\mathrm{TN}+\mathrm{FP})$. This page does not introduce new metric equations.

De Maesschalck et al. found that the original residual rule, using fitted rather than cross-validated training scores, can produce a large Type I error on their NIR tablet data. They examine leave-one-out predicted scores and Mahalanobis-type alternatives. Those modifications are not the page default. Pomerantsev and Rodionova review several popular SIMCA decision rules. The educational default $\alpha =0.05$ matches the original 95% F quantile stated by Vanden Branden and Hubert. It is not a universal optimum.

De Maesschalck et al. studied SIMCA decision criteria on near-infrared spectra of pharmaceutical tablets, including SNV pretreatment. That paper is a chemometric example of residual class modelling on NIR data. It is not a claim that one NIR result covers every spectroscopic problem.

Vitale et al. place SIMCA in authentication and related class-modelling tasks, where the question is compatibility with a reference class. Spectroscopic SIMCA is sensitive to the representation that defines the PCA class geometry. SNV, MSC, derivatives, baseline correction, scaling, and variable selection are modelling choices. They are not universal SIMCA requirements. If chosen from data, they belong inside validation. This page uses class-specific column centering as the required PCA convention. It does not require class-wise autoscaling, and it does not require one global scaler.

SIMCA is not PCA. PCA models variation in a matrix. SIMCA adds class labels to build separate PCA models and then applies an acceptance rule. SIMCA is not LDA: LDA uses class means, a shared covariance, and priors under the Open Lab Gaussian formulation. SIMCA is not PLS-DA: PLS-DA uses class-coded responses in a PLS framework. SIMCA is not k-NN: k-NN votes among nearby labelled neighbors. None of these comparisons is a ranking of accuracy.

Data-Driven SIMCA is a later PCA-based one-class formulation. It uses score distance and orthogonal distance with data-driven limits, as described by Kucheryavskiy, Rodionova, and Pomerantsev. DD-SIMCA is not the algorithm on this page.

scikit-learn does not provide a canonical SIMCA estimator. MATLAB has no official simca classifier in the Statistics and Machine Learning Toolbox. The listings implement Equations 1 to 14. Python uses SVD of the class-centered matrix, matching the PCA page. sklearn PCA mean-centers and does not scale by default, so class-specific sklearn PCA can supply the same subspace. MATLAB pca also column-centers by default. The MATLAB listing uses svd of the centered class matrix so the residual F-rule is explicit and does not depend on a third-party SIMCA toolbox. finv supplies the F quantile. Loading signs may flip. Orthogonal distances and membership should agree when the same residual F-rule is used.

The output is a Boolean membership matrix. It is not a forced class label and not a vector of posterior probabilities.

import numpy as npfrom scipy.stats import f as f_dist  def _validate_matrix(X):    X = np.asarray(X, dtype=float)    if X.ndim != 2:        raise ValueError("X must be a 2D array of samples by variables.")    if min(X.shape) == 0:        raise ValueError("X must have at least one row and one column.")    if not np.all(np.isfinite(X)):        raise ValueError("X must contain only finite values.")    return X  def _validate_xy(X, y):    X = _validate_matrix(X)    y = np.asarray(y).reshape(-1)    if X.shape[0] != y.shape[0]:        raise ValueError("X and y must have the same number of observations.")    return X, y  def simca_class_mean(Xk):    Xk = _validate_matrix(Xk)    return Xk.mean(axis=0)  def simca_class_pca(Xk, n_components):    Xk = _validate_matrix(Xk)    n, p = Xk.shape    A = int(n_components)    if A != n_components or A < 1:        raise ValueError("n_components must be an integer >= 1.")    if A >= p:        raise ValueError("n_components must be smaller than the number of variables.")    if n <= A + 1:        raise ValueError("Class size must exceed n_components + 1.")    mean = simca_class_mean(Xk)    Xc = Xk - mean    _U, _s, Vt = np.linalg.svd(Xc, full_matrices=False)    P = Vt[:A].T    return mean, P  def simca_orthogonal_distance(X, mean, P):    X = _validate_matrix(X)    mean = np.asarray(mean, dtype=float).reshape(-1)    P = np.asarray(P, dtype=float)    if X.shape[1] != mean.shape[0] or P.ndim != 2 or P.shape[0] != mean.shape[0]:        raise ValueError("mean and loadings must match the number of variables.")    Xc = X - mean    T = Xc @ P    E = Xc - T @ P.T    return np.sqrt(np.sum(E ** 2, axis=1))  def simca_class_residual_variance(od_train, p, n_components, n_k):    A = int(n_components)    denom = (p - A) * (n_k - A - 1)    if denom <= 0:        raise ValueError("Residual variance degrees of freedom must be positive.")    od_train = np.asarray(od_train, dtype=float).reshape(-1)    return np.sum(od_train ** 2) / denom  def simca_object_residual_variance(od, p, n_components):    A = int(n_components)    if p - A <= 0:        raise ValueError("Residual space requires p > n_components.")    od = np.asarray(od, dtype=float).reshape(-1)    return (od ** 2) / (p - A)  def simca_f_statistic(s2_new, s2_class):    s2_new = np.asarray(s2_new, dtype=float).reshape(-1)    s2_class = float(s2_class)    if s2_class <= 0:        raise ValueError("Class residual variance must be positive.")    return s2_new / s2_class  def simca_f_critical(p, n_components, n_k, alpha=0.05):    A = int(n_components)    d1 = p - A    d2 = (p - A) * (n_k - A - 1)    if d1 <= 0 or d2 <= 0:        raise ValueError("F degrees of freedom must be positive.")    if not (0 < alpha < 1):        raise ValueError("alpha must be in (0, 1).")    return float(f_dist.ppf(1.0 - alpha, d1, d2))  def fit_simca_class(Xk, n_components, alpha=0.05):    Xk = _validate_matrix(Xk)    n_k, p = Xk.shape    mean, P = simca_class_pca(Xk, n_components)    od_train = simca_orthogonal_distance(Xk, mean, P)    s2_class = simca_class_residual_variance(od_train, p, n_components, n_k)    f_crit = simca_f_critical(p, n_components, n_k, alpha=alpha)    return {        "mean": mean,        "loadings": P,        "n_components": int(n_components),        "n_k": n_k,        "p": p,        "alpha": float(alpha),        "s2_class": s2_class,        "f_critical": f_crit,        "od_train": od_train,    }  def predict_simca_class(model, X):    X = _validate_matrix(X)    if X.shape[1] != model["p"]:        raise ValueError("X must have the same number of variables as the class model.")    od = simca_orthogonal_distance(X, model["mean"], model["loadings"])    s2_new = simca_object_residual_variance(od, model["p"], model["n_components"])    F = simca_f_statistic(s2_new, model["s2_class"])    accepted = F <= model["f_critical"]    return accepted, F, od  def _component_counts(n_components, n_classes):    if np.isscalar(n_components):        return np.full(n_classes, int(n_components), dtype=int)    counts = np.asarray(n_components, dtype=int).reshape(-1)    if counts.shape[0] != n_classes:        raise ValueError("n_components must be a scalar or one value per class.")    return counts  def fit_simca(X, y, n_components, alpha=0.05):    X, y = _validate_xy(X, y)    classes = np.unique(y)    counts = _component_counts(n_components, classes.shape[0])    models = []    for i, k in enumerate(classes):        models.append(fit_simca_class(X[y == k], counts[i], alpha=alpha))    return {"classes": classes, "models": models, "alpha": float(alpha)}  def predict_simca_membership(model, X):    X = _validate_matrix(X)    n = X.shape[0]    n_classes = len(model["models"])    membership = np.zeros((n, n_classes), dtype=bool)    F = np.zeros((n, n_classes), dtype=float)    od = np.zeros((n, n_classes), dtype=float)    for j, class_model in enumerate(model["models"]):        accepted, Fj, odj = predict_simca_class(class_model, X)        membership[:, j] = accepted        F[:, j] = Fj        od[:, j] = odj    return membership, F, od 

1. 1.

   Wold, S. (1976). Pattern recognition by means of disjoint principal components models. Pattern Recognition, 8(3), 127-139.

   doi:10.1016/0031-3203(76)90014-5
2. 2.

   Wold, S., & Sjöström, M. (1977). SIMCA: A Method for Analyzing Chemical Data in Terms of Similarity and Analogy. In B. R. Kowalski (Ed.), Chemometrics: Theory and Application. ACS Symposium Series 52, 243-282.

   doi:10.1021/bk-1977-0052.ch012
3. 3.

   De Maesschalck, R., Candolfi, A., Massart, D. L., & Heuerding, S. (1999). Decision criteria for soft independent modelling of class analogy applied to near infrared data. Chemometrics and Intelligent Laboratory Systems, 47(1), 65-77.

   doi:10.1016/S0169-7439(98)00159-2
4. 4.

   Vanden Branden, K., & Hubert, M. (2005). Robust classification in high dimensions based on the SIMCA Method. Chemometrics and Intelligent Laboratory Systems, 79(1-2), 10-21.

   doi:10.1016/j.chemolab.2005.03.002
5. 5.

   Vitale, R., Cocchi, M., Biancolillo, A., Ruckebusch, C., & Marini, F. (2023). Class modelling by Soft Independent Modelling of Class Analogy: why, when, how? A tutorial. Analytica Chimica Acta, 1270, 341304.

   doi:10.1016/j.aca.2023.341304
6. 6.

   Pomerantsev, A. L., & Rodionova, O. Y. (2020). Popular decision rules in SIMCA: Critical review. Journal of Chemometrics, 34(8), e3250.

   doi:10.1002/cem.3250
7. 7.

   Kucheryavskiy, S., Rodionova, O., & Pomerantsev, A. (2024). A comprehensive tutorial on Data-Driven SIMCA: Theory and implementation in web. Journal of Chemometrics, 38(7), e3556.

   doi:10.1002/cem.3556
8. 8.

   Bro, R., & Smilde, A. K. (2014). Principal component analysis. Analytical Methods, 6(9), 2812-2831.

   doi:10.1039/C3AY41907J
9. 9.

   scikit-learn Developers (n.d.). PCA. scikit-learn documentation.

10. 10.

    The MathWorks, Inc. (n.d.). pca and Classification. MATLAB documentation.