Text and Image Detectors

PyOD’s EmbeddingOD chains foundation model encoders (sentence-transformers, OpenAI, HuggingFace) with any PyOD detector for text and image anomaly detection. Rankings from NLP-ADBench.

See Layer 1: Text and Image Anomaly Detection for usage.

pyod.models.embedding module

EmbeddingOD and MultiModalOD: Anomaly detection via foundation model embeddings.

EmbeddingOD chains any embedding encoder with any PyOD detector, enabling anomaly detection on text, image, and other non-tabular data through PyOD’s standard API. MultiModalOD extends this to multi-modal data by running separate detectors per modality and fusing their scores.

class pyod.models.embedding.EmbeddingOD(encoder, detector='LUNAR', contamination=0.1, batch_size=32, cache_embeddings=False, reduce_dim=None, standardize=True, random_state=None)[source]

Bases: BaseDetector

Anomaly detection on raw data via embedding + detector pipeline.

Chains any embedding encoder with any PyOD detector. Encode raw data (text, images, or other modalities) into numeric embeddings, then apply outlier detection in the embedding space.

This implements the two-step approach shown to outperform end-to-end methods in NLP-ADBench (Li et al., EMNLP 2025) and TAD-Bench (Cao et al., 2025).

Parameters:

  • encoder (str, BaseEncoder, SentenceTransformer instance, or callable) –

    Embedding encoder. Accepts: - Registry shortcut: ‘all-MiniLM-L6-v2’, ‘text-embedding-3-small’,

    ’dinov2-base’

    • HuggingFace model ID: ‘sentence-transformers/all-MiniLM-L6-v2’

    • Local filesystem path: ‘/path/to/local/weights’ — loaded without any network call, suitable for air-gapped environments.

    • Pre-instantiated SentenceTransformer: passed directly, no reload.

    • BaseEncoder instance

    • Callable: fn(X) -> np.ndarray of shape (n_samples, n_features)

  • detector (str or BaseDetector, optional (default='LUNAR')) – Any PyOD detector. String resolves to default-configured instance. Default is LUNAR (best performer in NLP-ADBench).

  • contamination (float, optional (default=0.1)) – Expected proportion of outliers in the dataset. Must be in (0, 0.5].

  • batch_size (int, optional (default=32)) – Batch size for encoding.

  • cache_embeddings (bool, optional (default=False)) – Cache training embeddings to avoid re-encoding. Recommended for API-based encoders (e.g., OpenAI).

  • reduce_dim (int or None, optional (default=None)) – If set, apply PCA to reduce embedding dimensionality before detection. Recommended for embeddings >1000 dims with distance-based detectors (KNN, LOF).

  • standardize (bool, optional (default=True)) – Apply StandardScaler to embeddings before detection. Matches the preprocessing pipeline in NLP-ADBench.

  • random_state (int, RandomState instance or None, optional (default=None)) – Controls stochastic parts of EmbeddingOD. The seed is forwarded to (a) the dimensionality-reduction PCA when reduce_dim is set (PCA may pick a randomized SVD solver on high-dimensional embeddings) and (b) the string-resolved inner detector when that detector class declares an explicit random_state parameter (e.g., the default 'LUNAR' preset, or 'IForest'). It does NOT control the external encoder’s own inference (e.g., sentence-transformers, DINOv2), which is treated as deterministic given fixed weights. When ADEngine(random_state=...) builds a preset plan, the engine seed flows here automatically.

decision_scores_

Outlier scores of the training data. Higher is more abnormal.

Type:

numpy array of shape (n_samples,)

threshold_

Score threshold based on contamination.

Type:

float

labels_

Binary labels of training data (0: inlier, 1: outlier).

Type:

numpy array of shape (n_samples,)

encoder_

The resolved encoder instance.

Type:

BaseEncoder

detector_

The resolved and fitted detector instance.

Type:

BaseDetector

Examples

>>> from pyod.models.embedding import EmbeddingOD
>>> clf = EmbeddingOD(encoder='all-MiniLM-L6-v2', detector='KNN')
>>> clf.fit(train_texts)
>>> scores = clf.decision_function(test_texts)
>>> labels = clf.predict(test_texts)

# Air-gapped: local filesystem weights >>> clf = EmbeddingOD(encoder=’/path/to/local/weights’, detector=’KNN’) >>> clf.fit(texts)

# Pre-instantiated model (e.g., shared across multiple classifiers) >>> from sentence_transformers import SentenceTransformer >>> my_model = SentenceTransformer(‘all-MiniLM-L6-v2’) >>> clf = EmbeddingOD(encoder=my_model, detector=’IForest’) >>> clf.fit(texts)

decision_function(X)[source]

Predict raw anomaly scores for X.

Parameters:

X (list or array-like) – Raw input data in the same format as fit().

Returns:

anomaly_scores – Anomaly scores. Higher is more abnormal.

Return type:

numpy array of shape (n_samples,)

fit(X, y=None)[source]

Fit detector on raw input data.

Encodes X into embeddings, applies preprocessing, then fits the inner detector.

Parameters:

  • X (list or array-like) – Raw input data (e.g., list of strings for text, list of PIL Images for images).

  • y (Ignored) – Not used, present for API consistency.

Returns:

self – Fitted estimator.

Return type:

object

classmethod for_audio(quality='balanced', **kwargs)[source]

Create an EmbeddingOD configured for audio anomaly detection.

Uses a handcrafted 74-dim acoustic feature encoder (20 MFCC, 12 chroma, and 5 spectral descriptors, each as mean and standard deviation over frames) followed by a classical PyOD detector. This embed-then-detect pattern with classical detectors is competitive on standard audio anomaly detection benchmarks and needs no GPU. Requires pyod[audio] (librosa, soundfile).

Input clips may be file paths, waveform arrays, or (waveform, sample_rate) tuples.

Parameters:

  • quality (str, optional (default='balanced')) –

    • ‘fast’: handcrafted features + IForest.

    • ’balanced’: handcrafted features + KNN.

    • ’best’: handcrafted features + LUNAR (requires torch).

  • **kwargs – Override any EmbeddingOD parameter.

Returns:

clf

Return type:

EmbeddingOD

classmethod for_image(quality='balanced', **kwargs)[source]

Create an EmbeddingOD configured for image anomaly detection.

Configurations are informed by AnomalyDINO (WACV 2025).

Parameters:

  • quality (str, optional (default='balanced')) –

    • ‘fast’: DINOv2-small (384d) + KNN.

    • ’balanced’: DINOv2-base (768d) + LOF.

    • ’best’: DINOv2-large (1024d) + KNN.

  • **kwargs – Override any EmbeddingOD parameter.

Returns:

clf

Return type:

EmbeddingOD

classmethod for_text(quality='balanced', **kwargs)[source]

Create an EmbeddingOD configured for text anomaly detection.

Configurations are informed by NLP-ADBench (EMNLP 2025).

Parameters:

  • quality (str, optional (default='balanced')) –

    • ‘fast’: MiniLM encoder (384d) + KNN. No API key needed.

    • ’balanced’: mpnet encoder (768d) + LUNAR. No API key needed.

    • ’best’: OpenAI large (3072d) + LUNAR. Requires API key.

  • **kwargs – Override any EmbeddingOD parameter.

Returns:

clf

Return type:

EmbeddingOD

predict_proba(X, method='linear', return_confidence=False)[source]

Predict the probability of a sample being an outlier.

Overrides the base implementation to handle list inputs (raw data such as text or images) which do not have a .shape attribute.

Parameters:

  • X (list or array-like) – Raw input data in the same format as fit().

  • method (str, optional (default='linear')) – Probability conversion method. One of ‘linear’ or ‘unify’.

  • return_confidence (boolean, optional (default=False)) – If True, also return the confidence of prediction.

Returns:

outlier_probability

Return type:

numpy array of shape (n_samples, n_classes)

set_predict_proba_request(*, method: bool | None | str = '$UNCHANGED$', return_confidence: bool | None | str = '$UNCHANGED$') → EmbeddingOD

Configure whether metadata should be requested to be passed to the predict_proba method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict_proba if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict_proba.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • method (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for method parameter in predict_proba.

  • return_confidence (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_confidence parameter in predict_proba.

Returns:

self – The updated object.

Return type:

object

set_predict_request(*, return_confidence: bool | None | str = '$UNCHANGED$') → EmbeddingOD

Configure whether metadata should be requested to be passed to the predict method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

return_confidence (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_confidence parameter in predict.

Returns:

self – The updated object.

Return type:

object

class pyod.models.embedding.MultiModalOD(modalities, combination='average', contamination=0.1, standardize_scores=True)[source]

Bases: BaseDetector

Multi-modal anomaly detection via score fusion.

Runs a separate detector per modality and combines their anomaly scores. Each modality can use a different detector and encoder. Score combination uses PyOD’s existing combination functions.

This is complementary to using MultiModalEncoder with EmbeddingOD (early/feature fusion). Score fusion is preferred when modalities have very different characteristics or when per-modality anomaly scores are independently meaningful.

Parameters:

  • modalities (dict of {str: BaseDetector}) – Maps modality name to a detector. Each detector can be: - An EmbeddingOD instance (for text/image modalities) - Any BaseDetector instance (for tabular modalities)

  • combination (str, optional (default='average')) – Score combination method. One of ‘average’, ‘maximization’, ‘median’.

  • contamination (float, optional (default=0.1)) – Expected proportion of outliers. Used for threshold and labels on the combined scores.

  • standardize_scores (bool, optional (default=True)) – Standardize per-modality scores to zero mean and unit variance before combination. Recommended when detectors produce scores on different scales.

decision_scores_

Combined outlier scores of the training data.

Type:

numpy array of shape (n_samples,)

threshold_

Score threshold based on contamination.

Type:

float

labels_

Binary labels (0: inlier, 1: outlier).

Type:

numpy array of shape (n_samples,)

detectors_

The fitted detectors per modality.

Type:

dict of {str: BaseDetector}

Examples

>>> from pyod.models.embedding import EmbeddingOD, MultiModalOD
>>> from pyod.models.knn import KNN
>>> clf = MultiModalOD(modalities={
...     'text': EmbeddingOD(encoder='all-MiniLM-L6-v2', detector='KNN'),
...     'tabular': KNN(),
... })
>>> data = {'text': train_texts, 'tabular': X_train}
>>> clf.fit(data)
>>> scores = clf.decision_function(data)
decision_function(X)[source]

Predict combined anomaly scores for X.

Parameters:

X (dict of {str: data}) – Maps modality name to test data. A modality value of None means that modality is entirely missing for all test samples; its score is imputed as 0. When standardize_scores=True (default), 0 is the training mean, so the missing modality contributes “average” to the combined score. When standardize_scores=False, 0 is a raw score and may not be neutral; enable standardization for principled missing-data handling. Note that imputation reduces variance in the fused score compared to training, so predict() thresholds may be less calibrated. Use decision_function() and apply custom thresholds for best results with missing modalities.

Returns:

anomaly_scores

Return type:

numpy array of shape (n_samples,)

fit(X, y=None)[source]

Fit a detector per modality on the input data.

Parameters:

  • X (dict of {str: data}) – Maps modality name to training data. Keys must match the modalities dict.

  • y (Ignored) – Not used, present for API consistency.

Returns:

self – Fitted estimator.

Return type:

object

set_predict_proba_request(*, method: bool | None | str = '$UNCHANGED$', return_confidence: bool | None | str = '$UNCHANGED$') → MultiModalOD

Configure whether metadata should be requested to be passed to the predict_proba method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict_proba if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict_proba.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

  • method (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for method parameter in predict_proba.

  • return_confidence (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_confidence parameter in predict_proba.

Returns:

self – The updated object.

Return type:

object

set_predict_request(*, return_confidence: bool | None | str = '$UNCHANGED$') → MultiModalOD

Configure whether metadata should be requested to be passed to the predict method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

return_confidence (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_confidence parameter in predict.

Returns:

self – The updated object.

Return type:

object