Filtering and Spectrum Processing (matchms 0.33.1)

Use this reference when building or debugging metadata-cleaning, peak-processing, or quality-control pipelines. The authoritative API is the matchms [filtering package](https://matchms.readthedocs.io/en/latest/api/matchms.filtering.html).

Filter Contract

Most matchms filters:

• accept one Spectrum as the first argument;
• return a Spectrum or None;
• default to clone=True, so direct calls usually return a modified copy;
• use None to indicate that a require_* condition failed; and
• should be assigned back when called directly.

spectrum = normalize_intensities(spectrum)
spectrum = require_minimum_number_of_peaks(spectrum, n_required=5)
if spectrum is None:
    # The spectrum failed a quality requirement.
    ...

Do not continue passing None through ordinary filters. SpectrumProcessor stops the chain and discards rejected spectra for you.

Prefer SpectrumProcessor

SpectrumProcessor accepts:

• a built-in filter name such as "normalize_intensities";
• a callable such as normalize_intensities; or
• (filter, parameter_dict), where filter can be a name or callable.

Built-in filters are automatically sorted into matchms's required filter order. Custom filters are appended unless an explicit position is supplied with parse_and_add_filter().

from matchms import SpectrumProcessor
from matchms.filtering import (
    default_filters,
    normalize_intensities,
    remove_peaks_relative_to_precursor_mz,
    require_minimum_number_of_peaks,
    require_precursor_mz,
    select_by_mz,
    select_by_relative_intensity,
)

# Run the aggregate metadata harmonizer before SpectrumProcessor. If included as
# one callable inside the processor, it is treated as custom and placed last.
spectra = [default_filters(spectrum) for spectrum in spectra]

processor = SpectrumProcessor(
    [
        (require_precursor_mz, {"minimum_accepted_mz": 50.0, "maximum_mz": 1500.0}),
        normalize_intensities,
        (remove_peaks_relative_to_precursor_mz, {"offset_to_precursor": -1.6}),
        (select_by_mz, {"mz_from": 20.0, "mz_to": 1500.0}),
        (select_by_relative_intensity, {"intensity_from": 0.01}),
        (require_minimum_number_of_peaks, {"n_required": 5}),
    ]
)

cleaned, report = processor.process_spectra(
    spectra,
    progress_bar=False,
    create_report=False,
)
print(processor.processing_steps)

Important current behavior:

• SpectrumProcessor is not callable.
• Use processor.process_spectrum(spectrum.clone()) to preserve one input. Without a processing report, process_spectrum passes clone=False to filters and can mutate its argument; it does not clone on entry.
• Use processor.process_spectra(spectra) for a list.
• process_spectra() returns (processed_spectra, processing_report).
• With create_report=False, the processor clones each input once and disables per-filter cloning where supported.
• With create_report=True, filters are cloned step by step to measure changes. The aggregate default_filters function is not in the processor's built-in order registry and does not expose a clone parameter. If passed directly, it is placed after registered filters and detailed reporting logs a warning. Run it before the processor or expand it into the nine filters below.
• process_spectrums() is a deprecated spelling.

What default_filters() Does

In 0.33.1, default_filters() applies exactly:

1. make_charge_int
2. add_compound_name
3. derive_adduct_from_name
4. derive_formula_from_name
5. clean_compound_name
6. interpret_pepmass
7. add_precursor_mz
8. derive_ionmode
9. correct_charge

It does not normalize peaks, add retention fields, harmonize structural identifiers, require precursor metadata, or enforce peak-count quality. Add those steps explicitly.

Peak Processing

Normalize and select peaks

• normalize_intensities(spectrum_in, clone=True, scaling=None) — normalize to maximum intensity 1 by default; pass scaling=(low, high) for min-max scaling.
• select_by_intensity(spectrum_in, intensity_from=10.0, intensity_to=200.0, clone=True) — keep an absolute intensity interval.
• select_by_relative_intensity(spectrum_in, intensity_from=0.0, intensity_to=1.0, clone=True) — keep a fraction-of-maximum interval.
• select_by_mz(spectrum_in, mz_from=0.0, mz_to=1000.0, clone=True) — crop the fragment m/z interval.

select_by_relative_intensity divides by the current maximum internally; prior normalization is optional, and retained intensities keep their original scale. Cropping away a base peak before this filter can change the retained peaks.

Reduce and clean peaks

• reduce_to_number_of_peaks(spectrum_in, n_required=0, n_max=inf, ratio_desired=None, clone=True) — keep the most intense peaks within count constraints.
• remove_noise_below_frequent_intensities(spectrum_in, min_count_of_frequent_intensities=5, noise_level_multiplier=2.0, clone=True) — estimate and remove repeated low-intensity noise.
• remove_peaks_around_precursor_mz(spectrum_in, mz_tolerance=17, clone=True) — remove peaks within a precursor-centered exclusion window.
• remove_peaks_relative_to_precursor_mz(spectrum_in, offset_to_precursor=-1.6, clone=True) — remove peaks above a cutoff relative to precursor m/z.
• remove_peaks_outside_top_k(spectrum_in, k=6, mz_window=50, clone=True) — retain peaks within mz_window Da of one of the globally k most intense peaks; this is not a per-window top-k filter.
• remove_profiled_spectra(spectrum_in, mz_window=0.5, clone=True) — reject spectra likely to contain profile-mode rather than centroided data.

These defaults are starting points, not instrument-independent truth. Record chosen windows and thresholds.

Peak quality requirements

• require_minimum_number_of_peaks(spectrum_in, n_required=10, ratio_required=None, clone=True)
• require_maximum_number_of_peaks(spectrum_in, maximum_number_of_fragments=1000, clone=True)
• require_minimum_number_of_high_peaks(spectrum_in, no_peaks=5, intensity_percent=2.0, clone=True)

Older examples commonly use invalid require_minimum_number_of_high_peaks(n_required=..., intensity_threshold=...) arguments. In 0.33.1 the names are no_peaks and intensity_percent, where the latter is a percentage rather than a 0-1 fraction.

Precursor, Adduct, Charge, and Ion Mode

• interpret_pepmass(spectrum_in, clone=True)
• add_precursor_mz(spectrum_in, clone=True)
• add_parent_mass(spectrum_in, estimate_from_adduct=True, overwrite_existing_entry=False, estimate_from_charge=True, clone=True)
• add_precursor_formula(spectrum_in, clone=True)
• make_charge_int(spectrum_in, clone=True)
• correct_charge(spectrum_in, clone=True)
• clean_adduct(spectrum_in, clone=True)
• derive_adduct_from_name(spectrum_in, remove_adduct_from_name=True, clone=True)
• derive_ionmode(spectrum_in, clone=True)
• require_correct_ionmode(spectrum_in, ion_mode_to_keep)
• require_matching_adduct_and_ionmode(spectrum)
• require_matching_adduct_precursor_mz_parent_mass(spectrum, tolerance=0.1)
• require_precursor_mz(spectrum_in, minimum_accepted_mz=10.0, maximum_mz=None, clone=True)

require_precursor_below_mz() is deprecated. Use require_precursor_mz(maximum_mz=...).

Modified-cosine and neutral-loss scoring require a valid precursor_mz. Parent-mass matching additionally requires a valid parent_mass. Check that masses are finite and positive: require_precursor_mz uses inclusive bounds and does not itself reject NaN or infinity. The bundled CLI adds that check.

Compound Names, Formulae, and Structures

Name and formula processing

• add_compound_name(spectrum_in, clone=True)
• clean_compound_name(spectrum_in, clone=True)
• derive_formula_from_name(spectrum_in, remove_formula_from_name=True, clone=True)
• derive_formula_from_smiles(spectrum_in, overwrite=True, clone=True)
• require_compound_name(spectrum)
• require_formula(spectrum)

SMILES, InChI, and InChIKey

• derive_inchi_from_smiles(spectrum_in, clone=True)
• derive_inchikey_from_inchi(spectrum_in, clone=True)
• derive_smiles_from_inchi(spectrum_in, clone=True)
• harmonize_undefined_inchi(...)
• harmonize_undefined_inchikey(...)
• harmonize_undefined_smiles(...)
• repair_inchi_inchikey_smiles(spectrum_in, clone=True)
• repair_not_matching_annotation(spectrum_in, clone=True)
• require_valid_annotation(spectrum)

derive_annotation_from_compound_name(spectrum_in, annotated_compound_names_file=None, mass_tolerance=0.1, clone=True) uses PubChemPy name search when no valid SMILES/InChI exists. It requires a plausible compound name and parent_mass, selecting an annotation by neutral mass agreement. The tolerance is an absolute mass difference, not ppm. It can read and append its optional CSV cache and can retry network failures for a long time. Run deliberately, retain original annotations, and preserve the cache with provenance. Name/mass agreement does not resolve isomers or validate the query identity. No credential is required for the public PubChem service.

Structure/mass repair

Current repair helpers include:

• repair_adduct_and_parent_mass_based_on_smiles
• repair_adduct_based_on_parent_mass
• repair_parent_mass_from_smiles
• repair_parent_mass_is_molar_mass
• repair_parent_mass_match_smiles_wrapper
• repair_smiles_of_salts
• require_parent_mass_match_smiles

Several require an explicit mass_tolerance. Do not silently "repair" library annotations without preserving original fields and logging the rule used.

Retention and MS-Level Metadata

• add_retention_time(spectrum_in, clone=True)
• add_retention_index(spectrum_in, clone=True)
• require_retention_time(spectrum_in, minimum_rt=None, maximum_rt=None, clone=True)
• require_retention_index(spectrum_in, clone=True)
• require_correct_ms_level(spectrum, required_ms_level=2)

Retention time and retention index are not interchangeable. Record units and the chromatographic method before using either as a matching constraint.

Fingerprints

add_fingerprint() still works in 0.33.1, but it is marked for removal in matchms 1.0. Prefer the top-level Fingerprints class:

from matchms import Fingerprints

fingerprints = Fingerprints(
    fingerprint_algorithm="morgan2",
    fingerprint_method="bit",
    nbits=2048,
)
fingerprints.compute_fingerprints(spectra)

Fingerprints maps valid InChIKeys to fingerprints and needs a valid InChIKey plus SMILES or InChI. In 0.33.1, FingerprintSimilarity still reads a "fingerprint" field from each spectrum, so bridge deliberately when that legacy similarity class is needed:

for spectrum in spectra:
    fingerprint = fingerprints.get_fingerprint_by_spectrum(spectrum)
    if fingerprint is not None:
        spectrum.set("fingerprint", fingerprint)

Do not describe fingerprint similarity as spectral similarity.

Custom Filters

A custom filter should accept a spectrum first and return a spectrum or None:

def require_fragment(spectrum_in, fragment_mz, tolerance=0.02):
    if spectrum_in is None:
        return None
    if any(abs(mz - fragment_mz) <= tolerance for mz in spectrum_in.peaks.mz):
        return spectrum_in
    return None


processor.parse_and_add_filter(
    (require_fragment, {"fragment_mz": 184.0733, "tolerance": 0.01})
)

Avoid mutation unless it is intentional and documented. If the custom filter modifies a spectrum, either implement a clone parameter consistently or clone inside the function.

Reproducibility Checklist

• Save processor.processing_steps.
• Record matchms and Python versions.
• Record whether metadata harmonization was enabled at import.
• Apply identical peak filters to query and reference collections.
• Preserve counts before and after every requirement filter.
• Preserve original metadata before repair/enrichment.
• Record network calls and external annotation sources.
• Test a few spectra with expected pass/fail behavior before batch processing.