Polynomial Baseline Removal

Overview

Polynomial Baseline Removal fits a smooth polynomial independently to every pixel spectrum and subtracts the fitted polynomial from that spectrum.

The method is intended to remove broad, slowly varying additive spectral background while preserving narrower residual spectral structure.

baseline(x) = c0 + c1*x + c2*x^2 + ... + cd*x^d where d is the selected polynomial degree.

Important - The Fit Is Per Pixel

Every spatial pixel receives its own polynomial baseline fit.

This is different from subtracting one common baseline from the entire hyperspectral cube.

Therefore, spatial pixels can have different baseline offsets and curvatures.

The GUI allows polynomial degrees 1 through 9.

Degree 1 fits a straight line.

Degree 2 fits a quadratic curve and can represent one broad curvature.

Degree 3 adds additional flexibility.

Higher degrees can follow progressively more complex baseline shapes.

Start with degree 1 or 2 whenever possible.

Increase the degree only when the baseline clearly contains broad curvature that a lower-order model cannot represent.

The lowest degree that adequately models the unwanted baseline is generally the safer choice.

WHY NOT ALWAYS USE A HIGH DEGREE?

A polynomial does not know which spectral structures are background and which are scientifically meaningful.

As polynomial degree increases, the fitted curve becomes more flexible.

A high-order polynomial can begin following genuine broad peaks, absorption bands, shoulders, or other spectral features and subtract them from the data.

Overfitting occurs when the polynomial follows spectral features or noise that should remain in the corrected spectrum.

Signs of overfitting can include:

• Important broad features becoming much smaller or disappearing.
• Artificial oscillations in the corrected spectrum.
• Unexpected positive/negative lobes around broad peaks.
• Strong dependence of results on small changes in polynomial degree.

Underfitting occurs when the polynomial degree is too low to represent the unwanted baseline.

Signs of underfitting can include:

• Residual slope across the corrected spectrum.
• Remaining broad curvature.
• Similar baseline drift still visible across many corrected spectra.

Use degree 1 when the unwanted baseline is approximately a straight offset/slope across the spectral range.

This is the most conservative option and is a good first test.

Use degree 2 when the baseline has one broad curvature that cannot be represented adequately by a straight line.

Degree 2 is often a useful compromise between flexibility and preservation of real spectral structure.

Use degree 3 when the unwanted baseline has clear asymmetric or more complex broad curvature.

Compare carefully against degree 2 to verify that the extra flexibility removes baseline rather than genuine spectral information.

Higher-order models should be used only when there is a clear reason.

They are increasingly capable of fitting broad real spectral features and can produce edge behavior or oscillations.

Do not choose a high degree merely because the corrected spectrum appears flatter.

The polynomial degree must be smaller than the number of spectral bands.

IDCubePro checks this before processing.

The polynomial fit uses a normalized spectral coordinate from -1 to 1 rather than the raw wavelength values.

This improves numerical conditioning, especially for higher polynomial powers.

The normalized coordinate is used only to construct the polynomial basis.

The wavelength vector stored in the IDCubePro dataset is preserved.

Important Implementation Detail

The current polynomial is fitted against evenly spaced spectral INDEX positions from -1 to 1.

It does not use the numerical spacing between physical wavelength values in the regression basis.

For datasets with approximately uniform wavelength sampling this is usually a reasonable computational coordinate.

For strongly nonuniform wavelength spacing, be aware that polynomial curvature is defined versus band index rather than exact physical wavelength.

IDCubePro constructs the polynomial design matrix once.

The complete hyperspectral cube is reshaped into pixels x bands.

All pixel spectra are then fitted simultaneously using matrix operations.

This is substantially more efficient than calling polyfit separately for every pixel.

The polynomial design matrix X has one row per spectral band and one column per polynomial coefficient.

IDCubePro solves the least-squares system for all pixel spectra simultaneously.

The fitted baselines are reconstructed from the polynomial coefficients and subtracted from all spectra.

Before fitting, NaN and Inf values in the reshaped spectral matrix are replaced with zero.

This prevents non-finite values from propagating through the matrix fit.

However, zero replacement is a computational safeguard rather than a scientifically optimal missing-data strategy.

If many bands contain invalid values, investigate and correct or remove those bands before polynomial baseline removal.

When Polynomial Baseline Removal Is A Good Candidate

• Spectra contain broad additive baseline drift.
• The baseline changes from pixel to pixel.
• The unwanted background is reasonably smooth.
• Narrower residual spectral features are the primary information of interest.
• A low-order polynomial provides a defensible model of the background.

Polynomial baseline correction is commonly considered when spectra contain broad instrumental drift, fluorescence-like background, slowly varying offset, or other smooth additive contributions.

Whether it is appropriate depends on the spectroscopy modality and the scientific endpoint.

• Broad spectral features themselves contain important information.
• The spectral range is short.
• There are relatively few bands.
• Strong peaks dominate the least-squares fit.
• The baseline is not polynomial-like.
• Spectra have sharp discontinuities.
• The signal-to-noise ratio is poor.
• Higher polynomial degrees are required to make the spectrum look flat.

Important - This Is An Unconstrained Whole-spectrum Fit

The current implementation fits the polynomial to ALL spectral bands.

It does not automatically identify baseline-only regions.

It does not mask peaks or absorption bands before fitting.

Consequently, strong genuine spectral features influence the polynomial coefficients.

If a spectrum contains a large broad peak, the least-squares polynomial may partially follow that peak.

Subtracting the fitted polynomial can therefore reduce the apparent amplitude or alter the shape of the feature.

Always inspect representative corrected spectra.

Polynomial Baseline Vs Manual Baseline Removal

Manual baseline removal allows the user to select baseline points from a representative background spectrum and interpolates through those points.

Polynomial Baseline Removal instead automatically fits a polynomial to every complete pixel spectrum.

Manual selection provides direct control over which points define the baseline.

Polynomial fitting is automated and spatially adaptive but is more influenced by genuine spectral features because every band contributes to the fit.

Polynomial Baseline Vs Background Roi Subtraction

A background-ROI subtraction method estimates one additive reference from a selected spatial region and applies it broadly.

Polynomial baseline removal estimates a separate smooth baseline for every pixel spectrum.

Polynomial Baseline Vs Background Spectrum Division

Background spectrum correction divides every pixel spectrum by a selected reference spectrum.

Polynomial baseline removal subtracts an independently fitted additive baseline.

Division and subtraction represent different measurement models.

MSC models each spectrum relative to a reference spectrum using an additive intercept and multiplicative slope.

Polynomial baseline removal does not require a reference spectrum and instead models wavelength-dependent additive curvature within each spectrum.

SNV subtracts one scalar spectral mean and divides by one scalar spectral standard deviation for each spectrum.

Polynomial baseline removal subtracts a wavelength-dependent fitted curve and does not standardize the residual by its standard deviation.

Polynomial Baseline Vs Continuum Removal

Continuum removal estimates an upper spectral continuum and divides each spectrum by that continuum, commonly emphasizing absorption depth and shape.

Polynomial baseline removal performs additive subtraction of a least-squares polynomial fitted across the spectrum.

Polynomial Baseline Vs Derivatives

First and second spectral derivatives can suppress broad baseline components while emphasizing changes in spectral shape.

Derivatives also amplify noise and change feature interpretation.

Polynomial subtraction retains a residual spectrum in intensity-like units but depends strongly on the fitted baseline model.

There is no universal preprocessing order.

Physical calibration, invalid-band removal, and correction of major acquisition artifacts should generally be considered before mathematical baseline modeling.

Whether polynomial correction should occur before or after smoothing, normalization, or derivatives depends on the measurement and downstream analysis.

Known detector-edge bands, saturated bands, dead bands, or other severe artifacts can distort the polynomial fit.

When appropriate, remove clearly invalid bands before baseline fitting.

Noise contributes to the least-squares fit, especially at higher polynomial degrees.

Appropriate denoising may improve baseline stability.

However, excessive smoothing can alter narrow real features.

Validate the combination of denoising and baseline correction rather than treating them independently.

Polynomial fits can behave poorly near the boundaries of the fitted spectral interval, especially at higher degrees.

Inspect both ends of the corrected spectrum for artificial excursions or curvature.

Negative residual values are normal after baseline subtraction.

They mean that the measured spectrum is below the fitted baseline at those wavelengths.

Negative residuals are not automatically an error, but their physical interpretation depends on the measurement.

Polynomial baseline subtraction changes absolute spectral intensities.

If broad intensity level or spectral area is itself a quantitative endpoint, baseline removal can alter that information.

Select the desired polynomial degree and click Apply.

IDCubePro validates the dataset and degree before changing the main application status to Processing.

The implementation performs the following steps:

1. Validate the hyperspectral cube.

2. Obtain or construct the wavelength vector.

3. Read the selected polynomial degree.

4. Reshape the cube into pixel spectra.

5. Replace non-finite spectral values with zero.

6. Construct the normalized polynomial design matrix.

7. Fit all spectra simultaneously.

8. Reconstruct all fitted baselines.

10. Reshape the corrected spectra back into a cube.

11. Update the working IDCubePro dataset.

12. Record processing history.

14. Refresh the current GUI.

The progress dialog is cancelable.

IDCubePro checks for cancellation at several points before committing the corrected dataset.

If processing is canceled before the dataset is updated, the corrected cube is not committed.

The application status changes to Processing when numerical computation begins.

Cleanup logic returns the application to Ready after successful completion, cancellation, or handled errors.

The corrected cube replaces myData.Images as the active working dataset.

Rows, columns, spectral bands, and wavelength vector remain unchanged.

The working filtered cube is marked active.

Polynomial Baseline Removal does not intentionally overwrite myDataOriginal.

The broader Reset Preprocessing workflow can therefore restore the preserved original cube when available.

• Normalized fitting coordinate.
• Wavelength minimum and maximum.

Quality Control - Before Processing

Inspect several representative raw spectra.

Determine whether the unwanted component is genuinely broad and smooth.

Choose the lowest plausible polynomial degree.

Quality Control - After Processing

Inspect corrected spectra from multiple spatial regions.

• Reduced broad baseline drift.
• Preservation of important peaks and absorption bands.
• Changes in broad spectral features.

When the correct degree is uncertain, compare degree 1 and degree 2 first.

If degree 2 clearly leaves systematic baseline curvature, test degree 3.

Do not simply continue increasing degree until the spectrum becomes flat.

The goal is to remove a defensible baseline, not all broad variation.

For quantitative workflows, consider defining an objective rule for polynomial degree rather than selecting it by visual preference.

Possible evaluation criteria include residual baseline structure, preservation of known spectral features, reproducibility across samples, and downstream validation performance.

Polynomial baseline removal can prevent broad offset and curvature from dominating principal components.

However, PCA may also use genuine broad spectral variation as meaningful information.

Compare PCA scores and loadings before and after baseline correction.

Baseline correction can help clustering focus on narrower spectral differences rather than broad background variation.

It can also reduce useful class separation if broad spectral shape distinguishes the classes.

For Supervised Machine Learning

If polynomial baseline removal is part of model training, the same degree and preprocessing sequence must be applied consistently to validation and prediction data.

Do not optimize polynomial degree on the final test dataset.

Baseline correction can improve regression when nuisance background variation obscures analyte-related features.

It can worsen regression when the polynomial removes signal correlated with concentration or another quantitative endpoint.

Use independent validation error to evaluate the preprocessing choice.

For Biological And Medical Data

Broad spectral variation can arise from tissue scattering, fluorescence, blood content, water, lipids, instrument response, or genuine biological composition.

Do not assume that every broad feature is unwanted baseline.

The appropriate polynomial degree should reflect the measurement physics and the endpoint.

Polynomial fitting is sometimes used to model broad fluorescence background beneath narrower spectral features.

However, if the fluorescence spectrum itself is the biological or chemical signal of interest, subtracting it as baseline would remove meaningful information.

Polynomial baseline subtraction is commonly used to reduce broad fluorescence background beneath narrower Raman features.

High-order fits can distort broad Raman bands or introduce artifacts, so low-order models and visual/quantitative validation remain important.

Broad reflectance curvature can contain important material information.

Polynomial baseline subtraction should therefore not be applied automatically to reflectance spectra merely to flatten them.

Continuum removal, derivatives, SNV, MSC, or no baseline correction may be more appropriate depending on the analytical goal.

Common Problem - Important Peak Is Reduced

The polynomial is probably fitting part of the genuine spectral feature.

Try a lower polynomial degree or consider a baseline method that allows explicit baseline regions or anchor points.

Common Problem - Baseline Remains

The selected degree may be too low, or the baseline may not be well described by a polynomial.

Increase the degree cautiously or consider another baseline model.

Common Problem - Oscillations Appear

The polynomial degree may be unnecessarily high.

Reduce the degree and inspect spectral-edge behavior.

Common Problem - Edge Artifacts

Polynomial extrapolation behavior within the ends of the fitted interval can become unstable with flexible high-order models.

Use a lower degree, remove unreliable edge bands, or consider a constrained baseline method.

Common Problem - Different Regions Need Different Degrees

The current implementation uses one selected polynomial degree for all pixels, although each pixel receives its own fitted coefficients.

If spectral baseline complexity differs greatly among materials, one global degree may not be ideal.

Common Problem - Many Invalid Bands

Because non-finite values are replaced with zero before fitting, a large number of invalid bands can strongly bias the polynomial.

Remove or repair invalid bands before baseline fitting whenever possible.

• Spectral wavelength range.
• Denoising or smoothing applied beforehand.
• Other preprocessing applied before and after baseline correction.
• Whether the same degree was used for every dataset.

1. Load and inspect the hyperspectral cube.

2. Perform necessary physical calibration.

3. Remove clearly invalid bands.

4. Inspect representative spectra.

5. Confirm that the unwanted component is broad, smooth, and additive.

6. Start with polynomial degree 1.

7. Apply and inspect corrected spectra.

8. If broad curvature remains, return to the original data and test degree 2.

9. Increase degree only when lower-order fits demonstrably underfit the baseline.

10. Check peaks, broad features, spectral edges, and several spatial regions.

11. Validate downstream PCA, clustering, classification, or regression.

12. Use the lowest degree that provides an adequate and scientifically defensible correction.

The purpose of polynomial baseline removal is not to make every spectrum flat.

The purpose is to estimate and subtract a scientifically defensible smooth additive background while preserving the spectral information relevant to the experiment.

Close Polynomial Baseline Removal help.