Custom Python Analysis
Run custom Python algorithms directly on the hyperspectral dataset currently loaded in IDCubePro.
Overview
The Custom Python Analysis workspace connects IDCubePro to an external Python installation and allows Python code to operate directly on the current hyperspectral cube.
Python can be used for numerical analysis, image processing, machine learning, statistics, dimensionality reduction, segmentation, visualization, and custom research workflows.
IDCubePro automatically provides the current dataset to Python and can display Python-generated results inside IDCubePro. A processed cube can also be transferred back into IDCubePro as the new working dataset.
Data Available to Python
The following variables are automatically available to Python scripts:
- cube — hyperspectral data arranged as rows × columns × spectral bands.
- wavelengths — one-dimensional spectral wavelength vector.
- metadata — dataset metadata made available to the Python workspace.
Python Editor and Runtime Information
The Python editor includes line numbers so traceback messages can be matched directly to the code shown in the editor. After execution, IDCubePro can also report Python execution time and total processing time.
This is especially useful when developing longer analysis scripts or comparing alternative algorithms on large hyperspectral datasets.
Python Environment
IDCubePro uses an external Python installation on the computer. When the Python Analysis Center opens, IDCubePro detects the active interpreter and checks the scientific packages used by the Python bridge.
The environment result is cached during the current IDCubePro session so reopening the Python Analysis Center is fast.
Use Check Python to force a fresh environment scan, especially after installing, removing, or changing Python packages.
Python Packages
Core packages:
- NumPy
- SciPy
Standard scientific packages:
- Matplotlib
- pandas
- scikit-learn
- scikit-image
Optional extended packages may include:
- OpenCV
- tifffile
- h5py
- openpyxl
- nibabel
- statsmodels
If a required or recommended package is missing or cannot be imported, IDCubePro can offer to install or repair it using the exact Python interpreter detected by IDCubePro.
Python Analysis Center Controls
- Open .py File — load an existing Python script into the editor.
- Clear — clear the Python code editor.
- Preview — run the script and inspect returned results before changing the active IDCubePro dataset.
- Return to IDCubePro — after preview, transfer a validated processed cube and its spectral axis back into IDCubePro.
- Close — close the Python Analysis Center.
- Help — open the built-in Python Analysis help window.
- Check Python — re-detect Python and re-test the configured package set.
- Install Packages — install or repair missing supported Python packages.
- Run Analysis — execute the current Python code using the loaded IDCubePro dataset.
Basic Workflow
1. Load a hyperspectral dataset in IDCubePro.
2. Open the Custom Python Analysis workspace.
3. Allow IDCubePro to detect Python and check the scientific environment.
4. Install or repair packages if IDCubePro reports that supported packages are missing.
5. Type Python code directly into the editor or open an existing .py file.
6. Click Run Analysis to execute the current script.
7. Inspect the returned result or datacube in the Preview window.
8. For a returned datacube, use the interactive band slider to inspect the result before accepting it.
9. Click Return to IDCubePro only when you want the validated output cube to become the active working dataset.
Returning a Simple Result
For one simple numeric, image, or spectral result, create a variable named result.
import numpy as np
result = np.mean(cube, axis=2)
A one-dimensional result whose length matches the wavelength vector can be displayed automatically as a spectrum using the wavelength axis.
Returning Multiple Results
For more control, create an outputs list. Each item is a Python dictionary describing one result.
import numpy as np
mean_spectrum = np.mean(cube, axis=(0, 1))
outputs = [
{
"type": "spectrum",
"title": "Mean Spectrum",
"data": mean_spectrum,
"x": wavelengths,
"xlabel": "Wavelength",
"ylabel": "Intensity"
}
]
Supported result types include image, rgb, spectrum, spectra, scatter, scatter3, surface, volume, isosurface, labelvolume, bar, histogram, table, datacube, and auto.
Preview Before Returning a Datacube
When a script creates output_cube, IDCubePro can preview the proposed returned dataset without immediately replacing the active working cube.
The Preview window can report the returned variable, result type, dimensions, numeric class, memory use, wavelength information, and execution timing. Three-dimensional datacubes are displayed with an interactive band slider so individual bands can be inspected before the dataset is accepted.
The active IDCubePro dataset is changed only after Return to IDCubePro is selected. Closing or cancelling the preview leaves the current dataset unchanged.
Returning a Processed Cube to IDCubePro
When Python creates a processed dataset, IDCubePro can preview the returned cube before it replaces the current working dataset. Use Return to IDCubePro only after inspecting and accepting the preview.
The preferred Python variables are:
output_cube = ...
output_wavelengths = ...
The returned cube must be three-dimensional and arranged as rows × columns × bands. The number of values in output_wavelengths must equal the number of bands in output_cube.
If output_cube or output_wavelengths are not explicitly created, IDCubePro can use the current Python variables cube and wavelengths as the returned dataset.
The returned data replace the current working dataset but do not replace the original source dataset stored by IDCubePro.
Datacube Result Viewer
Use the datacube result type when you want to inspect a three-dimensional hyperspectral result in the Python Results window.
outputs = [
{
"type": "datacube",
"data": output_cube,
"meta": {
"title": "Processed Datacube",
"wavelengths": output_wavelengths
}
}
]
The viewer supports interactive band browsing and uses wavelength metadata when the wavelength vector length matches the number of bands. If the third dimension represents components, clusters, or other transformed features instead of physical wavelengths, use component or band indices rather than reusing the original wavelength vector.
Example: PCA Cube
The following example performs PCA and returns the first 20 principal component score images as a new cube.
import numpy as np
from sklearn.decomposition import PCA
rows, cols, bands = cube.shape
X = cube.reshape(-1, bands).astype(float)
valid = np.all(np.isfinite(X), axis=1)
X_valid = X[valid]
n_components = min(20, bands)
pca = PCA(n_components=n_components)
scores_valid = pca.fit_transform(X_valid)
score_cube = np.full(
(rows * cols, n_components),
np.nan,
dtype=float
)
score_cube[valid, :] = scores_valid
output_cube = score_cube.reshape(
rows,
cols,
n_components
)
output_wavelengths = np.arange(
1,
n_components + 1,
dtype=float
)
outputs = [
{
"type": "image",
"title": "PCA Component 1",
"data": output_cube[:, :, 0]
}
]
For a PCA cube, the returned third-axis values represent component numbers rather than physical wavelengths.
Example: scikit-image Gaussian Smoothing
import numpy as np
from skimage.filters import gaussian
mean_image = np.mean(cube, axis=2)
smoothed = gaussian(
mean_image,
sigma=2
)
outputs = [
{
"type": "image",
"title": "Gaussian Smoothed",
"data": smoothed
}
]
Python Script Templates
The following templates can be copied directly into the IDCubePro Python editor. They are intended both as starting points for new analyses and as working examples of the IDCubePro input/output conventions.
Starter Template
Use this template when creating a new Python workflow. It documents the cube, wavelengths, and metadata variables provided by IDCubePro and shows the supported patterns for returning simple results, typed outputs, and datacubes.
Show starter template
"""
IDCubePro Python Example / Test Template
========================================
Example name:
<Short descriptive name>
Purpose
-------
Describe what this script does and why it is useful.
Example:
This script performs <analysis> on the active IDCubePro datacube
and returns <result>.
IDCubePro automatically provides
-------------------------------
cube
NumPy array with shape:
(rows, columns, spectral_bands)
wavelengths
One-dimensional NumPy array containing the spectral wavelength
associated with each band.
metadata
Dataset metadata supplied by IDCubePro, when available.
Expected behavior
-----------------
1. <Step 1>
2. <Step 2>
3. <Step 3>
4. <Describe what should appear in the Python Results window>
5. <Describe whether a new cube should be returned to IDCubePro>
Expected output
---------------
Input example:
cube.shape = (rows, columns, bands)
Expected result:
<Describe output type and dimensions>
If returning a new cube:
output_cube.shape = (rows, columns, new_bands)
and:
len(output_wavelengths) == output_cube.shape[2]
IDCubePro output conventions
----------------------------
Use "result" for a simple single result.
Example:
result = some_array
Use "outputs" for one or more explicitly typed results.
Example:
outputs = [
{
"type": "image",
"data": image,
"meta": {
"title": "Example Image"
}
}
]
Use the following variables when the processed datacube should be
returned to the active IDCubePro working dataset:
output_cube
output_wavelengths
The user must enable:
"Return cube to IDCubePro"
Notes
-----
- Preserve physical wavelengths only when the returned third dimension
still represents original spectral bands.
- PCA components, clusters, features, or other transformed dimensions
should not be presented as physical wavelengths unless scientifically
appropriate.
- Keep the script self-contained whenever possible.
- Print useful information so the user can verify dimensions and results.
"""
# =========================================================================
# IMPORTS
# =========================================================================
import numpy as np
# Add other packages here if needed.
#
# Example:
# import matplotlib.pyplot as plt
# from sklearn.decomposition import PCA
# =========================================================================
# USER SETTINGS
# =========================================================================
# Put easy-to-change parameters here.
#
# Example:
# n_components = 20
# threshold = 0.5
# =========================================================================
# VALIDATE INPUT
# =========================================================================
if cube.ndim != 3:
raise ValueError(
"Expected cube with shape (rows, columns, spectral_bands)."
)
rows, cols, bands = cube.shape
wavelengths = np.asarray(wavelengths).reshape(-1)
if len(wavelengths) != bands:
raise ValueError(
"The number of wavelength values must match cube.shape[2]."
)
print("Input cube shape:", cube.shape)
print("Number of wavelengths:", len(wavelengths))
# =========================================================================
# ANALYSIS
# =========================================================================
# -------------------------------------------------------------------------
# PUT YOUR ANALYSIS CODE HERE
# -------------------------------------------------------------------------
#
# Example:
#
# mean_image = np.mean(cube, axis=2)
#
# or:
#
# selected_bands = np.arange(0, bands, 2)
# new_cube = cube[:, :, selected_bands]
# new_wavelengths = wavelengths[selected_bands]
# =========================================================================
# OPTIONAL: PRINT / CHECK RESULTS
# =========================================================================
# Example:
#
# print("Output shape:", new_cube.shape)
# print("Output wavelengths:", new_wavelengths)
# =========================================================================
# OPTIONAL: RETURN A NEW DATACUBE TO IDCUBEPRO
# =========================================================================
# Uncomment and edit these lines when your analysis creates a new datacube.
#
# output_cube = new_cube
# output_wavelengths = new_wavelengths
# =========================================================================
# RETURN RESULT(S) TO IDCUBEPRO RESULTS WINDOW
# =========================================================================
# Choose ONE of the patterns below.
# -------------------------------------------------------------------------
# OPTION A: SIMPLE RESULT
# -------------------------------------------------------------------------
#
# result = mean_image
# -------------------------------------------------------------------------
# OPTION B: TYPED OUTPUT
# -------------------------------------------------------------------------
#
# outputs = [
# {
# "type": "image",
# "data": mean_image,
# "meta": {
# "title": "Mean Spectral Image"
# }
# }
# ]
# -------------------------------------------------------------------------
# OPTION C: DATACUBE OUTPUT
# -------------------------------------------------------------------------
#
# outputs = [
# {
# "type": "datacube",
# "data": new_cube,
# "meta": {
# "title": "Processed Datacube",
# "wavelengths": new_wavelengths
# }
# }
# ]
# =========================================================================
# REQUIRED
# =========================================================================
#
# Before running the script, make sure that at least one of the following
# is created:
#
# result
#
# or:
#
# outputs
#
# Otherwise IDCubePro will report that the Python script did not create
# a result.
Example: Random 20-Band Datacube
This working example randomly selects 20 spectral bands, preserves their original spectral order and wavelengths, displays the reduced cube as a datacube result, and makes it available for return to IDCubePro.
Show random 20-band example
"""
IDCubePro Python Example
========================
Example: Randomly select 20 spectral bands, build a new datacube,
preserve the corresponding wavelengths, display the result in the
Python Results window, and optionally return the new cube to IDCubePro.
Purpose
-------
This script is intended as a working example and regression test for the
IDCubePro Custom Python Analysis Center.
IDCubePro provides these variables automatically:
cube
NumPy array with shape:
(rows, columns, spectral_bands)
wavelengths
One-dimensional NumPy array containing the wavelength associated
with each spectral band.
Expected behavior
-----------------
1. The script randomly selects 20 spectral bands without replacement.
2. The selected bands are sorted so that the original spectral order is
preserved.
3. A new datacube is created with shape:
rows x columns x 20
4. The corresponding wavelength values are preserved.
5. The new datacube is shown in the IDCubePro Python Results window as
a "datacube" output.
6. If "Return cube to IDCubePro" is enabled in the Python Analysis Center,
the new cube becomes the active working cube in IDCubePro.
Expected output
---------------
For an original cube of shape, for example:
207 x 264 x 500
the returned cube should have shape:
207 x 264 x 20
The wavelength vector should contain exactly 20 values and:
len(output_wavelengths) == output_cube.shape[2]
Important IDCubePro variables
-----------------------------
outputs
Controls what is displayed in the Python Analysis Results window.
output_cube
output_wavelengths
Used by the "Return cube to IDCubePro" option to transfer a processed
cube back into the active IDCubePro working dataset.
Notes
-----
- This example preserves physical wavelengths because the returned bands
are original spectral bands.
- For PCA or other transformed data, the third dimension may no longer
represent physical wavelength and should be labeled appropriately.
- The custom "datacube" result type must be supported by renderOutput.m
in IDCubePro for the interactive datacube viewer to appear.
"""
import numpy as np
# -------------------------------------------------------------------------
# USER SETTINGS
# -------------------------------------------------------------------------
# Number of spectral bands to randomly select
n = 20
# -------------------------------------------------------------------------
# VALIDATE INPUT
# -------------------------------------------------------------------------
if cube.ndim != 3:
raise ValueError(
"Expected cube with shape (rows, columns, spectral_bands)."
)
rows, cols, bands = cube.shape
wavelengths = np.asarray(wavelengths)
if wavelengths.ndim != 1:
wavelengths = wavelengths.reshape(-1)
if len(wavelengths) != bands:
raise ValueError(
"The number of wavelength values must match cube.shape[2]."
)
n = min(n, bands)
# -------------------------------------------------------------------------
# RANDOMLY SELECT BANDS
# -------------------------------------------------------------------------
selected_bands = np.sort(
np.random.choice(
bands,
size=n,
replace=False
)
)
# -------------------------------------------------------------------------
# CREATE NEW DATACUBE
# -------------------------------------------------------------------------
new_cube = cube[:, :, selected_bands]
new_wavelengths = wavelengths[selected_bands]
# -------------------------------------------------------------------------
# REPORT RESULTS
# -------------------------------------------------------------------------
print("IDCubePro random-band datacube test")
print("-----------------------------------")
print("Original cube shape:", cube.shape)
print("New cube shape:", new_cube.shape)
print("\nSelected band indices:")
print(selected_bands)
print("\nSelected wavelengths:")
print(new_wavelengths)
# -------------------------------------------------------------------------
# RETURN PROCESSED CUBE TO IDCUBEPRO
#
# These variables are used when the user enables:
# "Return cube to IDCubePro"
# -------------------------------------------------------------------------
output_cube = new_cube
output_wavelengths = new_wavelengths
# -------------------------------------------------------------------------
# DISPLAY RESULT IN IDCUBEPRO PYTHON RESULTS WINDOW
#
# Requires IDCubePro renderOutput.m to support:
# type = "datacube"
# -------------------------------------------------------------------------
outputs = [
{
"type": "datacube",
"data": new_cube,
"meta": {
"title": "20 Random Spectral Bands",
"wavelengths": new_wavelengths
}
}
]
Additional templates can be added here for PCA, clustering, spectral extraction, image segmentation, spectral matching, and other common workflows.
Processing History
When a Python-generated cube is transferred back into IDCubePro, IDCubePro records the operation in the processing history.
The history entry can include the dataset dimensions and band counts before and after the Python operation, wavelength ranges, and whether the data were changed.
Matplotlib
Matplotlib can be used to create external Python figures.
Be aware that plt.show() normally pauses Python execution until the Matplotlib window is closed. IDCubePro therefore cannot finish receiving Python results until that external figure is closed.
For integrated IDCubePro results, return result or outputs instead of relying on plt.show().
Package Name Differences
Some Python import names differ from their package installation names:
- sklearn → scikit-learn
- skimage → scikit-image
- cv2 → opencv-python
Troubleshooting
ModuleNotFoundError means that a requested Python package is missing or cannot be imported by the Python interpreter used by IDCubePro.
Use Install Packages to install or repair supported packages, then use Check Python if a fresh environment scan is needed.
If a returned cube is rejected, confirm that it is a numeric 3D array and that the returned wavelength or component vector has exactly one value for each band.
If wavelength metadata does not match the number of returned bands, IDCubePro should use band or component indices rather than applying an inconsistent wavelength axis.
Only run Python code from sources you trust.