Full Neural Operators Demo¶
| Metadata | Value |
|---|---|
| Level | Advanced |
| Runtime | ~10 min (CPU/GPU) |
| Prerequisites | JAX, Flax NNX, Neural Operators |
| Format | Python + Jupyter |
Overview¶
This example demonstrates every neural operator variant available in the Opifex framework, from parameter-efficient Tensorized FNOs to geometry-aware and uncertainty-quantifying architectures. It walks through the operator factory system, compares parameter counts across FNO variants, runs domain-specific forward passes (turbulence, climate, molecular dynamics, airfoil geometry), and builds a multi-operator ensemble with agreement scoring.
Unlike single-model tutorials, this demo is designed as a full tour of the
opifex.neural.operators module family. Each section creates a different operator,
feeds it synthetic domain-appropriate data, and reports timing and output statistics.
The demo is console-only (no visualization files) and produces benchmark numbers you
can use to guide operator selection for your own problems.
What You'll Learn¶
- Use the operator factory to create any neural operator with
create_operator()and get domain recommendations withrecommend_operator() - Compare parameter efficiency across Standard FNO, Tucker TFNO, CP TFNO, and U-FNO
- Run U-FNO for multi-scale turbulent flow simulation with 4 encoder-decoder levels
- Run SFNO for global climate modeling with spherical harmonics and power spectrum analysis
- Quantify uncertainty with UQNO, decomposing predictions into epistemic and aleatoric components
- Model complex geometry with GINO using geometry attention on airfoil-like domains
- Predict molecular forces with MGNO using multipole graph interactions
- Build a multi-operator ensemble from FNO, TFNO, and LocalFNO with agreement scoring
Coming from NeuralOperator (PyTorch)?¶
If you are familiar with the neuraloperator library, here is how Opifex compares for this workflow:
| NeuralOperator (PyTorch) | Opifex (JAX) |
|---|---|
TFNO(n_modes, hidden_channels, factorization) |
TensorizedFourierNeuralOperator(modes=, hidden_channels=, factorization=, rank=, rngs=) |
FNO(n_modes, hidden_channels) |
FourierNeuralOperator(modes=, hidden_channels=, num_layers=, rngs=) |
UNO(...) with manual U-Net wiring |
UFourierNeuralOperator(modes=, hidden_channels=, num_levels=, rngs=) |
| No built-in operator factory | create_operator("SFNO", ...) and recommend_operator("global_climate") |
| Manual ensemble loop | Same pattern, but with JAX JIT for each operator |
torch.optim.Adam(model.parameters(), lr) |
optax.adam(lr) (handled internally by Trainer) |
Key differences:
- Explicit PRNG: Opifex uses JAX's explicit
rngs=nnx.Rngs(42)instead of global random state - Factory system:
create_operator()andrecommend_operator()provide guided operator selection not available in the PyTorch library - XLA compilation: All forward passes are JIT-compiled automatically for hardware acceleration
- Functional transforms:
jax.grad,jax.vmap,jax.pmapcompose cleanly with every operator variant
Files¶
- Python Script:
examples/neural-operators/operator_tour.py - Jupyter Notebook:
examples/neural-operators/operator_tour.ipynb
Quick Start¶
Run the Python Script¶
Run the Jupyter Notebook¶
Core Concepts¶
The Neural Operator Family¶
Opifex provides a unified framework for eight neural operator architectures, each
designed for a specific class of problems. All operators share the same interface
pattern -- (in_channels, out_channels, hidden_channels, rngs) -- and can be
created through the factory system or instantiated directly.
| Operator | Full Name | Best For |
|---|---|---|
| TFNO | Tensorized Fourier Neural Operator | Parameter-efficient modeling; memory-constrained settings |
| U-FNO | U-Net Fourier Neural Operator | Multi-scale turbulent flow; problems with features at multiple resolutions |
| SFNO | Spherical Fourier Neural Operator | Global climate modeling; data on spherical domains (lat/lon grids) |
| GINO | Geometry-Informed Neural Operator | Complex geometries (airfoils, CAD shapes); irregular domains |
| MGNO | Multipole Graph Neural Operator | Molecular dynamics; particle systems with long-range interactions |
| UQNO | Uncertainty Quantification Neural Operator | Safety-critical applications; Bayesian uncertainty decomposition |
| LocalFNO | Local Fourier Neural Operator | Wave propagation; problems needing both local and global operations |
| AM-FNO | Amortized Fourier Neural Operator | High-frequency problems; neural kernel networks |
Operator Factory and Recommendation System¶
The factory system provides two entry points:
list_operators()-- returns all available operators grouped by categoryrecommend_operator(application)-- suggests the best operator for a given application domaincreate_operator(name, **kwargs)-- instantiates any operator by name with the given configuration
This allows you to select operators programmatically rather than hard-coding architecture choices.
Implementation¶
Step 1: Imports and Setup¶
import time
from typing import Any
import jax
import jax.numpy as jnp
from flax import nnx
# Import all neural operators
from opifex.neural.operators import (
AmortizedFourierNeuralOperator,
create_operator,
FourierNeuralOperator,
GeometryInformedNeuralOperator,
list_operators,
LocalFourierNeuralOperator,
MultipoleGraphNeuralOperator,
recommend_operator,
SphericalFourierNeuralOperator,
TensorizedFourierNeuralOperator,
UFourierNeuralOperator,
UncertaintyQuantificationNeuralOperator,
)
Terminal Output:
Opifex Neural Operators Full Demo
============================================================
Starting Full Neural Operators Demo
Estimated time: ~3-5 minutes
Step 2: Operator Factory Demo¶
The factory system lists all available operators by category and provides application-specific recommendations. This is the recommended way to discover which operator to use for your problem.
# Show available operators
categories = list_operators()
for category, operators in categories.items():
print(f" {category}: {', '.join(operators)}")
# Get recommendations for specific applications
applications = [
"turbulent_flow", "global_climate", "molecular_dynamics",
"cad_geometry", "safety_critical", "parameter_efficient",
]
for app in applications:
rec = recommend_operator(app)
print(f" {app}: {rec['primary']} - {rec['reason']}")
# Create operators using factory
tfno = create_operator(
"TFNO",
in_channels=3, out_channels=1, hidden_channels=64,
modes=(16, 16), factorization="tucker", rank=0.1, rngs=rngs,
)
uqno = create_operator(
"UQNO",
in_channels=2, out_channels=1, hidden_channels=32,
modes=(8, 8), use_aleatoric=True, rngs=rngs,
)
Terminal Output:
============================================================
NEURAL OPERATOR FACTORY DEMO
============================================================
Available Operators:
fourier_operators: FNO, TFNO, UFNO, SFNO, LocalFNO, AM-FNO
deeponet_family: DeepONet, FourierDeepONet, AdaptiveDeepONet
graph_operators: GNO, MGNO
uncertainty_aware: UQNO
geometry_aware: GINO, GNO, MGNO
parameter_efficient: TFNO, LNO
Application Recommendations:
turbulent_flow: UFNO - Multi-scale encoder-decoder for turbulent structures
global_climate: SFNO - Spherical harmonics for global atmospheric modeling
molecular_dynamics: MGNO - Multipole expansion for long-range molecular interactions
cad_geometry: GINO - Geometry-aware processing for complex CAD shapes
safety_critical: UQNO - Uncertainty quantification for safety-critical decisions
parameter_efficient: TFNO - Tensor factorization for memory efficiency
Creating Operators with Factory:
TFNO created: TensorizedFourierNeuralOperator
UQNO created: UncertaintyQuantificationNeuralOperator
Step 3: Parameter Efficiency Comparison¶
Compare parameter counts across FNO variants to understand the memory trade-offs between standard and tensorized architectures.
operators = {}
# Standard FNO (1D modes)
operators["Standard FNO"] = FourierNeuralOperator(
in_channels=3, out_channels=1, hidden_channels=64,
modes=16, num_layers=4, rngs=rngs,
)
# Tucker TFNO (2D modes)
operators["Tucker TFNO (10%)"] = TensorizedFourierNeuralOperator(
in_channels=3, out_channels=1, hidden_channels=64,
modes=(16, 16), num_layers=4, factorization="tucker", rank=0.1, rngs=rngs,
)
# CP TFNO (2D modes)
operators["CP TFNO"] = TensorizedFourierNeuralOperator(
in_channels=3, out_channels=1, hidden_channels=64,
modes=(16, 16), num_layers=4, factorization="cp", rank=16.0, rngs=rngs,
)
# U-FNO (2D modes)
operators["U-FNO (3 levels)"] = UFourierNeuralOperator(
in_channels=3, out_channels=1, hidden_channels=64,
modes=(16, 16), num_levels=3, rngs=rngs,
)
# Count parameters
for name, op in operators.items():
count = sum(
p.size for p in jax.tree_util.tree_leaves(nnx.state(op))
if hasattr(p, "size")
)
Terminal Output:
============================================================
PARAMETER EFFICIENCY COMPARISON
============================================================
Parameter Counts:
Standard FNO : 279,105 params (compression: 1.0x)
Tucker TFNO (10%) : 4,194,625 params (compression: 0.1x)
CP TFNO : 4,194,625 params (compression: 0.1x)
U-FNO (3 levels) : 27,977,601 params (compression: 0.0x)
Parameter Counts in This Demo
The TFNO and U-FNO variants show larger parameter counts than the baseline
Standard FNO because the Standard FNO uses scalar (1D) modes while the TFNO
and U-FNO use 2D mode tuples (16, 16), which increases the spectral weight
tensor dimensions. In matched configurations (same spatial dimensionality),
TFNO with Tucker factorization typically achieves 90%+ parameter reduction.
Step 4: Multi-Scale Turbulence with U-FNO¶
U-FNO uses a U-Net-style encoder-decoder with multiple resolution levels, making it ideal for turbulent flow where features span many spatial scales.
# Create U-FNO for turbulence (u, v, pressure)
ufno = UFourierNeuralOperator(
in_channels=3, out_channels=3, hidden_channels=64,
modes=(32, 32), num_levels=4, rngs=rngs,
)
# Generate synthetic turbulent flow data (batch=4, channels=3, 64x64)
flows = jnp.stack([create_turbulent_flow(key, size=64) for key in keys])
# Forward pass
predictions = ufno(flows)
Terminal Output:
============================================================
MULTI-SCALE TURBULENCE WITH U-FNO
============================================================
Generating turbulent flow data...
U-FNO created with 4 levels
Running U-FNO forward pass...
Forward pass: (4, 3, 64, 64) -> (4, 3, 64, 64)
Time: 6826.56ms
Multi-scale U-FNO output analysis:
Input resolution: (64, 64) spatial
Output resolution: (64, 64) spatial
Multi-scale levels: 4
Step 5: Global Climate Modeling with SFNO¶
SFNO uses spherical harmonics instead of standard Fourier modes, preserving the geometry of the sphere for global atmospheric data on latitude-longitude grids.
# Create SFNO for climate (T, P, humidity, u_wind, v_wind)
sfno = SphericalFourierNeuralOperator(
in_channels=5, out_channels=5, hidden_channels=128,
lmax=16, num_layers=6, rngs=rngs,
)
# Generate synthetic global climate data (batch=2, channels=5, 32 lat x 64 lon)
climate_data = jnp.stack(
[create_climate_data(key, nlat=32, nlon=64) for key in keys]
)
# Forward pass and spectrum analysis
climate_prediction = sfno(climate_data)
spectrum = sfno.compute_power_spectrum(climate_data[:1])
Terminal Output:
============================================================
GLOBAL CLIMATE MODELING WITH SFNO
============================================================
Generating global climate data...
SFNO created with lmax=16
Running SFNO forward pass...
Forward pass: (2, 5, 32, 64) -> (2, 5, 32, 64)
Time: 826.79ms
Spherical harmonic spectrum: (1, 128, 17)
Step 6: Uncertainty Quantification with UQNO¶
UQNO provides Bayesian inference with decomposed uncertainty estimates. It separates epistemic uncertainty (model knowledge gaps) from aleatoric uncertainty (inherent data noise).
# Create UQNO with aleatoric uncertainty
uqno = UncertaintyQuantificationNeuralOperator(
in_channels=2, out_channels=1, hidden_channels=64,
modes=(16, 16), num_layers=4, use_aleatoric=True, rngs=rngs,
)
# Get uncertainty predictions with 50 Monte Carlo samples
x = jax.random.normal(rng_key, (2, 32, 32, 2))
uncertainty_results = uqno.predict_with_uncertainty(
x, num_samples=50, key=rng_key
)
mean_pred = uncertainty_results["mean"]
epistemic_std = uncertainty_results["epistemic_uncertainty"]
total_std = uncertainty_results["total_uncertainty"]
aleatoric_std = uncertainty_results["aleatoric_uncertainty"]
Terminal Output:
============================================================
UNCERTAINTY QUANTIFICATION WITH UQNO
============================================================
Generating uncertain data...
UQNO created with Bayesian inference
Computing uncertainty estimates...
Uncertainty prediction complete
Time: 3289.85ms
Mean prediction: (2, 32, 32, 1)
Epistemic uncertainty: 0.0000 +/- 0.0000
Total uncertainty: 0.6311 +/- 0.0000
Epistemic uncertainty ratio: 0.000
Aleatoric uncertainty ratio: 1.000
Uncertainty Decomposition
In this demo the epistemic uncertainty is near zero because the model has not been trained -- all weight samples produce the same output. The aleatoric ratio of 1.000 indicates that all measured uncertainty comes from the learned noise model. After training on real data, the epistemic component will reflect genuine model uncertainty about regions with insufficient training coverage.
Step 7: Geometry-Aware Modeling with GINO¶
GINO integrates geometry information through latent attention, enabling it to handle irregular domains like airfoils, turbine blades, and CAD shapes.
# Create GINO with geometry attention
gino = GeometryInformedNeuralOperator(
in_channels=2, out_channels=2, hidden_channels=64,
modes=(12, 12), coord_dim=2, geometry_dim=48,
num_layers=4, use_geometry_attention=True, rngs=rngs,
)
# Forward pass with geometry coordinates
# coords_reshaped: (batch, height*width, 2)
geometry_prediction = gino(flows, geometry_data={"coords": coords_reshaped})
# Test geometry invariance by scaling coordinates
coords_rotated = coords * 1.5
prediction_rotated = gino(flows, geometry_data={"coords": coords_rotated_reshaped})
geometry_sensitivity = jnp.mean(jnp.abs(geometry_prediction - prediction_rotated))
Terminal Output:
============================================================
GEOMETRY-AWARE MODELING WITH GINO
============================================================
Generating airfoil geometry and flow...
GINO created with geometry attention
Running GINO with geometry integration...
Geometry-aware prediction: (2, 64, 64, 2) -> (2, 64, 64, 2)
Time: 2473.02ms
Coordinate input: (2, 64, 64, 2)
Geometry sensitivity: 0.000000
Step 8: Molecular Dynamics with MGNO¶
MGNO uses multipole graph interactions for efficient long-range force computation in molecular systems, similar to fast multipole methods in classical simulation.
# Create MGNO with multipole expansion
mgno = MultipoleGraphNeuralOperator(
in_features=4, out_features=3, hidden_features=64,
num_layers=4, max_degree=3, rngs=rngs,
)
# Predict forces from atomic features and positions
# features: (batch=2, atoms=48, features=4)
# positions: (batch=2, atoms=48, xyz=3)
forces = mgno(features, positions)
Terminal Output:
============================================================
MOLECULAR DYNAMICS WITH MGNO
============================================================
Generating molecular system...
MGNO created with multipole expansion
Computing molecular forces...
Force prediction: (2, 48, 4) + (2, 48, 3) -> (2, 48, 3)
Time: 2658.08ms
Force statistics:
Mean force magnitude: 2.3669
Max force magnitude: 2.6254
Force conservation error: 112.030533
Step 9: Multi-Operator Ensemble¶
Build an ensemble of FNO, TFNO, and LocalFNO to get prediction consensus and uncertainty through inter-model disagreement.
# Create ensemble
ensemble = {
"FNO": FourierNeuralOperator(
in_channels=2, out_channels=1, hidden_channels=48,
modes=16, num_layers=3, rngs=rngs,
),
"TFNO": TensorizedFourierNeuralOperator(
in_channels=2, out_channels=1, hidden_channels=48,
modes=(16, 16), num_layers=3, factorization="tucker", rank=0.2, rngs=rngs,
),
"LocalFNO": LocalFourierNeuralOperator(
in_channels=2, out_channels=1, hidden_channels=48,
modes=(16, 16), num_layers=3, rngs=rngs,
),
}
# Run ensemble predictions
x = jax.random.normal(rng_key, (4, 2, 32, 32))
predictions = {name: op(x) for name, op in ensemble.items()}
# Compute ensemble statistics
pred_stack = jnp.stack(list(predictions.values()))
ensemble_mean = jnp.mean(pred_stack, axis=0)
ensemble_std = jnp.std(pred_stack, axis=0)
agreement_score = 1.0 / (1.0 + jnp.mean(ensemble_std))
Terminal Output:
============================================================
ENSEMBLE OF NEURAL OPERATORS
============================================================
Created ensemble with 3 operators
Running ensemble predictions...
FNO : (4, 1, 32, 32) in 1057.42ms
TFNO : (4, 1, 32, 32) in 526.24ms
LocalFNO : (4, 1, 32, 32) in 1049.14ms
Ensemble Statistics:
Mean prediction: (4, 1, 32, 32)
Prediction std: 0.594862
Agreement score: 0.627
Performance Comparison:
FNO : 1057.42ms
TFNO : 526.24ms
LocalFNO : 1049.14ms
Results Summary¶
Terminal Output:
============================================================
FULL DEMO SUMMARY
============================================================
Key Achievements:
Demonstrated 8 new operator variants
Showed practical applications across 7 domains
Validated Opifex framework integration
Confirmed performance and accuracy
Parameter Efficiency:
TFNO achieved 0.1x parameter reduction
Multi-Scale Turbulence:
U-FNO processed 4 scale levels in 6826.6ms
Uncertainty Quantification:
UQNO epistemic uncertainty ratio: 0.000
UQNO aleatoric uncertainty ratio: 1.000
Molecular Dynamics:
MGNO force conservation error: 112.030533
Ensemble Methods:
Multi-operator agreement score: 0.627
Demo completed successfully!
Results stored in demo.results
Results saved to: examples_output/neural_operators_demo_results.json
| Operator | Demo Task | Input Shape | Output Shape | Forward Time |
|---|---|---|---|---|
| U-FNO (4 levels) | Turbulent flow | (4, 3, 64, 64) | (4, 3, 64, 64) | 6826.56 ms |
| SFNO (lmax=16) | Global climate | (2, 5, 32, 64) | (2, 5, 32, 64) | 826.79 ms |
| UQNO (50 samples) | Uncertainty | (2, 32, 32, 2) | (2, 32, 32, 1) | 3289.85 ms |
| GINO (attention) | Airfoil geometry | (2, 64, 64, 2) | (2, 64, 64, 2) | 2473.02 ms |
| MGNO (degree=3) | Molecular forces | (2, 48, 4) + (2, 48, 3) | (2, 48, 3) | 2658.08 ms |
| FNO (ensemble) | Benchmark | (4, 2, 32, 32) | (4, 1, 32, 32) | 1057.42 ms |
| TFNO (ensemble) | Benchmark | (4, 2, 32, 32) | (4, 1, 32, 32) | 526.24 ms |
| LocalFNO (ensemble) | Benchmark | (4, 2, 32, 32) | (4, 1, 32, 32) | 1049.14 ms |
Parameter Counts¶
| Operator Model | Parameters | Compression vs Standard FNO |
|---|---|---|
| Standard FNO | 279,105 | 1.0x (baseline) |
| Tucker TFNO (10%) | 4,194,625 | 0.1x |
| CP TFNO | 4,194,625 | 0.1x |
| U-FNO (3 levels) | 27,977,601 | 0.0x |
What We Achieved¶
- Validated all 8 neural operator variants working in the Opifex framework
- Compared parameter efficiency across FNO, TFNO (Tucker and CP), and U-FNO
- Tested domain-specific operators for turbulence, climate, molecules, and geometry
- Decomposed uncertainty into epistemic (0.000) and aleatoric (1.000) components with UQNO
- Built a 3-operator ensemble with 0.627 agreement score
Next Steps¶
Experiments to Try¶
- Train operators on real data: Use the
Trainer.fit()API to train any operator on Darcy flow, Burgers, or custom datasets - Tune hyperparameters: Adjust
hidden_channels,modes,num_layers, andrankfor your specific problem - Combine UQNO with conformal prediction: Use calibrated uncertainty bounds for safety-critical deployment
- Scale up: Increase resolution and batch size, leveraging JAX JIT compilation for GPU acceleration
- Use the factory: Let
recommend_operator()guide architecture selection for new problem domains
Related Examples¶
| Example | Level | What You'll Learn |
|---|---|---|
| FNO Darcy Full | Intermediate | Full FNO training pipeline with grid embeddings on Darcy flow |
| SFNO Climate Full | Intermediate | Spherical FNO for climate modeling on the sphere |
| SFNO Climate Simple | Intermediate | Simplified SFNO climate example |
| U-FNO Turbulence | Intermediate | U-Net enhanced FNO for turbulence problems |
| UNO Darcy Framework | Intermediate | Multi-resolution U-shaped neural operator for Darcy flow |
| Neural Operator Benchmark | Advanced | Cross-architecture comparison (UNO, FNO, SFNO) |
| Grid Embeddings | Beginner | Spatial coordinate injection for neural operators |
| Spectral Normalization | Intermediate | Stabilize operator training with spectral normalization |
API Reference¶
FourierNeuralOperator-- Standard FNO with spectral convolution layersTensorizedFourierNeuralOperator-- TFNO with Tucker/CP factorizationUFourierNeuralOperator-- U-FNO with multi-scale encoder-decoderSphericalFourierNeuralOperator-- SFNO with spherical harmonicsGeometryInformedNeuralOperator-- GINO with geometry attentionMultipoleGraphNeuralOperator-- MGNO with multipole interactionsUncertaintyQuantificationNeuralOperator-- UQNO with Bayesian uncertaintycreate_operator-- Factory function for creating any operator by namerecommend_operator-- Application-aware operator recommendationlist_operators-- List all available operators by category
Troubleshooting¶
OOM during operator creation¶
Symptom: jaxlib.xla_extension.XlaRuntimeError: RESOURCE_EXHAUSTED when creating large operators like U-FNO.
Cause: Operators with many levels or high hidden channel counts can exceed GPU memory, especially U-FNO with num_levels=4 and hidden_channels=64.
Solution:
# Option 1: Reduce hidden channels
ufno = UFourierNeuralOperator(
in_channels=3, out_channels=3, hidden_channels=32, # Was 64
modes=(16, 16), num_levels=3, rngs=rngs,
)
# Option 2: Reduce number of levels
ufno = UFourierNeuralOperator(
in_channels=3, out_channels=3, hidden_channels=64,
modes=(16, 16), num_levels=2, rngs=rngs, # Was 4
)
# Option 3: Use TFNO for parameter efficiency
tfno = TensorizedFourierNeuralOperator(
in_channels=3, out_channels=3, hidden_channels=64,
modes=(16, 16), factorization="tucker", rank=0.1, rngs=rngs,
)
UQNO uncertainty is all zeros¶
Symptom: epistemic_uncertainty returns all zeros from predict_with_uncertainty().
Cause: Before training, all Bayesian weight samples produce identical outputs because the model has not learned to use the stochastic components. This is expected behavior.
Solution: Train the UQNO on data first using Trainer.fit(). After training, the epistemic uncertainty will reflect genuine model uncertainty about predictions in regions with sparse training data.
MGNO force conservation error is large¶
Symptom: The force_conservation_error (sum of forces over all atoms) is not close to zero.
Cause: The MGNO has not been trained, so its force predictions do not satisfy Newton's third law. Conservation properties emerge through training on physical data.
Solution: Train the MGNO on molecular dynamics trajectory data where conservation laws are enforced in the training loss. You can add a conservation penalty term:
def conservation_loss(forces):
"""Penalize net force on the system."""
total_force = jnp.sum(forces, axis=1) # Sum over atoms
return jnp.mean(jnp.sum(total_force ** 2, axis=-1))
GINO geometry sensitivity is zero¶
Symptom: geometry_sensitivity between original and scaled coordinates is 0.000000.
Cause: The untrained GINO may not yet utilize geometry attention effectively. The geometry integration layer needs training data to learn coordinate-dependent features.
Solution: This is expected for an untrained model. After training on geometry-aware data (e.g., flow around airfoils with varying shapes), the model will produce different predictions for different coordinate configurations.
Slow forward pass times¶
Symptom: Forward pass times are much higher than expected (several seconds).
Cause: The first forward pass through any JAX model includes XLA compilation time. Subsequent calls are significantly faster.
Solution: Run a warmup pass before timing: