Optimization API Reference¶

Overview¶

The Opifex optimization module provides full optimization algorithms and meta-learning approaches for scientific computing, including production optimization, learn-to-optimize algorithms, control systems, and quantum-aware optimization.

Core Optimization Components¶

Meta-Optimization Framework¶

Advanced meta-optimization algorithms that learn to optimize across families of related problems.

Meta-optimization algorithms for scientific machine learning.

This package implements meta-learning approaches to optimization including learn-to-optimize (L2O) algorithms, adaptive learning rate scheduling, warm-starting strategies, and performance monitoring. All implementations follow FLAX NNX patterns and are designed for scientific computing applications.

Key Features

Learn-to-optimize (L2O) meta-learning algorithms
Adaptive learning rate scheduling with multiple strategies
Warm-starting based on problem similarity
Performance monitoring and analytics
Quantum-aware optimization adaptations
Integration with existing training infrastructure

Author: Opifex Framework Team Date: December 2024 License: MIT

MetaOptimizerConfig `dataclass` ¶

MetaOptimizerConfig(meta_algorithm: str = 'l2o', base_optimizer: str = 'adam', meta_learning_rate: float = 0.0001, adaptation_steps: int = 10, warm_start_strategy: str = 'previous_params', performance_tracking: bool = True, memory_efficient: bool = True, quantum_aware: bool = False, scf_adaptation: bool = False, energy_convergence_tracking: bool = False, chemical_accuracy_target: float = 0.001)

Configuration for meta-optimization algorithms.

This configuration class defines all parameters for meta-optimization including algorithm selection, adaptation strategies, and performance monitoring settings.

Attributes:

Name	Type	Description
`meta_algorithm`	`str`	Meta-optimization algorithm ('l2o', 'adaptive_lr', 'warm_start')
`base_optimizer`	`str`	Base optimizer to enhance ('adam', 'sgd', 'rmsprop', 'adamw')
`meta_learning_rate`	`float`	Learning rate for meta-parameters
`adaptation_steps`	`int`	Number of steps for adaptation
`warm_start_strategy`	`str`	Strategy for warm-starting ('previous_params', 'similar_problems')
`performance_tracking`	`bool`	Enable performance monitoring
`memory_efficient`	`bool`	Use memory-efficient implementations
`quantum_aware`	`bool`	Enable quantum-specific adaptations
`scf_adaptation`	`bool`	Enable SCF convergence acceleration
`energy_convergence_tracking`	`bool`	Track energy convergence for quantum systems
`chemical_accuracy_target`	`float`	Target chemical accuracy (kcal/mol)

MetaOptimizer ¶

MetaOptimizer(config: MetaOptimizerConfig, *, rngs: Rngs)

Integrated meta-optimization system.

This class provides a complete meta-optimization system that integrates learn-to-optimize algorithms, adaptive learning rate scheduling, warm-starting strategies, and performance monitoring.

Attributes:

Name	Type	Description
`config`		Meta-optimizer configuration
`l2o_engine`		Learn-to-optimize engine
`learning_rate_scheduler`		Adaptive learning rate scheduler
`warm_start_strategy`		Warm-starting strategy
`performance_monitor`		Performance monitoring system
`current_step`		Current optimization step

Parameters:

Name	Type	Description	Default
`config`	`MetaOptimizerConfig`	Meta-optimizer configuration	required
`rngs`	`Rngs`	Random number generators	required

init_optimizer_state ¶

init_optimizer_state(params: Array) -> Any

Initialize optimizer state.

Parameters:

Name	Type	Description	Default
`params`	`Array`	Initial parameters	required

Returns:

Type	Description
`Any`	Initial optimizer state

step ¶

step(loss_fn: Callable[[Array], Array], params: Array, opt_state: Any, step: int) -> tuple[Array, Any, dict[str, Any]]

Perform single meta-optimization step.

Parameters:

Name	Type	Description	Default
`loss_fn`	`Callable[[Array], Array]`	Loss function to optimize	required
`params`	`Array`	Current parameters	required
`opt_state`	`Any`	Current optimizer state	required
`step`	`int`	Current step number	required

Returns:

Type	Description
`tuple[Array, Any, dict[str, Any]]`	Tuple of (new_params, new_opt_state, meta_info)

store_optimization_result ¶

store_optimization_result(params: Array, problem_features: Array) -> None

Store optimization result for future warm-starting.

Parameters:

Name	Type	Description	Default
`params`	`Array`	Final optimized parameters	required
`problem_features`	`Array`	Features characterizing the problem	required

get_warm_start_params ¶

get_warm_start_params(current_problem_features: Array, target_shape: tuple[int, ...]) -> Array

Get warm-start parameters for new problem.

Parameters:

Name	Type	Description	Default
`current_problem_features`	`Array`	Features of current problem	required
`target_shape`	`tuple[int, ...]`	Target shape for parameters	required

Returns:

Type	Description
`Array`	Warm-start parameters

quantum_step ¶

quantum_step(energy_fn: Callable[[Array], Array], orbital_coeffs: Array, opt_state: Any, scf_context: dict[str, Any], step: int) -> tuple[Array, Any, dict[str, Any]]

Perform quantum-aware meta-optimization step.

Parameters:

Name	Type	Description	Default
`energy_fn`	`Callable[[Array], Array]`	Energy function to minimize	required
`orbital_coeffs`	`Array`	Current orbital coefficients	required
`opt_state`	`Any`	Current optimizer state	required
`scf_context`	`dict[str, Any]`	SCF iteration context	required
`step`	`int`	Current step number	required

Returns:

Type	Description
`tuple[Array, Any, dict[str, Any]]`	Tuple of (new_coeffs, new_opt_state, quantum_info)

PerformanceMonitor ¶

PerformanceMonitor(metrics: list[str] | None = None, window_size: int = 100, tracking_frequency: int = 1, convergence_tolerance: float = 1e-06, convergence_patience: int = 10, analytics_enabled: bool = False, quantum_aware: bool = False)

Performance monitoring and analytics for meta-optimization.

This class provides full performance monitoring capabilities including metric tracking, convergence detection, and performance analytics for optimization algorithms.

Attributes:

Name	Type	Description
`metrics`		List of metrics to track
`window_size`		Size of rolling window for metrics
`tracking_frequency`		Frequency of metric updates
`convergence_tolerance`		Tolerance for convergence detection
`convergence_patience`		Patience for convergence detection
`analytics_enabled`		Enable detailed analytics
`quantum_aware`		Enable quantum-specific metrics

Parameters:

Name	Type	Description	Default
`metrics`	`list[str] \| None`	List of metrics to track	`None`
`window_size`	`int`	Rolling window size	`100`
`tracking_frequency`	`int`	How often to update metrics	`1`
`convergence_tolerance`	`float`	Tolerance for convergence	`1e-06`
`convergence_patience`	`int`	Patience for convergence detection	`10`
`analytics_enabled`	`bool`	Enable detailed analytics	`False`
`quantum_aware`	`bool`	Enable quantum metrics	`False`

update_metrics ¶

update_metrics(step: int, **metric_values: float) -> None

Update tracked metrics.

Parameters:

Name	Type	Description	Default
`step`	`int`	Current optimization step	required
`**metric_values`	`float`	Metric values to update	`{}`

get_metric_history ¶

get_metric_history(metric: str) -> list[float]

Get history of a specific metric.

Parameters:

Name	Type	Description	Default
`metric`	`str`	Metric name	required

Returns:

Type	Description
`list[float]`	List of metric values

check_convergence ¶

check_convergence(metric: str) -> bool

Check if a metric has converged.

Parameters:

Name	Type	Description	Default
`metric`	`str`	Metric name	required

Returns:

Type	Description
`bool`	True if metric has converged

get_performance_analytics ¶

get_performance_analytics() -> dict[str, Any]

Get full performance analytics.

Returns:

Type	Description
`dict[str, Any]`	Dictionary containing performance analytics

get_quantum_analytics ¶

get_quantum_analytics() -> dict[str, Any]

Get quantum-specific performance analytics.

Returns:

Type	Description
`dict[str, Any]`	Dictionary containing quantum analytics

LearnToOptimize ¶

LearnToOptimize(meta_network_layers: list[int] | None = None, base_optimizer: str = 'adam', meta_learning_rate: float = 0.0001, unroll_steps: int = 20, adaptive_step_size: bool = False, quantum_aware: bool = False, scf_integration: bool = False, *, rngs: Rngs)

Bases: Module

Learn-to-optimize (L2O) meta-learning system.

This class implements learn-to-optimize algorithms that use neural networks to learn optimization strategies from data. The meta-network learns to predict good parameter updates based on gradient information and optimization history.

Attributes:

Name	Type	Description
`meta_network`		Neural network for learning optimization rules
`base_optimizer`		Base optimization algorithm
`meta_learning_rate`		Learning rate for meta-network training
`unroll_steps`		Number of unrolling steps for meta-gradient computation
`adaptive_step_size`		Enable adaptive step size learning
`quantum_aware`		Enable quantum-specific adaptations
`scf_integration`		Enable SCF convergence acceleration

Parameters:

Name	Type	Description	Default
`meta_network_layers`	`list[int] \| None`	Architecture of meta-network	`None`
`base_optimizer`	`str`	Base optimizer to enhance	`'adam'`
`meta_learning_rate`	`float`	Learning rate for meta-network training	`0.0001`
`unroll_steps`	`int`	Number of unroll steps for meta-gradients	`20`
`adaptive_step_size`	`bool`	Enable adaptive step size learning	`False`
`quantum_aware`	`bool`	Enable quantum-specific optimizations	`False`
`scf_integration`	`bool`	Enable SCF convergence acceleration	`False`
`rngs`	`Rngs`	Random number generators for initialization	required

compute_update ¶

compute_update(gradient: Array, previous_updates: Array, loss_history: Array | None = None) -> Array

Compute parameter update using meta-network.

Parameters:

Name	Type	Description	Default
`gradient`	`Array`	Current gradient	required
`previous_updates`	`Array`	History of previous updates	required
`loss_history`	`Array \| None`	History of loss values	`None`

Returns:

Type	Description
`Array`	Predicted parameter update

compute_meta_gradients ¶

compute_meta_gradients(loss_fn: Callable[[Array], Array], initial_params: Array) -> dict[str, Array]

Compute meta-gradients for meta-network training.

Parameters:

Name	Type	Description	Default
`loss_fn`	`Callable[[Array], Array]`	Loss function for optimization problem	required
`initial_params`	`Array`	Initial parameters for optimization	required

Returns:

Type	Description
`dict[str, Array]`	Meta-gradients for meta-network parameters

compute_adaptive_update ¶

compute_adaptive_update(gradient: Array, previous_updates: Array) -> Array

Compute adaptive parameter update.

compute_quantum_update ¶

compute_quantum_update(orbital_params: Array, scf_history: Array) -> Array

Compute quantum-aware parameter update.

Parameters:

Name	Type	Description	Default
`orbital_params`	`Array`	Orbital coefficient parameters	required
`scf_history`	`Array`	SCF convergence history	required

Returns:

Type	Description
`Array`	Quantum-adapted parameter update

AdaptiveLearningRateScheduler ¶

AdaptiveLearningRateScheduler(schedule_type: str = 'cosine_annealing', initial_lr: float = 0.001, final_lr: float = 1e-06, adaptation_period: int = 100, warmup_steps: int = 0, patience: int = 5, factor: float = 0.5, min_lr: float = 1e-08, **kwargs: Any)

Adaptive learning rate scheduling for meta-optimization.

This class implements various adaptive learning rate scheduling strategies including cosine annealing, performance-based adaptation, and quantum-aware scheduling for scientific applications.

Attributes:

Name	Type	Description
`schedule_type`		Type of scheduling algorithm
`initial_lr`		Initial learning rate
`final_lr`		Final learning rate (for annealing schedules)
`adaptation_period`		Period for adaptation cycles
`warmup_steps`		Number of warmup steps
`patience`		Patience for performance-based adaptation
`factor`		Factor for learning rate reduction
`min_lr`		Minimum learning rate

Parameters:

Name	Type	Description	Default
`schedule_type`	`str`	Type of scheduling ('cosine_annealing', 'performance_based', 'quantum_aware')	`'cosine_annealing'`
`initial_lr`	`float`	Initial learning rate	`0.001`
`final_lr`	`float`	Final learning rate	`1e-06`
`adaptation_period`	`int`	Period for complete adaptation cycle	`100`
`warmup_steps`	`int`	Number of warmup steps	`0`
`patience`	`int`	Patience for performance-based adaptation	`5`
`factor`	`float`	Reduction factor for learning rate	`0.5`
`min_lr`	`float`	Minimum allowed learning rate	`1e-08`
`**kwargs`	`Any`	Additional scheduler-specific parameters	`{}`

get_learning_rate ¶

get_learning_rate(step: int) -> Array

Get learning rate for current step.

Parameters:

Name	Type	Description	Default
`step`	`int`	Current optimization step	required

Returns:

Type	Description
`Array`	Learning rate for current step

adapt_from_performance ¶

adapt_from_performance(loss_history: list[float]) -> float

Adapt learning rate based on performance history.

Parameters:

Name	Type	Description	Default
`loss_history`	`list[float]`	Recent loss values	required

Returns:

Type	Description
`float`	Adapted learning rate

adapt_from_quantum_metrics ¶

adapt_from_quantum_metrics(scf_errors: list[float], energy_changes: list[float]) -> float

Adapt learning rate based on quantum mechanical metrics.

Parameters:

Name	Type	Description	Default
`scf_errors`	`list[float]`	SCF convergence errors	required
`energy_changes`	`list[float]`	Energy change magnitudes	required

Returns:

Type	Description
`float`	Quantum-adapted learning rate

WarmStartingStrategy ¶

WarmStartingStrategy(strategy_type: str = 'parameter_transfer', similarity_threshold: float = 0.8, adaptation_steps: int = 5, memory_size: int = 10, adaptation_ratio: float = 0.9, similarity_metric: str = 'cosine', min_similarity: float = 0.7)

Warm-starting strategies for optimization acceleration.

This class implements various warm-starting strategies to accelerate optimization by leveraging information from previous optimizations or similar problems.

Attributes:

Name	Type	Description
`strategy_type`		Type of warm-starting strategy
`similarity_threshold`		Threshold for problem similarity
`adaptation_steps`		Steps for parameter adaptation
`memory_size`		Size of optimization memory
`adaptation_ratio`		Ratio for optimizer state adaptation

Parameters:

Name	Type	Description	Default
`strategy_type`	`str`	Strategy type ('parameter_transfer', 'optimizer_state_transfer', 'molecular_similarity')	`'parameter_transfer'`
`similarity_threshold`	`float`	Threshold for considering problems similar	`0.8`
`adaptation_steps`	`int`	Number of adaptation steps	`5`
`memory_size`	`int`	Maximum number of previous optimizations to remember	`10`
`adaptation_ratio`	`float`	Ratio for adapting previous states	`0.9`
`similarity_metric`	`str`	Metric for similarity computation	`'cosine'`
`min_similarity`	`float`	Minimum similarity for warm-starting	`0.7`

get_warm_start_params ¶

get_warm_start_params(previous_params: Array, current_problem_features: Array) -> Array

Get warm-start parameters based on parameter transfer.

Parameters:

Name	Type	Description	Default
`previous_params`	`Array`	Parameters from previous optimization	required
`current_problem_features`	`Array`	Features of current problem	required

Returns:

Type	Description
`Array`	Warm-start parameters for current problem

adapt_optimizer_state ¶

adapt_optimizer_state(previous_opt_state: dict[str, Any]) -> dict[str, Any]

Adapt optimizer state for warm-starting.

Parameters:

Name	Type	Description	Default
`previous_opt_state`	`dict[str, Any]`	Previous optimizer state	required

Returns:

Type	Description
`dict[str, Any]`	Adapted optimizer state

get_molecular_warm_start ¶

get_molecular_warm_start(previous_fingerprints: Array, previous_params: Array, current_fingerprint: Array) -> Array

Get warm-start parameters based on molecular similarity.

Parameters:

Name	Type	Description	Default
`previous_fingerprints`	`Array`	Fingerprints of previous molecules	required
`previous_params`	`Array`	Parameters for previous molecules	required
`current_fingerprint`	`Array`	Fingerprint of current molecule	required

Returns:

Type	Description
`Array`	Warm-start parameters based on most similar molecule

Production Optimization¶

Enterprise-grade optimization systems for deployment and scaling in production environments.

Production optimization for Opifex framework.

This module implements the Hybrid Performance Platform for production deployment with adaptive JIT compilation, GPU memory management, and performance optimization.

Version 7.4: Production Optimization Implementation Selected Architecture: Hybrid Performance Platform + Intelligent Edge + Adaptive Optimization

CallableModule ¶

Bases: Protocol

Protocol for callable modules.

OptimizationStrategy ¶

Bases: Enum

JIT optimization strategies for different workload patterns.

WorkloadProfile `dataclass` ¶

WorkloadProfile(batch_size: int, sequence_length: int, memory_footprint: float, compute_intensity: float, latency_requirement: float, throughput_requirement: float, model_complexity: str)

Profiling data for production workloads.

PerformanceMetrics `dataclass` ¶

PerformanceMetrics(latency_ms: float, throughput_rps: float, memory_usage_gb: float, gpu_utilization: float, energy_efficiency: float, improvement_factor: float)

Performance metrics for optimization validation.

OptimizedModel `dataclass` ¶

OptimizedModel(model: Module, optimization_type: OptimizationStrategy, performance_metrics: PerformanceMetrics, optimization_metadata: dict[str, Any])

Container for optimized model with performance metadata.

AdaptiveJAXOptimizer ¶

AdaptiveJAXOptimizer(performance_threshold: float = 1.1, memory_efficiency_target: float = 0.85, cache_size: int = 100)

Bases: Module

Adaptive JIT optimization for JAX-based neural operators.

This class implements intelligent JIT compilation strategies based on workload patterns, providing optimal performance for production deployments.

Parameters:

Name	Type	Description	Default
`performance_threshold`	`float`	Minimum performance improvement factor to accept optimization	`1.1`
`memory_efficiency_target`	`float`	Target memory efficiency (0-1 scale)	`0.85`
`cache_size`	`int`	Number of optimization strategies to cache	`100`

analyze_workload_patterns ¶

analyze_workload_patterns(workload: WorkloadProfile) -> OptimizationStrategy

Analyze workload to select optimal optimization strategy.

apply_aggressive_kernel_fusion ¶

apply_aggressive_kernel_fusion(model: Module) -> Module

Apply aggressive kernel fusion for compute-intensive workloads.

apply_memory_optimization ¶

apply_memory_optimization(model: Module) -> Module

Apply memory optimization for large models.

apply_latency_optimization ¶

apply_latency_optimization(model: Module) -> Module

Apply latency optimization for real-time inference.

apply_balanced_optimization ¶

apply_balanced_optimization(model: Module) -> Module

Apply balanced optimization for general workloads.

benchmark_model_performance ¶

benchmark_model_performance(model: Module, workload: WorkloadProfile) -> PerformanceMetrics

Benchmark model performance for given workload.

optimize_neural_operator ¶

optimize_neural_operator(model: Module, workload: WorkloadProfile) -> OptimizedModel

Optimize neural operator for production workload.

Parameters:

Name	Type	Description	Default
`model`	`Module`	Neural operator model to optimize	required
`workload`	`WorkloadProfile`	Workload profile for optimization	required

Returns:

Type	Description
`OptimizedModel`	OptimizedModel with performance improvements

IntelligentGPUMemoryManager ¶

IntelligentGPUMemoryManager(fragmentation_threshold: float = 0.15, gc_trigger_threshold: float = 0.85, pool_sizes: dict[str, tuple[int, int]] | None = None)

Bases: Module

Advanced GPU memory management for production workloads.

Implements intelligent allocation, fragmentation prevention, and multi-model inference optimization.

Parameters:

Name	Type	Description	Default
`fragmentation_threshold`	`float`	Maximum acceptable fragmentation (0-1)	`0.15`
`gc_trigger_threshold`	`float`	Memory usage threshold to trigger GC (0-1)	`0.85`
`pool_sizes`	`dict[str, tuple[int, int]] \| None`	Memory pool sizes as {pool_name: (min_size_mb, max_size_mb)}	`None`

select_memory_pool ¶

select_memory_pool(size_mb: float) -> str

Select appropriate memory pool for allocation size.

estimate_model_memory_usage ¶

estimate_model_memory_usage(model: Module, batch_size: int) -> float

Estimate memory usage for model inference (in MB).

optimize_multi_model_allocation ¶

optimize_multi_model_allocation(models: list[tuple[Module, int]]) -> dict[str, Any]

Optimize memory allocation for multiple concurrent models.

Parameters:

Name	Type	Description	Default
`models`	`list[tuple[Module, int]]`	List of (model, batch_size) tuples	required

Returns:

Type	Description
`dict[str, Any]`	Allocation plan with memory optimization strategy

HybridPerformancePlatform ¶

HybridPerformancePlatform(jit_optimizer: AdaptiveJAXOptimizer | None = None, memory_manager: IntelligentGPUMemoryManager | None = None, performance_monitor: PerformanceMonitor | None = None, scientific_integrator: ScientificComputingIntegrator | None = None, predictive_scaler: PredictiveScaler | None = None, edge_network: IntelligentEdgeNetwork | None = None, deployment_system: AdaptiveDeploymentSystem | None = None, resource_manager: GlobalResourceManager | None = None, physics_domain: PhysicsDomain = GENERAL, target_latency_ms: float = 0.5, *, rngs: Rngs)

Bases: Module

Hybrid Performance Platform for production optimization.

This is the main orchestrator for Version 7.4 Production Optimization, integrating all 6 major components: JIT optimization, edge network, adaptive deployment, performance monitoring, scientific validation, and global resource management.

optimize_for_production ¶

optimize_for_production(model: Module, workload: WorkloadProfile) -> OptimizedModel

Full production optimization for a model.

This method now includes Version 7.4 enhancements: - AI-powered performance monitoring - Scientific computing validation - Predictive scaling recommendations

start_continuous_monitoring `async` ¶

start_continuous_monitoring() -> None

Start continuous performance monitoring for production systems.

stop_continuous_monitoring `async` ¶

stop_continuous_monitoring() -> None

Stop continuous performance monitoring.

get_comprehensive_status ¶

get_comprehensive_status() -> dict[str, Any]

Get full status of the hybrid performance platform.

get_model_input_features ¶

get_model_input_features(model: Module) -> int

Extract the correct input feature dimension from a model.

This utility function provides a robust way to determine the expected input dimension for any model, preventing dimension mismatch errors.

Parameters:

Name	Type	Description	Default
`model`	`Module`	The model to inspect	required

Returns:

Name	Type	Description
`int`	`int`	The input feature dimension

Raises:

Type	Description
`ValueError`	If input features cannot be determined

Performance Monitoring¶

AI-powered performance monitoring with predictive scaling and anomaly detection.

Performance monitoring and prediction for Opifex production optimization.

This module implements AI-powered performance monitoring, anomaly detection, and predictive scaling for the Version 7.4 Production Optimization system.

Part of: Hybrid Performance Platform + Intelligent Edge + Adaptive Optimization

AnomalySeverity ¶

Bases: Enum

Severity levels for performance anomalies.

Anomaly `dataclass` ¶

Anomaly(anomaly_id: str, timestamp: float, severity: AnomalySeverity, anomaly_type: str, description: str, metrics: dict[str, float], confidence: float, recommended_action: str)

Performance anomaly detection result.

PerformanceMetrics `dataclass` ¶

PerformanceMetrics(timestamp: float, latency_ms: float, throughput_rps: float, memory_usage_gb: float, gpu_utilization: float, cpu_utilization: float, energy_efficiency: float, error_rate: float, numerical_stability: float = 0.0, conservation_score: float = 0.0, physics_consistency: float = 0.0, time_series_data: dict[str, list[float]] = dict(), feature_matrix: ndarray | None = None, gpu_metrics: dict[str, float] = dict(), conservation_metrics: dict[str, float] = dict(), numerical_metrics: dict[str, float] = dict(), physics_metrics: dict[str, float] = dict())

Full performance metrics for monitoring.

PredictionResult `dataclass` ¶

PredictionResult(predicted_latency: float, predicted_throughput: float, predicted_memory_usage: float, confidence_interval: tuple[float, float], prediction_horizon_minutes: int, recommended_scaling_action: str)

Result of performance prediction.

PerformanceMonitorProtocol ¶

Bases: Protocol

Protocol for performance monitoring implementations.

collect_metrics `async` ¶

collect_metrics() -> PerformanceMetrics

Collect current performance metrics.

analyze_trends `async` ¶

analyze_trends(metrics_history: list[PerformanceMetrics]) -> dict[str, Any]

Analyze performance trends.

PerformancePredictor ¶

PerformancePredictor(input_features: int = 32, hidden_features: int = 128, prediction_horizon: int = 60, *, rngs: Rngs)

Bases: Module

Neural network-based performance predictor.

AIAnomalyDetector ¶

AIAnomalyDetector(input_features: int = 16, hidden_features: int = 64, anomaly_threshold: float = 0.8, *, rngs: Rngs)

Bases: Module

AI-powered anomaly detection for scientific computing performance.

detect_anomalies ¶

detect_anomalies(metrics: ndarray) -> tuple[ndarray, ndarray]

Detect anomalies and return reconstruction error.

PerformanceMonitor ¶

PerformanceMonitor(anomaly_detector: AIAnomalyDetector, performance_predictor: PerformancePredictor, collection_interval: float = 1.0)

Real-time performance monitoring system.

start_monitoring `async` ¶

start_monitoring() -> None

Start continuous performance monitoring.

stop_monitoring `async` ¶

stop_monitoring() -> None

Stop continuous monitoring.

collect_current_metrics `async` ¶

collect_current_metrics() -> PerformanceMetrics

Collect current system performance metrics.

detect_performance_anomalies `async` ¶

detect_performance_anomalies(metrics: PerformanceMetrics) -> list[Anomaly]

Detect performance anomalies using AI models.

predict_future_performance `async` ¶

predict_future_performance() -> PredictionResult

Predict future performance metrics.

handle_anomalies `async` ¶

handle_anomalies(anomalies: list[Anomaly]) -> None

Handle detected anomalies.

handle_predictions `async` ¶

handle_predictions(predictions: PredictionResult) -> None

Handle performance predictions.

PredictiveScaler ¶

PredictiveScaler(performance_monitor: PerformanceMonitor, scale_up_threshold: float = 1.2, scale_down_threshold: float = 0.8, min_replicas: int = 1, max_replicas: int = 10)

Predictive scaling engine for scientific workloads.

evaluate_scaling_decision `async` ¶

evaluate_scaling_decision() -> dict[str, Any]

Evaluate whether scaling is needed based on predictions.

execute_scaling_action `async` ¶

execute_scaling_action(scaling_decision: dict[str, Any]) -> bool

Execute the scaling action.

Adaptive Deployment¶

AI-driven deployment strategies with automatic rollback capabilities.

Adaptive Deployment System for Opifex production optimization.

This module implements AI-driven deployment strategies, canary deployments, A/B testing, and automatic rollback for the Version 7.4 Production Optimization system.

Part of: Hybrid Performance Platform + Intelligent Edge + Adaptive Optimization

DeploymentStrategy ¶

Bases: Enum

Deployment strategies for production releases.

DeploymentStatus ¶

Bases: Enum

Status of deployment operations.

RollbackTrigger ¶

Bases: Enum

Triggers for automatic rollback.

DeploymentConfig `dataclass` ¶

DeploymentConfig(strategy: DeploymentStrategy, traffic_split_percentage: float, rollout_duration_minutes: int, health_check_interval_seconds: int, success_threshold_percentage: float, error_threshold_percentage: float, latency_threshold_ms: float, auto_rollback_enabled: bool, monitoring_enabled: bool, a_b_test_duration_hours: int = 24, feature_flag_percentage: float = 0.0)

Configuration for deployment strategies.

DeploymentMetrics `dataclass` ¶

DeploymentMetrics(deployment_id: str, timestamp: float, success_rate: float, error_rate: float, latency_p50_ms: float, latency_p95_ms: float, latency_p99_ms: float, throughput_rps: float, cpu_utilization: float, memory_utilization: float, gpu_utilization: float, user_satisfaction_score: float = 0.0, numerical_accuracy: float = 0.0, conservation_score: float = 0.0, physics_consistency: float = 0.0)

Metrics collected during deployment.

DeploymentState `dataclass` ¶

DeploymentState(deployment_id: str, status: DeploymentStatus, strategy: DeploymentStrategy, start_time: float, current_traffic_percentage: float, target_traffic_percentage: float, metrics_history: list[DeploymentMetrics] = list(), rollback_triggers: list[RollbackTrigger] = list(), health_checks_passed: int = 0, health_checks_failed: int = 0)

Current state of a deployment.

RollbackDecision `dataclass` ¶

RollbackDecision(should_rollback: bool, trigger: RollbackTrigger, confidence: float, reason: str, rollback_strategy: str, estimated_rollback_time_minutes: int)

Decision result for rollback evaluation.

DeploymentAI ¶

DeploymentAI(input_features: int = 24, metric_features: int = 12, hidden_features: int = 128, decision_threshold: float = 0.7, *, rngs: Rngs)

Bases: Module

AI-driven deployment decision engine.

select_deployment_strategy ¶

select_deployment_strategy(system_features: ndarray) -> tuple[DeploymentStrategy, float]

Select optimal deployment strategy based on system state.

predict_rollback_probability ¶

predict_rollback_probability(deployment_metrics: ndarray) -> float

Predict probability that deployment should be rolled back.

optimize_traffic_split ¶

optimize_traffic_split(current_metrics: ndarray, target_metrics: ndarray) -> float

Optimize traffic split percentage for gradual rollout.

CanaryController ¶

CanaryController(deployment_ai: DeploymentAI, initial_traffic_percentage: float = 5.0, progression_steps: list[float] | None = None, evaluation_period_minutes: int = 10)

Controller for canary deployments with automatic progression.

start_canary_deployment `async` ¶

start_canary_deployment(deployment_id: str, config: DeploymentConfig) -> bool

Start a new canary deployment.

TrafficShaper ¶

TrafficShaper(deployment_ai: DeploymentAI, max_traffic_change_per_minute: float = 10.0)

Intelligent traffic shaping for load distribution.

optimize_traffic_distribution `async` ¶

optimize_traffic_distribution(deployments: dict[str, DeploymentState], target_distribution: dict[str, float]) -> dict[str, float]

Optimize traffic distribution across deployments.

RollbackEngine ¶

RollbackEngine(deployment_ai: DeploymentAI, rollback_threshold: float = 0.8, evaluation_window_minutes: int = 5)

Automatic rollback engine with performance-based triggers.

evaluate_rollback_decision `async` ¶

evaluate_rollback_decision(deployment_state: DeploymentState, config: DeploymentConfig) -> RollbackDecision

Evaluate whether deployment should be rolled back.

AdaptiveDeploymentSystem ¶

AdaptiveDeploymentSystem(deployment_ai: DeploymentAI, canary_controller: CanaryController, traffic_shaper: TrafficShaper, rollback_engine: RollbackEngine)

Main orchestrator for adaptive deployment with AI-driven strategies.

deploy_model `async` ¶

deploy_model(deployment_id: str, config: DeploymentConfig, system_features: ndarray) -> dict[str, Any]

Deploy model using AI-selected strategy.

monitor_deployments `async` ¶

monitor_deployments() -> None

Monitor all active deployments for health and rollback conditions.

get_deployment_status ¶

get_deployment_status(deployment_id: str) -> dict[str, Any] | None

Get current status of deployment.

get_system_statistics ¶

get_system_statistics() -> dict[str, Any]

Get full deployment system statistics.

Global Resource Management¶

Multi-cloud optimization with cost intelligence and sustainability tracking.

Global Resource Management for Opifex deployment and production.

This package implements multi-cloud optimization, GPU pool management, cost intelligence, and sustainability tracking for production deployments.

NO BACKWARD COMPATIBILITY - Clean package structure with breaking changes.

CostController ¶

CostController(budget_limit_usd_per_day: float = 10000.0, cost_optimization_interval: int = 3600, savings_target_percentage: float = 20.0)

Cost optimization and budget management for multi-cloud resources.

track_resource_cost ¶

track_resource_cost(allocation_id: str, cost_usd_per_hour: float, duration_hours: float, provider: CloudProvider, resource_type: ResourceType) -> None

Track cost for resource allocation.

analyze_cost_optimization_opportunities ¶

analyze_cost_optimization_opportunities(active_allocations: dict[str, ResourceAllocation]) -> CostOptimization

Analyze opportunities for cost optimization.

check_budget_alerts ¶

check_budget_alerts() -> dict[str, Any]

Check for budget alerts and violations.

get_cost_analytics ¶

get_cost_analytics() -> dict[str, Any]

Get full cost analytics and insights.

GlobalResourceManager ¶

GlobalResourceManager(resource_orchestrator: ResourceOrchestrator, gpu_pool_manager: GPUPoolManager, cost_controller: CostController, sustainability_tracker: SustainabilityTracker)

Main orchestrator for global resource management with multi-cloud optimization.

Coordinates resource allocation, cost optimization, sustainability tracking, and GPU memory management across multiple cloud providers.

Parameters:

Name	Type	Description	Default
`resource_orchestrator`	`ResourceOrchestrator`	ResourceOrchestrator for allocation decisions	required
`gpu_pool_manager`	`GPUPoolManager`	GPUPoolManager for GPU memory management	required
`cost_controller`	`CostController`	CostController for cost tracking and optimization	required
`sustainability_tracker`	`SustainabilityTracker`	SustainabilityTracker for carbon footprint tracking	required

allocate_resources_with_intelligence `async` ¶

allocate_resources_with_intelligence(resource_requirements: dict[ResourceType, int], constraints: dict[str, Any] | None = None, sustainability_priority: bool = True) -> dict[str, Any]

Allocate resources with full intelligence and optimization.

Parameters:

Name	Type	Description	Default
`resource_requirements`	`dict[ResourceType, int]`	Dictionary mapping resource types to quantities	required
`constraints`	`dict[str, Any] \| None`	Optional constraints for allocation	`None`
`sustainability_priority`	`bool`	Whether to prioritize sustainability	`True`

Returns:

Type	Description
`dict[str, Any]`	Dictionary containing allocation result, GPU allocations, cost estimate,
`dict[str, Any]`	carbon footprint, performance estimate, and allocation strategy

Raises:

Type	Description
`ValueError`	If no eligible resource pools found

start_resource_monitoring `async` ¶

start_resource_monitoring() -> None

Start full resource monitoring and optimization.

Monitors GPU memory, checks budget alerts, and updates sustainability metrics every 5 minutes until stopped.

stop_resource_monitoring `async` ¶

stop_resource_monitoring() -> None

Stop resource monitoring.

get_comprehensive_resource_status ¶

get_comprehensive_resource_status() -> dict[str, Any]

Get full status of all resource management components.

Returns:

Type	Description
`dict[str, Any]`	Dictionary containing status of orchestration, GPU management,
`dict[str, Any]`	cost optimization, cost analytics, sustainability, and monitoring

GPUPoolManager ¶

GPUPoolManager(resource_orchestrator: Any, memory_optimization_threshold: float = 0.85, pool_rebalancing_interval: int = 300)

Intelligent GPU pool management with multi-model inference optimization.

Parameters:

Name	Type	Description	Default
`resource_orchestrator`	`Any`	ResourceOrchestrator instance for coordination	required
`memory_optimization_threshold`	`float`	Utilization threshold to trigger optimization (0-1)	`0.85`
`pool_rebalancing_interval`	`int`	Interval between rebalancing attempts in seconds	`300`

create_gpu_pool ¶

create_gpu_pool(pool_id: str, gpu_type: ResourceType, gpu_count: int, memory_per_gpu_gb: float, provider: CloudProvider, region: str) -> bool

Create a new GPU pool.

Parameters:

Name	Type	Description	Default
`pool_id`	`str`	Unique identifier for the pool	required
`gpu_type`	`ResourceType`	Type of GPU resources	required
`gpu_count`	`int`	Number of GPUs in the pool	required
`memory_per_gpu_gb`	`float`	Memory per GPU in GB	required
`provider`	`CloudProvider`	Cloud provider hosting the pool	required
`region`	`str`	Geographic region of the pool	required

Returns:

Type	Description
`bool`	True if pool was created successfully

allocate_gpu_memory ¶

allocate_gpu_memory(model_hash: str, memory_requirement_gb: float, preferred_pool_id: str | None = None) -> dict[str, Any]

Allocate GPU memory for model deployment.

Parameters:

Name	Type	Description	Default
`model_hash`	`str`	Unique identifier for the model	required
`memory_requirement_gb`	`float`	Required memory in GB	required
`preferred_pool_id`	`str \| None`	Optional preferred pool for allocation	`None`

Returns:

Type	Description
`dict[str, Any]`	Dictionary with allocation result and details

deallocate_gpu_memory ¶

deallocate_gpu_memory(model_hash: str) -> bool

Deallocate GPU memory for a model.

Parameters:

Name	Type	Description	Default
`model_hash`	`str`	Model identifier	required

Returns:

Type	Description
`bool`	True if deallocation was successful

optimize_memory_layout `async` ¶

optimize_memory_layout() -> dict[str, Any]

Optimize memory layout across GPU pools.

Returns:

Type	Description
`dict[str, Any]`	Dictionary with optimization results

get_pool_statistics ¶

get_pool_statistics() -> dict[str, Any]

Get full GPU pool statistics.

Returns:

Type	Description
`dict[str, Any]`	Dictionary with pool statistics

ResourceOrchestrator ¶

ResourceOrchestrator(optimization_objective: OptimizationObjective = BALANCE_COST_PERFORMANCE, learning_rate: float = 0.001, *, rngs: Rngs)

Bases: Module

Multi-cloud resource orchestrator with intelligent allocation.

Parameters:

Name	Type	Description	Default
`optimization_objective`	`OptimizationObjective`	The optimization objective for resource allocation	`BALANCE_COST_PERFORMANCE`
`learning_rate`	`float`	Learning rate for neural network optimization (currently unused)	`0.001`
`rngs`	`Rngs`	Random number generators for neural network initialization	required

register_resource_pool ¶

register_resource_pool(pool: ResourcePool) -> bool

Register a new resource pool.

Parameters:

Name	Type	Description	Default
`pool`	`ResourcePool`	ResourcePool to register	required

Returns:

Type	Description
`bool`	True if registration was successful

update_pool_status ¶

update_pool_status(pool_id: str, utilization: float, available_capacity: int) -> bool

Update resource pool status.

Parameters:

Name	Type	Description	Default
`pool_id`	`str`	ID of the pool to update	required
`utilization`	`float`	Current utilization (0-1)	required
`available_capacity`	`int`	Available capacity in the pool	required

Returns:

Type	Description
`bool`	True if update was successful, False if pool not found

optimize_resource_allocation ¶

optimize_resource_allocation(resource_requirements: dict[ResourceType, int], constraints: dict[str, Any] | None = None) -> ResourceAllocation

Optimize resource allocation across multi-cloud infrastructure.

Parameters:

Name	Type	Description	Default
`resource_requirements`	`dict[ResourceType, int]`	Dictionary mapping resource types to quantities	required
`constraints`	`dict[str, Any] \| None`	Optional constraints for allocation (cost, performance, etc.)	`None`

Returns:

Type	Description
`ResourceAllocation`	ResourceAllocation with selected pools and estimates

Raises:

Type	Description
`ValueError`	If no eligible resource pools found

SustainabilityTracker ¶

SustainabilityTracker(carbon_reduction_target_percentage: float = 30.0, renewable_energy_preference: bool = True)

Carbon footprint tracking and sustainability optimization.

track_carbon_emissions ¶

track_carbon_emissions(allocation_id: str, carbon_footprint_kg: float, provider: CloudProvider, region: str, renewable_energy_percentage: float) -> None

Track carbon emissions for resource allocation.

calculate_sustainability_metrics ¶

calculate_sustainability_metrics() -> SustainabilityMetrics

Calculate full sustainability metrics.

optimize_for_sustainability ¶

optimize_for_sustainability(available_pools: list[ResourcePool]) -> list[ResourcePool]

Optimize resource selection for sustainability.

CloudProvider ¶

Bases: Enum

Supported cloud providers for multi-cloud optimization.

CostOptimization `dataclass` ¶

CostOptimization(current_cost_usd_per_hour: float, optimized_cost_usd_per_hour: float, potential_savings_percentage: float, recommendations: list[str], alternative_configurations: list[dict[str, Any]], cost_breakdown_by_provider: dict[CloudProvider, float], roi_analysis: dict[str, float])

Cost optimization analysis and recommendations.

OptimizationObjective ¶

Bases: Enum

Optimization objectives for resource allocation.

ResourceAllocation `dataclass` ¶

ResourceAllocation(allocation_id: str, requested_resources: dict[ResourceType, int], allocated_resources: dict[str, ResourcePool], start_time: float, end_time: float | None, cost_estimate_usd: float, performance_estimate: float, carbon_footprint_kg: float, allocation_strategy: str)

Resource allocation request and result.

ResourcePool `dataclass` ¶

ResourcePool(pool_id: str, provider: CloudProvider, region: str, resource_type: ResourceType, total_capacity: int, available_capacity: int, reserved_capacity: int, cost_per_hour_usd: float, performance_score: float, carbon_efficiency: float, availability_sla: float, current_utilization: float = 0.0, maintenance_window: str = '02:00-04:00 UTC')

Resource pool configuration and status.

ResourceType ¶

Bases: Enum

Types of computational resources.

SustainabilityMetrics `dataclass` ¶

SustainabilityMetrics(total_carbon_footprint_kg: float, carbon_per_compute_unit: float, renewable_energy_percentage: float, carbon_offset_cost_usd: float, sustainability_score: float, green_computing_recommendations: list[str])

Sustainability and carbon footprint metrics.

Intelligent Edge Network¶

Global edge computing with sub-millisecond latency optimization.

Intelligent Edge Network for Opifex production optimization.

This module implements global distribution with sub-millisecond latency optimization, edge caching, and regional failover for the Version 7.4 Production Optimization system.

Part of: Hybrid Performance Platform + Intelligent Edge + Adaptive Optimization

EdgeRegion ¶

Bases: Enum

Global edge regions for intelligent distribution.

FailoverStrategy ¶

Bases: Enum

Failover strategies for edge network.

EdgeNodeMetrics `dataclass` ¶

EdgeNodeMetrics(node_id: str, region: EdgeRegion, latency_ms: float, throughput_rps: float, cpu_utilization: float, memory_utilization: float, gpu_utilization: float, error_rate: float, cache_hit_ratio: float, bandwidth_mbps: float, concurrent_connections: int, health_score: float, last_update: float = time())

Performance metrics for edge nodes.

LatencyProfile `dataclass` ¶

LatencyProfile(workload_type: str, target_latency_ms: float, latency_sla_ms: float, latency_percentile: float, optimization_priority: float, geographic_distribution: dict[EdgeRegion, float])

Latency optimization profile for different workload types.

CacheEntry `dataclass` ¶

CacheEntry(cache_key: str, model_hash: str, input_hash: str, cached_result: ndarray, creation_time: float, last_access_time: float, access_count: int, expiry_time: float, size_bytes: int, compression_ratio: float = 1.0)

Cache entry for edge model and result caching.

FailoverResult `dataclass` ¶

FailoverResult(original_region: EdgeRegion, failover_region: EdgeRegion, failover_reason: str, latency_impact_ms: float, success: bool, failover_time_ms: float)

Result of failover operation.

EdgeGateway ¶

EdgeGateway(primary_regions: list[EdgeRegion] | None = None, latency_target_ms: float = 0.5, max_failover_attempts: int = 3)

Bases: Module

Intelligent edge gateway for global request distribution.

register_edge_node ¶

register_edge_node(node_id: str, region: EdgeRegion, initial_metrics: EdgeNodeMetrics) -> bool

Register a new edge node with the gateway.

update_node_metrics ¶

update_node_metrics(node_id: str, metrics: EdgeNodeMetrics) -> bool

Update metrics for an existing edge node.

select_optimal_region ¶

select_optimal_region(client_region: EdgeRegion, workload_profile: LatencyProfile, required_capacity: float) -> EdgeRegion

Select optimal edge region for request routing.

LatencyOptimizer ¶

LatencyOptimizer(target_latency_ms: float = 0.5, optimization_window_seconds: int = 60, learning_rate: float = 0.001, *, rngs: Rngs)

Bases: Module

Sub-millisecond latency optimizer for edge distribution.

predict_route_latency ¶

predict_route_latency(route_features: ndarray) -> ndarray

Predict latency for given route configuration.

optimize_route_selection ¶

optimize_route_selection(available_routes: ndarray) -> ndarray

Optimize route selection for minimal latency.

adaptive_latency_optimization ¶

adaptive_latency_optimization(current_latencies: ndarray, target_latency: float, route_options: ndarray) -> dict[str, Any]

Perform adaptive latency optimization.

EdgeCache ¶

EdgeCache(max_cache_size_gb: float = 10.0, max_entries: int = 10000, ttl_seconds: int = 3600, compression_threshold_mb: float = 100.0)

High-performance edge cache for models and results.

generate_cache_key ¶

generate_cache_key(model_hash: str, input_hash: str, parameters: dict[str, Any] | None = None) -> str

Generate cache key for model inference.

put ¶

put(model_hash: str, input_hash: str, result: ndarray, parameters: dict[str, Any] | None = None) -> bool

Store result in cache.

get ¶

get(model_hash: str, input_hash: str, parameters: dict[str, Any] | None = None) -> ndarray | None

Retrieve result from cache.

get_cache_statistics ¶

get_cache_statistics() -> dict[str, Any]

Get cache performance statistics.

RegionalFailover ¶

RegionalFailover(edge_gateway: EdgeGateway, failover_strategy: FailoverStrategy = LOWEST_LATENCY, health_check_interval: float = 10.0, failover_threshold: float = 0.5)

Regional failover system for edge network resilience.

start_health_monitoring `async` ¶

start_health_monitoring() -> None

Start continuous health monitoring for failover detection.

stop_health_monitoring `async` ¶

stop_health_monitoring() -> None

Stop health monitoring.

IntelligentEdgeNetwork ¶

IntelligentEdgeNetwork(edge_gateway: EdgeGateway, latency_optimizer: LatencyOptimizer, edge_cache: EdgeCache, regional_failover: RegionalFailover, target_latency_ms: float = 0.5)

Main orchestrator for intelligent edge network with global distribution.

process_inference_request `async` ¶

process_inference_request(model_hash: str, input_data: ndarray, client_region: EdgeRegion, workload_profile: LatencyProfile, parameters: dict[str, Any] | None = None) -> dict[str, Any]

Process inference request with intelligent edge routing and caching.

get_network_statistics ¶

get_network_statistics() -> dict[str, Any]

Get full network performance statistics.

start_edge_services `async` ¶

start_edge_services() -> None

Start all edge network services.

stop_edge_services `async` ¶

stop_edge_services() -> None

Stop all edge network services.

Scientific Computing Integration¶

Physics-aware optimization with scientific validation and benchmarking.

Scientific computing integration for Opifex production optimization.

This module implements physics-informed optimization, numerical validation, and conservation checking for the Version 7.4 Production Optimization system.

Part of: Hybrid Performance Platform + Intelligent Edge + Adaptive Optimization

PhysicsDomain ¶

Bases: Enum

Scientific computing domains.

PhysicsMetrics `dataclass` ¶

PhysicsMetrics(domain: PhysicsDomain, conservation_violations: dict[ConservationLaw, float] = dict(), symmetry_preservation: float = 0.0, numerical_stability: float = 0.0, energy_conservation_error: float = 0.0, momentum_conservation_error: float = 0.0, mass_conservation_error: float = 0.0, unitarity_preservation: float = 0.0, thermodynamic_consistency: float = 0.0, boundary_condition_accuracy: float = 0.0)

Physics-specific performance metrics.

NumericalValidationResult `dataclass` ¶

NumericalValidationResult(is_valid: bool, precision_score: float, stability_score: float, convergence_rate: float, condition_number: float, validation_errors: list[str] = list(), recommendations: list[str] = list())

Result of numerical validation.

ConservationCheckResult `dataclass` ¶

ConservationCheckResult(law: ConservationLaw, is_conserved: bool, violation_magnitude: float, tolerance: float, relative_error: float, time_evolution_consistency: bool = True)

Result of conservation law checking.

ScientificBenchmarkResult `dataclass` ¶

ScientificBenchmarkResult(benchmark_name: str, domain: PhysicsDomain, accuracy_score: float, reference_value: float, computed_value: float, relative_error: float, meets_accuracy_threshold: bool, chemical_accuracy: bool = False)

Result of scientific benchmark validation.

PhysicsProfilerProtocol ¶

Bases: Protocol

Protocol for physics profiling implementations.

profile_physics_metrics ¶

profile_physics_metrics(model_output: ndarray, reference_data: dict[str, Any]) -> PhysicsMetrics

Profile physics-specific metrics.

validate_domain_constraints ¶

validate_domain_constraints(model_output: ndarray, domain: PhysicsDomain) -> bool

Validate domain-specific constraints.

PhysicsProfiler ¶

PhysicsProfiler(domain: PhysicsDomain, validation_tolerances: dict[str, float] | None = None)

Physics-informed profiler for domain-specific optimization.

profile_physics_metrics ¶

profile_physics_metrics(model_output: ndarray, reference_data: dict[str, Any], time_series: list[ndarray] | None = None) -> PhysicsMetrics

Profile full physics-specific metrics.

NumericalValidator ¶

NumericalValidator(precision_threshold: float = 1e-06, stability_threshold: float = 0.001)

Numerical precision and stability validator.

validate_numerical_precision ¶

validate_numerical_precision(computed_values: ndarray, reference_values: ndarray) -> NumericalValidationResult

Validate numerical precision against reference values.

check_conservation_law ¶

check_conservation_law(computed_quantity: ndarray, reference_quantity: ndarray, law: ConservationLaw, tolerance: float | None = None) -> ConservationCheckResult

Check specific conservation law.

ScientificBenchmarkValidator ¶

ScientificBenchmarkValidator(domain: PhysicsDomain)

Validator for scientific computing benchmarks.

validate_benchmark ¶

validate_benchmark(benchmark_name: str, computed_value: float, reference_value: float, accuracy_type: str = 'relative_error') -> ScientificBenchmarkResult

Validate against a specific benchmark.

validate_multiple_benchmarks ¶

validate_multiple_benchmarks(benchmarks: dict[str, tuple[float, float]]) -> list[ScientificBenchmarkResult]

Validate multiple benchmarks.

ScientificComputingIntegrator ¶

ScientificComputingIntegrator(domain: PhysicsDomain, physics_profiler: PhysicsProfiler | None = None, numerical_validator: NumericalValidator | None = None, benchmark_validator: ScientificBenchmarkValidator | None = None)

Main integrator for scientific computing optimization.

comprehensive_scientific_validation ¶

comprehensive_scientific_validation(model_output: ndarray, reference_data: dict[str, Any], benchmarks: dict[str, tuple[float, float]] | None = None, time_series: list[ndarray] | None = None) -> dict[str, Any]

Perform full scientific validation.

optimize_for_scientific_accuracy ¶

optimize_for_scientific_accuracy(model_output: ndarray, validation_results: dict[str, Any]) -> dict[str, Any]

Generate optimization recommendations based on scientific validation.

Learn-to-Optimize (L2O) Algorithms¶

Advanced neural optimization methods that achieve significant speedups on learned problem families.

L2O Engine¶

Core learn-to-optimize engine with parametric optimization solvers.

Learn-to-Optimize (L2O) Engine for unified optimization strategies.

This module implements a unified L2O engine that integrates parametric programming solvers with existing gradient-based meta-optimization algorithms, providing a full optimization framework for scientific computing applications.

Key Features: - Unified interface for parametric and gradient-based optimization - Automatic algorithm selection based on problem characteristics - Integration with existing MetaOptimizer framework - Performance comparison and benchmarking capabilities - Meta-learning across related optimization problems

L2OEngineConfig `dataclass` ¶

L2OEngineConfig(solver_type: str = 'parametric', problem_encoder_layers: list[int] | None = None, use_traditional_fallback: bool = True, enable_meta_learning: bool = True, integration_mode: str = 'unified', speedup_threshold: float = 100.0, performance_tracking: bool = True, adaptive_selection: bool = True)

Configuration for the L2O engine integration.

This configuration controls how parametric solvers integrate with the existing meta-optimization framework.

OptimizationProblemEncoder ¶

OptimizationProblemEncoder(input_dim: int, output_dim: int, hidden_layers: list[int], *, rngs: Rngs)

Bases: Module

Neural network encoder for optimization problem representations.

This encoder transforms optimization problem specifications and parameters into dense embeddings that can be processed by neural optimization algorithms.

Parameters:

Name	Type	Description	Default
`input_dim`	`int`	Dimension of input problem parameters	required
`output_dim`	`int`	Dimension of output encoding	required
`hidden_layers`	`list[int]`	Hidden layer dimensions	required
`rngs`	`Rngs`	Random number generators	required

encode_problem ¶

encode_problem(problem: OptimizationProblem, problem_params: Array) -> Array

Encode a single optimization problem.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem specification	required
`problem_params`	`Array`	Problem parameters (objective coefficients, etc.)	required

Returns:

Type	Description
`Array`	Dense encoding of the optimization problem

encode_problem_batch ¶

encode_problem_batch(problems: list[OptimizationProblem], problem_params_batch: Array) -> Array

Encode a batch of optimization problems.

Parameters:

Name	Type	Description	Default
`problems`	`list[OptimizationProblem]`	List of optimization problem specifications	required
`problem_params_batch`	`Array`	Batch of problem parameters	required

Returns:

Type	Description
`Array`	Batch of problem encodings

ParametricOptimizationSolver ¶

ParametricOptimizationSolver(solver_config: SolverConfig, l2o_config: L2OEngineConfig, input_dim: int, output_dim: int, *, rngs: Rngs)

Bases: Module

Integrated parametric optimization solver with L2O engine capabilities.

This class combines the parametric programming solver with problem encoding and performance measurement capabilities for integration with the L2O engine.

Parameters:

Name	Type	Description	Default
`solver_config`	`SolverConfig`	Configuration for parametric solver	required
`l2o_config`	`L2OEngineConfig`	Configuration for L2O engine integration	required
`input_dim`	`int`	Input dimension for problem parameters	required
`output_dim`	`int`	Output dimension for optimization variables	required
`rngs`	`Rngs`	Random number generators	required

solve_optimization_problem ¶

solve_optimization_problem(problem: OptimizationProblem, problem_params: Array, enable_fallback: bool = True) -> tuple[Array, dict[str, Any]]

Solve optimization problem end-to-end.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem specification	required
`problem_params`	`Array`	Problem parameters	required
`enable_fallback`	`bool`	Enable traditional solver fallback	`True`

Returns:

Type	Description
`tuple[Array, dict[str, Any]]`	Tuple of (solution, metadata)

measure_performance ¶

measure_performance(problem: OptimizationProblem, problem_params: Array) -> dict[str, Any]

Measure performance compared to traditional methods.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem	required
`problem_params`	`Array`	Problem parameters	required

Returns:

Type	Description
`dict[str, Any]`	Performance measurement results

L2OEngine ¶

L2OEngine(l2o_config: L2OEngineConfig, meta_config: MetaOptimizerConfig, *, rngs: Rngs)

Unified Learn-to-Optimize engine integrating multiple optimization strategies.

This engine provides a unified interface for parametric and gradient-based optimization, with automatic algorithm selection and meta-learning capabilities.

Parameters:

Name	Type	Description	Default
`l2o_config`	`L2OEngineConfig`	L2O engine configuration	required
`meta_config`	`MetaOptimizerConfig`	Meta-optimizer configuration	required
`rngs`	`Rngs`	Random number generators	required

solve_parametric_problem ¶

solve_parametric_problem(problem: OptimizationProblem, problem_params: Array) -> Array

Solve optimization problem using parametric solver.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem specification	required
`problem_params`	`Array`	Problem parameters	required

Returns:

Type	Description
`Array`	Optimization solution

solve_gradient_problem ¶

solve_gradient_problem(loss_fn: Callable[[Array], Array], initial_params: Array, steps: int = 100) -> Array

Solve optimization problem using gradient-based L2O.

Parameters:

Name	Type	Description	Default
`loss_fn`	`Callable[[Array], Array]`	Loss function to minimize	required
`initial_params`	`Array`	Initial parameters	required
`steps`	`int`	Number of optimization steps	`100`

Returns:

Type	Description
`Array`	Optimized parameters

solve_automatically ¶

solve_automatically(problem: OptimizationProblem, problem_params: Array) -> tuple[str, Array]

Automatically select and apply best optimization algorithm.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem	required
`problem_params`	`Array`	Problem parameters	required

Returns:

Type	Description
`tuple[str, Array]`	Tuple of (algorithm_used, solution)

solve_with_meta_learning ¶

solve_with_meta_learning(problem: OptimizationProblem, problem_params: Array, problem_id: int = 0) -> tuple[Array, dict[str, Any]]

Solve optimization problem with meta-learning.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem	required
`problem_params`	`Array`	Problem parameters	required
`problem_id`	`int`	Problem identifier for tracking	`0`

Returns:

Type	Description
`tuple[Array, dict[str, Any]]`	Tuple of (solution, metadata)

compare_all_solvers ¶

compare_all_solvers(problem: OptimizationProblem, problem_params: Array) -> dict[str, dict[str, Any]]

Compare performance of all available solvers.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem	required
`problem_params`	`Array`	Problem parameters	required

Returns:

Type	Description
`dict[str, dict[str, Any]]`	Comparison results for each solver

recommend_algorithm ¶

recommend_algorithm(problem: OptimizationProblem, problem_params: Array) -> str

Recommend best algorithm for given problem.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem	required
`problem_params`	`Array`	Problem parameters	required

Returns:

Type	Description
`str`	Recommended algorithm name

solve_with_recommendation ¶

solve_with_recommendation(problem: OptimizationProblem, problem_params: Array) -> Array

Solve using recommended algorithm.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem	required
`problem_params`	`Array`	Problem parameters	required

Returns:

Type	Description
`Array`	Solution using recommended algorithm

optimize_with_meta_framework ¶

optimize_with_meta_framework(loss_fn: Callable[[Array], Array], initial_params: Array, steps: int = 50) -> tuple[Array, list[dict[str, Any]]]

Optimize using integrated meta-framework.

Parameters:

Name	Type	Description	Default
`loss_fn`	`Callable[[Array], Array]`	Loss function to minimize	required
`initial_params`	`Array`	Initial parameters	required
`steps`	`int`	Number of optimization steps	`50`

Returns:

Type	Description
`tuple[Array, list[dict[str, Any]]]`	Tuple of (final_params, optimization_history)

solve_physics_informed ¶

solve_physics_informed(physics_loss_fn: Callable[[Array], Array], initial_params: Array, steps: int = 100) -> Array

Solve physics-informed optimization problems.

Parameters:

Name	Type	Description	Default
`physics_loss_fn`	`Callable[[Array], Array]`	Physics-informed loss function	required
`initial_params`	`Array`	Initial parameters	required
`steps`	`int`	Number of optimization steps	`100`

Returns:

Type	Description
`Array`	Physics-informed solution

Advanced Meta-Learning¶

MAML, Reptile, and gradient-based meta-learning approaches.

Advanced Learn-to-Optimize (L2O) Meta-Learning Algorithms.

This module implements advanced meta-learning algorithms for optimization including MAML (Model-Agnostic Meta-Learning), Reptile, and gradient-based meta-learning strategies. These algorithms enable few-shot adaptation to new optimization problems and self-improving optimization capabilities.

Key Features: - MAML for few-shot optimization adaptation - Reptile algorithm for first-order meta-learning - Gradient-based meta-learning for parameter initialization - Meta-L2O integration for self-improving optimization - Integration with existing L2O engine and MetaOptimizer framework

MAMLConfig `dataclass` ¶

MAMLConfig(inner_learning_rate: float = 0.001, meta_learning_rate: float = 0.0001, inner_steps: int = 5, meta_batch_size: int = 8, adaptation_steps: int = 10, second_order: bool = True, enable_adaptation_rate_learning: bool = True, task_distribution_diversity: float = 0.8, convergence_tolerance: float = 1e-06)

Configuration for Model-Agnostic Meta-Learning (MAML) optimization.

MAML enables few-shot adaptation to new optimization problems by learning parameter initializations that allow rapid adaptation with gradient descent.

ReptileConfig `dataclass` ¶

ReptileConfig(meta_learning_rate: float = 0.001, inner_learning_rate: float = 0.01, inner_steps: int = 10, meta_batch_size: int = 16, adaptation_momentum: float = 0.9, task_sampling_strategy: str = 'uniform', gradient_clipping: float = 1.0, convergence_patience: int = 5)

Configuration for Reptile meta-learning algorithm.

Reptile is a first-order meta-learning algorithm that is simpler than MAML but still effective for few-shot learning by moving towards parameters that work well on task-specific optimization.

GradientBasedMetaLearningConfig `dataclass` ¶

GradientBasedMetaLearningConfig(optimizer_network_layers: list[int] | None = None, meta_learning_rate: float = 0.0001, gradient_unroll_steps: int = 20, learned_lr_bounds: tuple[float, float] = (1e-06, 1.0), momentum_adaptation: bool = True, curvature_adaptation: bool = False, problem_conditioning: bool = True, numerical_stability_epsilon: float = 1e-08)

Configuration for gradient-based meta-learning strategies.

This includes various gradient-based approaches for learning optimization strategies including learned optimizers and meta-gradient methods.

MAMLOptimizer ¶

MAMLOptimizer(config: MAMLConfig, l2o_config: L2OEngineConfig, optimizer_input_dim: int, optimizer_output_dim: int, *, rngs: Rngs)

Bases: Module

Model-Agnostic Meta-Learning (MAML) optimizer for L2O.

MAML learns parameter initializations that enable rapid adaptation to new optimization problems with just a few gradient steps. This implementation integrates with the existing L2O framework.

Parameters:

Name	Type	Description	Default
`config`	`MAMLConfig`	MAML configuration	required
`l2o_config`	`L2OEngineConfig`	L2O engine configuration for integration	required
`optimizer_input_dim`	`int`	Input dimension for the meta-optimizer	required
`optimizer_output_dim`	`int`	Output dimension for optimization parameters	required
`rngs`	`Rngs`	Random number generators	required

meta_learn_on_task_distribution ¶

meta_learn_on_task_distribution(task_distribution: list[tuple[OptimizationProblem, Array]], meta_optimizer_state: Any, meta_step: int) -> tuple[dict[str, Any], float]

Perform MAML meta-learning on a distribution of optimization tasks.

Parameters:

Name	Type	Description	Default
`task_distribution`	`list[tuple[OptimizationProblem, Array]]`	List of (problem, parameters) tuples for meta-learning	required
`meta_optimizer_state`	`Any`	Current meta-optimizer state	required
`meta_step`	`int`	Current meta-learning step	required

Returns:

Type	Description
`tuple[dict[str, Any], float]`	Updated meta-optimizer state and meta-loss

adapt_to_new_task ¶

adapt_to_new_task(problem: OptimizationProblem, problem_params: Array, adaptation_steps: int | None = None) -> Array

Rapidly adapt to a new optimization task using learned initialization.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	New optimization problem to adapt to	required
`problem_params`	`Array`	Parameters for the new problem	required
`adaptation_steps`	`int \| None`	Number of adaptation steps (uses config default if None)	`None`

Returns:

Type	Description
`Array`	Adapted parameters for the new task

ReptileOptimizer ¶

ReptileOptimizer(config: ReptileConfig, l2o_config: L2OEngineConfig, optimizer_input_dim: int, optimizer_output_dim: int, *, rngs: Rngs)

Bases: Module

Reptile meta-learning optimizer for L2O.

Reptile is a first-order meta-learning algorithm that learns parameter initializations by repeatedly sampling tasks, taking gradient steps on each task, and moving the initialization towards the adapted parameters.

Parameters:

Name	Type	Description	Default
`config`	`ReptileConfig`	Reptile configuration	required
`l2o_config`	`L2OEngineConfig`	L2O engine configuration for integration	required
`optimizer_input_dim`	`int`	Input dimension for problem characteristics	required
`optimizer_output_dim`	`int`	Output dimension for optimization parameters	required
`rngs`	`Rngs`	Random number generators	required

meta_learn_reptile_step ¶

meta_learn_reptile_step(task_distribution: list[tuple[OptimizationProblem, Array]], meta_step: int) -> dict[str, Any]

Perform one Reptile meta-learning step.

Parameters:

Name	Type	Description	Default
`task_distribution`	`list[tuple[OptimizationProblem, Array]]`	List of optimization tasks for meta-learning	required
`meta_step`	`int`	Current meta-learning step	required

Returns:

Type	Description
`dict[str, Any]`	Dictionary with meta-learning metrics

adapt_to_task ¶

adapt_to_task(problem: OptimizationProblem, problem_params: Array, adaptation_steps: int) -> Array

Adapt meta-parameters to a specific task using Reptile approach.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem to adapt to	required
`problem_params`	`Array`	Problem parameters	required
`adaptation_steps`	`int`	Number of adaptation steps	required

Returns:

Type	Description
`Array`	Task-adapted parameters

GradientBasedMetaLearner ¶

GradientBasedMetaLearner(config: GradientBasedMetaLearningConfig, l2o_config: L2OEngineConfig, problem_dim: int, *, rngs: Rngs)

Bases: Module

Gradient-based meta-learning for learning optimization strategies.

This class implements neural networks that learn to optimize by processing gradients and optimization histories to produce effective parameter updates.

Parameters:

Name	Type	Description	Default
`config`	`GradientBasedMetaLearningConfig`	Configuration for gradient-based meta-learning	required
`l2o_config`	`L2OEngineConfig`	L2O engine configuration	required
`problem_dim`	`int`	Dimension of optimization problems	required
`rngs`	`Rngs`	Random number generators	required

compute_learned_update ¶

compute_learned_update(gradients: Array, previous_update: Array, loss_history: Array, problem_features: Array, step: int) -> tuple[Array, dict[str, Any]]

Compute parameter update using learned optimization strategy.

Parameters:

Name	Type	Description	Default
`gradients`	`Array`	Current gradients	required
`previous_update`	`Array`	Previous parameter update	required
`loss_history`	`Array`	History of loss values	required
`problem_features`	`Array`	Problem-specific features	required
`step`	`int`	Current optimization step	required

Returns:

Type	Description
`tuple[Array, dict[str, Any]]`	Parameter update and optimization metrics

meta_train_on_optimization_trajectories ¶

meta_train_on_optimization_trajectories(optimization_trajectories: list[dict[str, Any]], meta_optimizer_state: Any) -> tuple[Any, float]

Meta-train the learned optimizer on optimization trajectories.

Parameters:

Name	Type	Description	Default
`optimization_trajectories`	`list[dict[str, Any]]`	List of optimization trajectories for training	required
`meta_optimizer_state`	`Any`	Current meta-optimizer state	required

Returns:

Type	Description
`tuple[Any, float]`	Updated meta-optimizer state and meta-loss

MetaL2OIntegration ¶

MetaL2OIntegration(l2o_engine: L2OEngine, maml_config: MAMLConfig | None = None, reptile_config: ReptileConfig | None = None, gb_config: GradientBasedMetaLearningConfig | None = None, *, rngs: Rngs)

Bases: Module

Meta-L2O Integration for self-improving optimization framework.

This class integrates MAML, Reptile, and gradient-based meta-learning with the existing L2O engine to create a self-improving optimization framework that learns from optimization experience.

Parameters:

Name	Type	Description	Default
`l2o_engine`	`L2OEngine`	Existing L2O engine to enhance	required
`maml_config`	`MAMLConfig \| None`	Configuration for MAML (optional)	`None`
`reptile_config`	`ReptileConfig \| None`	Configuration for Reptile (optional)	`None`
`gb_config`	`GradientBasedMetaLearningConfig \| None`	Configuration for gradient-based meta-learning (optional)	`None`
`rngs`	`Rngs`	Random number generators	required

solve_with_meta_learning ¶

solve_with_meta_learning(problem: OptimizationProblem, problem_params: Array, meta_learning_strategy: str = 'auto') -> tuple[Array, dict[str, Any]]

Solve optimization problem using meta-learning enhanced L2O.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem to solve	required
`problem_params`	`Array`	Problem parameters	required
`meta_learning_strategy`	`str`	Strategy to use ('maml', 'reptile', 'gradient_based', 'auto')	`'auto'`

Returns:

Type	Description
`tuple[Array, dict[str, Any]]`	Solution and optimization metrics

trigger_meta_learning_update ¶

trigger_meta_learning_update(task_distribution: list[tuple[OptimizationProblem, Array]]) -> dict[str, Any]

Trigger meta-learning updates across all enabled algorithms.

Parameters:

Name	Type	Description	Default
`task_distribution`	`list[tuple[OptimizationProblem, Array]]`	Distribution of tasks for meta-learning	required

Returns:

Type	Description
`dict[str, Any]`	Meta-learning results and metrics

Adaptive Schedulers¶

Bayesian and performance-aware scheduling algorithms.

Adaptive Learning Rate Schedulers for L2O Framework.

This module implements intelligent learning rate adaptation strategies that enhance the Learn-to-Optimize framework with performance-aware, multiscale, and Bayesian optimization capabilities for optimal scheduler parameter selection.

Key Features: - Performance-aware scheduling based on convergence detection - Multiscale scheduling for different network components - Bayesian optimization for automatic parameter tuning - Seamless integration with existing L2O framework - >50% improvement in convergence speed through adaptive scheduling

MetaSchedulerConfig `dataclass` ¶

MetaSchedulerConfig(base_learning_rate: float = 0.001, min_learning_rate: float = 1e-06, max_learning_rate: float = 0.1, convergence_window: int = 10, patience: int = 5, adaptation_factor: float = 0.5, multiscale_components: list[str] | None = None, bayesian_optimization_steps: int = 20, enable_performance_awareness: bool = True, enable_multiscale: bool = False, enable_bayesian_optimization: bool = False)

Configuration for adaptive learning rate schedulers.

This configuration controls all aspects of adaptive scheduling including performance awareness, multiscale adaptation, and Bayesian optimization.

PerformanceAwareScheduler ¶

PerformanceAwareScheduler(config: MetaSchedulerConfig, *, rngs: Rngs)

Bases: Module

Performance-aware learning rate scheduler based on convergence detection.

This scheduler monitors optimization progress and adapts learning rates based on loss improvement patterns, convergence detection, and stagnation handling.

Parameters:

Name	Type	Description	Default
`config`	`MetaSchedulerConfig`	Scheduler configuration	required
`rngs`	`Rngs`	Random number generators	required

update_learning_rate ¶

update_learning_rate(loss: float) -> float

Update learning rate based on current loss.

Parameters:

Name	Type	Description	Default
`loss`	`float`	Current optimization loss	required

Returns:

Type	Description
`float`	Updated learning rate

is_converged ¶

is_converged() -> bool

Check if optimization has converged based on loss variance.

Returns:

Type	Description
`bool`	True if converged, False otherwise

reset ¶

reset()

Reset scheduler state.

MultiscaleScheduler ¶

MultiscaleScheduler(config: MetaSchedulerConfig, *, rngs: Rngs)

Bases: Module

Multiscale learning rate scheduler for different network components.

This scheduler maintains separate learning rates for different components of the neural network, allowing fine-grained control over optimization.

Parameters:

Name	Type	Description	Default
`config`	`MetaSchedulerConfig`	Scheduler configuration	required
`rngs`	`Rngs`	Random number generators	required

get_component_learning_rates ¶

get_component_learning_rates() -> dict[str, float]

Get current learning rates for all components.

Returns:

Type	Description
`dict[str, float]`	Dictionary mapping component names to learning rates

update_component_learning_rate ¶

update_component_learning_rate(component: str, loss: float) -> float

Update learning rate for a specific component.

Parameters:

Name	Type	Description	Default
`component`	`str`	Component name	required
`loss`	`float`	Current loss for this component	required

Returns:

Type	Description
`float`	Updated learning rate for the component

create_component_optimizers ¶

create_component_optimizers() -> dict[str, GradientTransformation]

Create optimizers for all components with current learning rates.

Returns:

Type	Description
`dict[str, GradientTransformation]`	Dictionary mapping component names to optimizers

reset_all_components ¶

reset_all_components()

Reset all component schedulers.

BayesianSchedulerOptimizer ¶

BayesianSchedulerOptimizer(config: MetaSchedulerConfig, *, rngs: Rngs)

Bases: Module

Bayesian optimization for automatic scheduler parameter tuning.

This scheduler uses Bayesian optimization to automatically discover optimal scheduler parameters based on optimization performance feedback.

Parameters:

Name	Type	Description	Default
`config`	`MetaSchedulerConfig`	Scheduler configuration	required
`rngs`	`Rngs`	Random number generators	required

suggest_scheduler_parameters ¶

suggest_scheduler_parameters() -> dict[str, float]

Suggest scheduler parameters using Bayesian optimization.

Returns:

Type	Description
`dict[str, float]`	Dictionary of suggested scheduler parameters

update_with_performance ¶

update_with_performance(parameters: dict[str, float], performance: float)

Update with performance feedback for given parameters.

Parameters:

Name	Type	Description	Default
`parameters`	`dict[str, float]`	Scheduler parameters used	required
`performance`	`float`	Performance achieved (lower is better)	required

get_best_parameters ¶

get_best_parameters() -> dict[str, float]

Get best scheduler parameters from history.

Returns:

Type	Description
`dict[str, float]`	Best scheduler parameters

SchedulerIntegration ¶

SchedulerIntegration(config: MetaSchedulerConfig, *, rngs: Rngs)

Bases: Module

Integration class for adaptive schedulers with L2O framework.

This class coordinates all adaptive scheduling strategies and provides a unified interface for integration with the existing L2O framework.

Parameters:

Name	Type	Description	Default
`config`	`MetaSchedulerConfig`	Scheduler configuration	required
`rngs`	`Rngs`	Random number generators	required

create_adaptive_optimizer ¶

create_adaptive_optimizer() -> GradientTransformation

Create adaptive optimizer with current learning rates.

Returns:

Type	Description
`GradientTransformation`	Adaptive optimizer

integrate_with_l2o_engine ¶

integrate_with_l2o_engine(l2o_engine: L2OEngine) -> L2OEngine

Integrate adaptive schedulers with L2O engine.

Parameters:

Name	Type	Description	Default
`l2o_engine`	`L2OEngine`	Existing L2O engine	required

Returns:

Type	Description
`L2OEngine`	Enhanced L2O engine with adaptive schedulers

update_schedulers ¶

update_schedulers(step: int, loss: float) -> dict[str, Any]

Update all enabled schedulers with current step and loss.

Parameters:

Name	Type	Description	Default
`step`	`int`	Current optimization step	required
`loss`	`float`	Current loss value	required

Returns:

Type	Description
`dict[str, Any]`	Dictionary of current learning rates from all schedulers

auto_optimize_parameters ¶

auto_optimize_parameters(optimization_function: Callable[[dict[str, float]], float], num_trials: int = 10) -> dict[str, float]

Automatically optimize scheduler parameters using Bayesian optimization.

Parameters:

Name	Type	Description	Default
`optimization_function`	`Callable[[dict[str, float]], float]`	Function that takes scheduler parameters and returns performance (lower is better)	required
`num_trials`	`int`	Number of trials for parameter optimization	`10`

Returns:

Type	Description
`dict[str, float]`	Best scheduler parameters found

reset_all_schedulers ¶

reset_all_schedulers()

Reset all scheduler states.

create_l2o_engine_with_adaptive_schedulers ¶

create_l2o_engine_with_adaptive_schedulers(l2o_config: Any, meta_config: Any, scheduler_config: MetaSchedulerConfig | None = None, *, rngs: Rngs) -> L2OEngine

Create L2O engine with adaptive schedulers.

Parameters:

Name	Type	Description	Default
`l2o_config`	`Any`	L2O engine configuration	required
`meta_config`	`Any`	Meta optimizer configuration	required
`scheduler_config`	`MetaSchedulerConfig \| None`	Adaptive scheduler configuration	`None`
`rngs`	`Rngs`	Random number generators	required

Returns:

Type	Description
`L2OEngine`	L2O engine with adaptive scheduling capabilities

Multi-Objective Optimization¶

Pareto frontier approximation and multi-objective optimization.

Multi-Objective Learn-to-Optimize (L2O) Framework.

This module implements neural network-based multi-objective optimization algorithms that can simultaneously optimize multiple conflicting objectives using learned strategies.

Key Features: - Multi-objective optimization with neural Pareto frontier approximation - Learned scalarization strategies for objective combination - Multi-objective MAML adaptation for different objective combinations - Performance indicators (hypervolume, spread, convergence metrics) - Integration with existing L2O framework for enhanced optimization

MultiObjectiveConfig `dataclass` ¶

MultiObjectiveConfig(num_objectives: int = 2, pareto_points_target: int = 100, scalarization_strategy: str = 'learned', diversity_pressure: float = 0.1, convergence_tolerance: float = 1e-06, max_pareto_iterations: int = 500, hypervolume_reference_point: list[float] | None = None, adaptive_weights: bool = True, dominated_solution_filtering: bool = True)

Configuration for multi-objective L2O optimization.

This configuration defines parameters for simultaneously optimizing multiple conflicting objectives using neural network-based strategies.

ParetoFrontierOptimizer ¶

ParetoFrontierOptimizer(config: MultiObjectiveConfig, problem_dimension: int, *, rngs: Rngs)

Bases: Module

Neural Pareto frontier approximation for multi-objective optimization.

This optimizer learns to approximate the Pareto frontier using neural networks and can generate diverse solutions along the frontier efficiently.

Parameters:

Name	Type	Description	Default
`config`	`MultiObjectiveConfig`	Multi-objective configuration	required
`problem_dimension`	`int`	Dimension of optimization problem	required
`rngs`	`Rngs`	Random number generators	required

generate_pareto_solutions ¶

generate_pareto_solutions(objective_functions: list[Callable[[Array], Array]], preference_vectors: Array | None = None) -> tuple[Array, Array]

Generate diverse solutions along the Pareto frontier.

Parameters:

Name	Type	Description	Default
`objective_functions`	`list[Callable[[Array], Array]]`	List of objective functions to optimize	required
`preference_vectors`	`Array \| None`	Optional preference vectors for guided search	`None`

Returns:

Type	Description
`tuple[Array, Array]`	Tuple of (solutions, objective_values)

optimize_pareto_frontier ¶

optimize_pareto_frontier(objective_functions: list[Callable[[Array], Array]], constraint_function: Callable[[Array], Array] | None = None) -> dict[str, Any]

Optimize the neural network to better approximate Pareto frontier.

Parameters:

Name	Type	Description	Default
`objective_functions`	`list[Callable[[Array], Array]]`	List of objective functions	required
`constraint_function`	`Callable[[Array], Array] \| None`	Optional constraint function	`None`

Returns:

Type	Description
`dict[str, Any]`	Optimization results and metrics

ObjectiveScalarizer ¶

ObjectiveScalarizer(config: MultiObjectiveConfig, problem_features_dim: int, *, rngs: Rngs)

Bases: Module

Learned scalarization strategies for multi-objective optimization.

This module learns optimal ways to combine multiple objectives into single objectives for efficient optimization using neural networks.

Parameters:

Name	Type	Description	Default
`config`	`MultiObjectiveConfig`	Multi-objective configuration	required
`problem_features_dim`	`int`	Dimension of problem feature vectors	required
`rngs`	`Rngs`	Random number generators	required

learn_scalarization_weights ¶

learn_scalarization_weights(problem_features: Array, objective_values_history: Array, performance_feedback: Array) -> Array

Learn optimal scalarization weights based on problem characteristics.

Parameters:

Name	Type	Description	Default
`problem_features`	`Array`	Features describing the optimization problem	required
`objective_values_history`	`Array`	History of objective values achieved	required
`performance_feedback`	`Array`	Feedback on solution quality	required

Returns:

Type	Description
`Array`	Learned scalarization weights

scalarize_objectives ¶

scalarize_objectives(objectives: Array, weights: Array, strategy: str | None = None) -> Array

Convert multiple objectives to single scalar value.

Parameters:

Name	Type	Description	Default
`objectives`	`Array`	Array of objective values	required
`weights`	`Array`	Scalarization weights	required
`strategy`	`str \| None`	Scalarization strategy override	`None`

Returns:

Type	Description
`Array`	Scalar objective value

PerformanceIndicators ¶

Performance indicators for multi-objective optimization quality assessment.

This class provides various metrics to evaluate the quality of Pareto frontier approximations and multi-objective solutions.

compute_hypervolume `staticmethod` ¶

compute_hypervolume(pareto_front: Array, reference_point: Array) -> Array

Compute hypervolume indicator for Pareto front quality.

Parameters:

Name	Type	Description	Default
`pareto_front`	`Array`	Array of Pareto optimal solutions (objectives)	required
`reference_point`	`Array`	Reference point for hypervolume calculation	required

Returns:

Type	Description
`Array`	Hypervolume value

compute_spread_indicator `staticmethod` ¶

compute_spread_indicator(pareto_front: Array) -> Array

Compute spread (diversity) indicator for Pareto front.

Parameters:

Name	Type	Description	Default
`pareto_front`	`Array`	Array of Pareto optimal solutions	required

Returns:

Type	Description
`Array`	Spread indicator value

compute_convergence_indicator `staticmethod` ¶

compute_convergence_indicator(pareto_front: Array, true_pareto_front: Array | None = None) -> Array

Compute convergence indicator measuring closeness to true Pareto front.

Parameters:

Name	Type	Description	Default
`pareto_front`	`Array`	Approximated Pareto front	required
`true_pareto_front`	`Array \| None`	True Pareto front (if known)	`None`

Returns:

Type	Description
`Array`	Convergence indicator value

MultiObjectiveL2OEngine ¶

MultiObjectiveL2OEngine(config: MultiObjectiveConfig, l2o_engine: L2OEngine, problem_dimension: int, *, rngs: Rngs)

Bases: Module

Core multi-objective L2O optimization engine.

This engine integrates Pareto frontier optimization, learned scalarization, and performance assessment for full multi-objective optimization.

Parameters:

Name	Type	Description	Default
`config`	`MultiObjectiveConfig`	Multi-objective configuration	required
`l2o_engine`	`L2OEngine`	Base L2O engine for single-objective optimization	required
`problem_dimension`	`int`	Dimension of optimization problems	required
`rngs`	`Rngs`	Random number generators	required

solve_multi_objective_problem ¶

solve_multi_objective_problem(objective_functions: list[Callable[[Array], Array]], problem_features: Array, constraint_function: Callable[[Array], Array] | None = None, true_pareto_front: Array | None = None) -> dict[str, Any]

Solve multi-objective optimization problem using L2O strategies.

Parameters:

Name	Type	Description	Default
`objective_functions`	`list[Callable[[Array], Array]]`	List of objective functions to optimize	required
`problem_features`	`Array`	Features characterizing the problem	required
`constraint_function`	`Callable[[Array], Array] \| None`	Optional constraint function	`None`
`true_pareto_front`	`Array \| None`	Optional true Pareto front for evaluation	`None`

Returns:

Type	Description
`dict[str, Any]`	Full optimization results and metrics

solve_with_preference ¶

solve_with_preference(objective_functions: list[Callable[[Array], Array]], preference_vector: Array, problem_features: Array) -> tuple[Array, dict[str, Any]]

Solve multi-objective problem with user preference vector.

Parameters:

Name	Type	Description	Default
`objective_functions`	`list[Callable[[Array], Array]]`	List of objective functions	required
`preference_vector`	`Array`	User preference weights for objectives	required
`problem_features`	`Array`	Problem characterization features	required

Returns:

Type	Description
`tuple[Array, dict[str, Any]]`	Solution and optimization metrics

Parametric Solvers¶

Neural networks for parametric programming and constraint satisfaction.

Parametric Programming Solver Network for Learn-to-Optimize (L2O).

This module implements neural network-based optimization algorithms that learn to solve families of optimization problems with significant speedup over traditional methods.

Key Features: - Neural networks for parametric optimization problems - Support for quadratic, linear, and nonlinear programming - Constraint handling through penalty and barrier methods - >100x speedup over traditional solvers on learned families - Integration with Optimistix for traditional solver fallback

OptimizationProblem `dataclass` ¶

OptimizationProblem(problem_type: str, dimension: int, constraints: dict[str, Any] | None = None)

Represents an optimization problem with type, dimension, and constraints.

This class encapsulates the mathematical specification of optimization problems that can be solved by the parametric solver network.

ConstraintHandler ¶

ConstraintHandler(method: str = 'penalty', penalty_weight: float = 1.0, barrier_parameter: float = 0.1)

Handles constraint satisfaction through penalty, barrier, and projection methods.

This class implements various constraint handling techniques for optimization problems with equality and inequality constraints.

Parameters:

Name	Type	Description	Default
`method`	`str`	Constraint handling method ("penalty", "barrier", "projection")	`'penalty'`
`penalty_weight`	`float`	Weight for penalty method	`1.0`
`barrier_parameter`	`float`	Parameter for barrier method	`0.1`

compute_penalty ¶

compute_penalty(x: Array, constraint: Array, constraint_type: str = 'equality') -> Array

Compute penalty for constraint violation using vectorized operations.

Parameters:

Name	Type	Description	Default
`x`	`Array`	Decision variables (can be batched)	required
`constraint`	`Array`	Constraint coefficients	required
`constraint_type`	`str`	Type of constraint ("equality" or "inequality")	`'equality'`

Returns:

Type	Description
`Array`	Penalty value for constraint violation

compute_barrier ¶

compute_barrier(x: Array, constraint: Array) -> Array

Compute barrier function for inequality constraints.

Parameters:

Name	Type	Description	Default
`x`	`Array`	Decision variables (can be batched)	required
`constraint`	`Array`	Constraint coefficients for g(x) >= 0	required

Returns:

Type	Description
`Array`	Barrier function value

project_to_feasible ¶

project_to_feasible(x: Array, bounds: tuple[float, float]) -> Array

Project variables to feasible region (box constraints).

Parameters:

Name	Type	Description	Default
`x`	`Array`	Decision variables to project	required
`bounds`	`tuple[float, float]`	(lower_bound, upper_bound) for box constraints	required

Returns:

Type	Description
`Array`	Projected variables within bounds

SolverConfig `dataclass` ¶

SolverConfig(hidden_sizes: list | None = None, activation: Callable = gelu, learning_rate: float = 0.001, max_iterations: int = 1000, tolerance: float = 1e-06, use_traditional_fallback: bool = True)

Configuration for parametric programming solver network.

This dataclass contains all hyperparameters and settings for the neural network-based optimization solver.

ParametricProgrammingSolver ¶

ParametricProgrammingSolver(config: SolverConfig, input_dim: int, output_dim: int, *, rngs: Rngs)

Bases: Module

Neural network-based parametric programming solver.

This class implements a neural network that learns to solve families of optimization problems with significant speedup over traditional methods.

The architecture consists of: - Encoder network: Maps problem parameters to latent representation - Decoder network: Maps latent representation to optimization solutions - Constraint handler: Ensures solution feasibility

Parameters:

Name	Type	Description	Default
`config`	`SolverConfig`	Solver configuration with hyperparameters	required
`input_dim`	`int`	Dimension of input problem parameters	required
`output_dim`	`int`	Dimension of output optimization variables	required
`rngs`	`Rngs`	Random number generators for initialization	required

measure_speedup ¶

measure_speedup(traditional_time: float, neural_time: float) -> float

Measure speedup compared to traditional optimization methods.

Parameters:

Name	Type	Description	Default
`traditional_time`	`float`	Time taken by traditional solver (seconds)	required
`neural_time`	`float`	Time taken by neural solver (seconds)	required

Returns:

Type	Description
`float`	Speedup factor (traditional_time / neural_time)

solve_with_fallback ¶

solve_with_fallback(problem_params: Array) -> Array

Solve with traditional solver fallback for difficult problems.

Parameters:

Name	Type	Description	Default
`problem_params`	`Array`	Problem parameter vectors	required

Returns:

Type	Description
`Array`	Solutions with fallback handling

compare_with_traditional ¶

compare_with_traditional(problem_params: Array) -> dict[str, Any]

Compare performance with traditional optimization methods.

Parameters:

Name	Type	Description	Default
`problem_params`	`Array`	Batch of problems to solve	required

Returns:

Type	Description
`dict[str, Any]`	Performance comparison results

Constraint Learning¶

Automated constraint satisfaction learning algorithms.

Constraint Satisfaction Learning for Learn-to-Optimize (L2O).

This module implements neural network-based constraint satisfaction that learns to project optimization variables to feasible sets and correct constraint violations.

Key Features: - Neural networks learning feasible set projections - Automatic constraint violation detection and correction - Support for equality and inequality constraints - Integration with symbolic constraint specification - Real-time constraint satisfaction (<1ms inference)

ConstraintSpecification `dataclass` ¶

ConstraintSpecification(constraint_type: str, expression: str, coefficients: Array, variables: list[str])

Represents a constraint specification with type, expression, and coefficients.

This class encapsulates mathematical constraints that can be processed by the constraint satisfaction learning system.

evaluate ¶

evaluate(x: Array) -> Array

Evaluate constraint violation for given variable values.

Parameters:

Name	Type	Description	Default
`x`	`Array`	Variable values to evaluate	required

Returns:

Type	Description
`Array`	Constraint violation value (0 means satisfied)

ConstraintViolationDetector ¶

ConstraintViolationDetector(constraints: list[ConstraintSpecification])

Detects and quantifies constraint violations for optimization variables.

This class provides methods to check whether given variable values satisfy specified constraints and quantify the degree of violation.

Parameters:

Name	Type	Description	Default
`constraints`	`list[ConstraintSpecification]`	List of constraint specifications to check against	required

detect_violations ¶

detect_violations(x: Array) -> dict[str, Any]

Detect constraint violations for given variable values.

Parameters:

Name	Type	Description	Default
`x`	`Array`	Variable values to check (can be 1D for single point or 2D for batch)	required

Returns:

Type	Description
`dict[str, Any]`	Dictionary containing violation information

SymbolicConstraintEncoder ¶

SymbolicConstraintEncoder(embedding_dim: int = 16)

Encodes symbolic constraint expressions into neural network embeddings.

This class converts mathematical constraint expressions into vector representations that can be processed by neural networks.

Parameters:

Name	Type	Description	Default
`embedding_dim`	`int`	Dimension of constraint embeddings	`16`

encode_constraint ¶

encode_constraint(constraint: ConstraintSpecification) -> Array

Convert constraint specification to neural network embedding.

Parameters:

Name	Type	Description	Default
`constraint`	`ConstraintSpecification`	Constraint specification to encode	required

Returns:

Type	Description
`Array`	Vector embedding representing the constraint

ProjectorConfig `dataclass` ¶

ProjectorConfig(hidden_sizes: list[int] | None = None, embedding_dim: int = 16)

Configuration for ConstraintProjector neural network.

ConstraintProjector ¶

ConstraintProjector(input_dim: int, config: ProjectorConfig, rngs: Rngs)

Bases: Module

Neural network that projects variables to constraint-feasible regions.

This module learns to map constraint-violating points to nearby points that satisfy the given constraints.

Parameters:

Name	Type	Description	Default
`input_dim`	`int`	Dimension of optimization variables	required
`config`	`ProjectorConfig`	Configuration for the neural network	required
`rngs`	`Rngs`	Random number generators for parameter initialization	required

project ¶

project(points: Array, constraint_embedding: Array) -> Array

Project points to satisfy constraints.

Parameters:

Name	Type	Description	Default
`points`	`Array`	Points to project (shape: [..., input_dim])	required
`constraint_embedding`	`Array`	Constraint representation (shape: [embedding_dim])	required

Returns:

Type	Description
`Array`	Projected points that should satisfy constraints

FeasibilityLearner ¶

FeasibilityLearner(input_dim: int, constraints: list[ConstraintSpecification], config: ProjectorConfig | None = None, rngs: Rngs | None = None)

Main system for learning constraint satisfaction and feasible projections.

This class combines constraint detection, symbolic encoding, and neural projection to provide a complete constraint satisfaction learning system.

Parameters:

Name	Type	Description	Default
`input_dim`	`int`	Dimension of optimization variables	required
`constraints`	`list[ConstraintSpecification]`	List of constraint specifications	required
`config`	`ProjectorConfig \| None`	Configuration for the projector network	`None`
`rngs`	`Rngs \| None`	Random number generators	`None`

satisfy_constraints ¶

satisfy_constraints(variables: Array) -> Array

Project variables to satisfy all constraints.

Parameters:

Name	Type	Description	Default
`variables`	`Array`	Input variables that may violate constraints	required

Returns:

Type	Description
`Array`	Variables projected to satisfy constraints

check_feasibility ¶

check_feasibility(variables: Array) -> dict[str, Any]

Check if variables satisfy all constraints.

Parameters:

Name	Type	Description	Default
`variables`	`Array`	Variables to check for feasibility	required

Returns:

Type	Description
`dict[str, Any]`	Dictionary with feasibility information

Reinforcement Learning Optimization¶

RL-based optimization strategy selection and learning.

Reinforcement Learning-Based Optimization Strategy for L2O Framework.

This module implements a Deep Q-Network (DQN) agent that learns optimization strategies dynamically, making meta-decisions about when to apply different optimization algorithms.

The RL agent observes optimization problem features, convergence history, and resource constraints to select optimal algorithms, adjust hyperparameters, and determine stopping criteria.

RLOptimizationConfig `dataclass` ¶

RLOptimizationConfig(state_dim: int = 64, action_dim: int = 12, hidden_dims: Sequence[int] = (256, 256, 128), learning_rate: float = 0.0001, discount_factor: float = 0.99, epsilon_start: float = 1.0, epsilon_end: float = 0.01, epsilon_decay: float = 0.995, replay_buffer_size: int = 10000, batch_size: int = 32, target_update_freq: int = 100, reward_convergence_weight: float = 0.4, reward_quality_weight: float = 0.4, reward_efficiency_weight: float = 0.2, max_episode_length: int = 1000)

Configuration for reinforcement learning-based optimization.

Attributes:

Name	Type	Description
`state_dim`	`int`	Dimension of state representation
`action_dim`	`int`	Number of discrete actions available
`hidden_dims`	`Sequence[int]`	Hidden layer dimensions for DQN
`learning_rate`	`float`	Learning rate for DQN training
`discount_factor`	`float`	Reward discount factor (gamma)
`epsilon_start`	`float`	Initial exploration rate
`epsilon_end`	`float`	Final exploration rate
`epsilon_decay`	`float`	Exploration decay rate
`replay_buffer_size`	`int`	Size of experience replay buffer
`batch_size`	`int`	Batch size for DQN training
`target_update_freq`	`int`	Frequency to update target network
`reward_convergence_weight`	`float`	Weight for convergence speed in reward
`reward_quality_weight`	`float`	Weight for solution quality in reward
`reward_efficiency_weight`	`float`	Weight for computational efficiency in reward
`max_episode_length`	`int`	Maximum optimization steps per episode

DQNNetwork ¶

DQNNetwork(state_dim: int, action_dim: int, hidden_dims: Sequence[int] = (256, 256, 128), *, rngs: Rngs)

Bases: Module

Deep Q-Network for optimization strategy selection.

The DQN takes optimization state as input and outputs Q-values for each possible action (algorithm selection, hyperparameter adjustment, etc.).

Parameters:

Name	Type	Description	Default
`state_dim`	`int`	Dimension of state representation	required
`action_dim`	`int`	Number of discrete actions	required
`hidden_dims`	`Sequence[int]`	Hidden layer dimensions	`(256, 256, 128)`
`rngs`	`Rngs`	Random number generators	required

StateEncoder ¶

StateEncoder(output_dim: int = 64, *, rngs: Rngs)

Bases: Module

Encoder for optimization problem state representation.

Converts optimization problem features, convergence history, and resource constraints into a fixed-size state vector for the DQN.

Parameters:

Name	Type	Description	Default
`output_dim`	`int`	Dimension of encoded state	`64`
`rngs`	`Rngs`	Random number generators	required

Experience `dataclass` ¶

Experience(state: Array, action: int, reward: float, next_state: Array, done: bool)

Single experience for replay buffer.

ExperienceReplayBuffer ¶

ExperienceReplayBuffer(capacity: int)

Experience replay buffer for DQN training.

Stores optimization experiences and provides efficient sampling for training.

Parameters:

Name	Type	Description	Default
`capacity`	`int`	Maximum number of experiences to store	required

push ¶

push(experience: Experience)

Add experience to buffer.

Parameters:

Name	Type	Description	Default
`experience`	`Experience`	Optimization experience to store	required

sample ¶

sample(batch_size: int) -> list[Experience]

Sample batch of experiences.

Parameters:

Name	Type	Description	Default
`batch_size`	`int`	Number of experiences to sample	required

Returns:

Type	Description
`list[Experience]`	Batch of sampled experiences

RewardFunction ¶

RewardFunction(config: RLOptimizationConfig)

Reward function for RL-based optimization.

Computes rewards based on convergence speed, solution quality, and computational efficiency.

Parameters:

Name	Type	Description	Default
`config`	`RLOptimizationConfig`	RL optimization configuration	required

compute_reward ¶

compute_reward(objective_improvement: float, convergence_speed: float, computational_cost: float, constraint_violation: float = 0.0) -> float

Compute reward for optimization step.

Parameters:

Name	Type	Description	Default
`objective_improvement`	`float`	Improvement in objective function value	required
`convergence_speed`	`float`	Rate of convergence (higher is better)	required
`computational_cost`	`float`	Computational resources used (lower is better)	required
`constraint_violation`	`float`	Degree of constraint violation (lower is better)	`0.0`

Returns:

Type	Description
`float`	Computed reward value

RLOptimizationAgent ¶

RLOptimizationAgent(config: RLOptimizationConfig, *, rngs: Rngs)

Bases: Module

Reinforcement learning agent for optimization strategy selection.

Uses Deep Q-Network to learn optimal optimization strategies based on problem characteristics and optimization progress.

Parameters:

Name	Type	Description	Default
`config`	`RLOptimizationConfig`	RL optimization configuration	required
`rngs`	`Rngs`	Random number generators	required

encode_state ¶

encode_state(problem: OptimizationProblem, convergence_history: Array, resource_usage: dict[str, float]) -> Array

Encode current optimization state.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Current optimization problem	required
`convergence_history`	`Array`	Recent optimization progress	required
`resource_usage`	`dict[str, float]`	Current resource consumption	required

Returns:

Type	Description
`Array`	Encoded state representation

select_action ¶

select_action(state: Array, training: bool = True) -> int

Select action using epsilon-greedy policy.

Parameters:

Name	Type	Description	Default
`state`	`Array`	Current optimization state	required
`training`	`bool`	Whether in training mode	`True`

Returns:

Type	Description
`int`	Selected action index

store_experience ¶

store_experience(state: Array, action: int, reward: float, next_state: Array, done: bool)

Store experience in replay buffer.

Parameters:

Name	Type	Description	Default
`state`	`Array`	Current state	required
`action`	`int`	Action taken	required
`reward`	`float`	Reward received	required
`next_state`	`Array`	Next state	required
`done`	`bool`	Whether episode is complete	required

train_step ¶

train_step() -> dict[str, float]

Perform single training step.

Returns:

Type	Description
`dict[str, float]`	Training metrics

ActionInterpreter ¶

ActionInterpreter()

Interprets DQN actions into optimization strategy modifications.

Maps discrete action indices to specific optimization algorithms, hyperparameter adjustments, and stopping criteria.

interpret_action ¶

interpret_action(action: int, current_config: MetaSchedulerConfig) -> tuple[str, dict[str, Any]]

Interpret action into optimization strategy modification.

Parameters:

Name	Type	Description	Default
`action`	`int`	Action index from DQN	required
`current_config`	`MetaSchedulerConfig`	Current optimization configuration	required

Returns:

Type	Description
`tuple[str, dict[str, Any]]`	Tuple of (action_type, parameters)

RLOptimizationEngine ¶

RLOptimizationEngine(config: RLOptimizationConfig, meta_optimizer: MetaOptimizer | None = None, *, rngs: Rngs)

Bases: Module

Main RL-based optimization engine.

Integrates the RL agent with existing L2O framework to provide intelligent optimization strategy selection and adaptation.

Parameters:

Name	Type	Description	Default
`config`	`RLOptimizationConfig`	RL optimization configuration	required
`meta_optimizer`	`MetaOptimizer \| None`	Existing meta-optimizer for integration	`None`
`rngs`	`Rngs`	Random number generators	required

solve_with_rl ¶

solve_with_rl(problem: OptimizationProblem, max_iterations: int = 1000, training: bool = True) -> dict[str, Any]

Solve optimization problem using RL-guided strategy selection.

Parameters:

Name	Type	Description	Default
`problem`	`OptimizationProblem`	Optimization problem to solve	required
`max_iterations`	`int`	Maximum optimization iterations	`1000`
`training`	`bool`	Whether to train the RL agent	`True`

Returns:

Type	Description
`dict[str, Any]`	Optimization results with RL metrics

get_performance_metrics ¶

get_performance_metrics() -> dict[str, Any]

Get RL agent performance metrics.

Returns:

Type	Description
`dict[str, Any]`	Performance metrics dictionary

Control Systems¶

Differentiable predictive control components for scientific machine learning.

System Identification¶

Neural networks that learn system dynamics from data.

System Identification Networks for Learn-to-Optimize (L2O).

This module implements neural network-based system identification that learns to model dynamical systems with physics constraints, online adaptation, and control integration.

Key Features: - Neural networks for learning system dynamics - Physics-constrained system identification - Online learning and adaptation capabilities - Integration with control policy optimization - Validation on benchmark control systems

PhysicsConstraint `dataclass` ¶

PhysicsConstraint(name: str, constraint_type: str, tolerance: float = 0.001, weight: float = 1.0)

Represents a physics constraint for system identification.

This class encapsulates physical laws and constraints that must be enforced during the learning process.

BenchmarkValidationResult `dataclass` ¶

BenchmarkValidationResult(benchmark_name: str, metrics: dict[str, float], validation_passed: bool, details: dict[str, Any] | None = None)

Results from benchmark validation.

SystemIdentifier ¶

SystemIdentifier(state_dim: int, input_dim: int, hidden_dim: int = 64, num_layers: int = 3, activation: Callable = gelu, *, rngs: Rngs, dtype: dtype = float32)

Bases: Module

Neural network-based system identification.

This module learns to predict the next state of a dynamical system given the current state and input.

validate_on_benchmark ¶

validate_on_benchmark(benchmark_name: str, test_data: dict[str, Array]) -> BenchmarkValidationResult

Validate system identification on benchmark problem.

Parameters:

Name	Type	Description	Default
`benchmark_name`	`str`	Name of the benchmark	required
`test_data`	`dict[str, Array]`	Test data containing states, inputs, targets	required

Returns:

Type	Description
`BenchmarkValidationResult`	Validation results

integrate_with_l2o_solver ¶

integrate_with_l2o_solver() -> dict[str, Any]

Integration interface with L2O optimization components.

Returns:

Type	Description
`dict[str, Any]`	Integration status and configuration

PhysicsConstrainedSystemID ¶

PhysicsConstrainedSystemID(state_dim: int, input_dim: int, constraints: Sequence[PhysicsConstraint], hidden_dim: int = 64, *, rngs: Rngs, dtype: dtype = float32)

Bases: SystemIdentifier

Physics-constrained system identification.

Extends basic system identification with physics constraints and conservation laws.

compute_energy ¶

compute_energy(state: Array) -> Array

Compute energy of the system state.

Parameters:

Name	Type	Description	Default
`state`	`Array`	System state vector	required

Returns:

Type	Description
`Array`	Scalar energy value

predict_with_constraints ¶

predict_with_constraints(state: Array, input_val: Array) -> dict[str, Any]

Predict next state with constraint checking.

Parameters:

Name	Type	Description	Default
`state`	`Array`	Current state	required
`input_val`	`Array`	Input vector	required

Returns:

Type	Description
`dict[str, Any]`	Dictionary with prediction and constraint violation info

OnlineSystemLearner ¶

OnlineSystemLearner(state_dim: int, input_dim: int, learning_rate: float = 0.001, adaptation_rate: float = 0.95, buffer_size: int = 100, adaptive_lr: bool = False, *, rngs: Rngs, dtype: dtype = float32)

Bases: SystemIdentifier

Online learning system identification.

Adapts the system model in real-time based on new observations.

update_online ¶

update_online(state: Array, input_val: Array, target: Array) -> dict[str, Any]

Update model with new observation.

Parameters:

Name	Type	Description	Default
`state`	`Array`	Current state	required
`input_val`	`Array`	Input that was applied	required
`target`	`Array`	Observed next state	required

Returns:

Type	Description
`dict[str, Any]`	Update results including loss and adaptation metrics

get_memory_info ¶

get_memory_info() -> dict[str, Any]

Get memory management information.

Returns:

Type	Description
`dict[str, Any]`	Memory statistics

ControlIntegratedSystemID ¶

ControlIntegratedSystemID(state_dim: int, input_dim: int, control_dim: int, hidden_dim: int = 64, *, rngs: Rngs, dtype: dtype = float32)

Bases: SystemIdentifier

System identification integrated with control policy learning.

Jointly optimizes system identification and control policy for improved performance.

compute_control_action ¶

compute_control_action(current_state: Array, target_state: Array) -> Array

Compute control action to reach target state.

Parameters:

Name	Type	Description	Default
`current_state`	`Array`	Current system state	required
`target_state`	`Array`	Desired target state	required

Returns:

Type	Description
`Array`	Control action

joint_optimization ¶

joint_optimization(states: Array, targets: Array) -> dict[str, float]

Joint optimization of system ID and control policy.

Parameters:

Name	Type	Description	Default
`states`	`Array`	State trajectory	required
`targets`	`Array`	Target trajectory	required

Returns:

Type	Description
`dict[str, float]`	Optimization losses

simulate_closed_loop ¶

simulate_closed_loop(initial_state: Array, target_state: Array, steps: int) -> dict[str, Array]

Simulate closed-loop system with learned control.

Parameters:

Name	Type	Description	Default
`initial_state`	`Array`	Starting state	required
`target_state`	`Array`	Target state to reach	required
`steps`	`int`	Number of simulation steps	required

Returns:

Type	Description
`dict[str, Array]`	Simulation results

validate_control_benchmark ¶

validate_control_benchmark(benchmark_name: str, initial_state: Array, reference_trajectory: Array, steps: int) -> BenchmarkValidationResult

Validate control performance on benchmark.

Parameters:

Name	Type	Description	Default
`benchmark_name`	`str`	Name of control benchmark	required
`initial_state`	`Array`	Starting state	required
`reference_trajectory`	`Array`	Desired trajectory	required
`steps`	`int`	Number of steps	required

Returns:

Type	Description
`BenchmarkValidationResult`	Validation results

integrate_constraint_learning ¶

integrate_constraint_learning() -> dict[str, Any]

Integration with constraint learning from Version 5.1.

Returns:

Type	Description
`dict[str, Any]`	Constraint satisfaction integration results

SystemDynamicsModel ¶

SystemDynamicsModel(model_type: str, state_dim: int, input_dim: int, hidden_dims: Sequence[int] | None = None, *, rngs: Rngs, dtype: dtype = float32)

Bases: Module

Parameterizable system dynamics model.

Supports both linear and nonlinear system representations.

Model Predictive Control¶

Differentiable MPC frameworks with safety guarantees.

Model Predictive Control (MPC) Framework for Opifex.

Provides differentiable MPC implementation with neural network-based predictive models, constraint handling and projection, real-time control policy optimization, and safety-critical system support.

MPCConfig `dataclass` ¶

MPCConfig(horizon: int = 10, control_dim: int = 2, state_dim: int = 4, prediction_steps: int | None = None, objective_weights: dict[str, float] | None = None, max_iterations: int = 50, tolerance: float = 0.0001, time_limit: float = 0.01, learning_rate: float = 0.01)

Configuration for MPC controller.

MPCResult ¶

Bases: NamedTuple

Result from MPC computation.

OptimizationResult ¶

Bases: NamedTuple

Result from optimization.

BatchMPCResult ¶

Bases: NamedTuple

Result from batch MPC computation.

PredictiveModel ¶

PredictiveModel(state_dim: int, control_dim: int, hidden_dims: list[int] | None = None, prediction_horizon: int = 10, model_type: str = 'neural', physics_informed: bool = False, conservation_laws: list[str] | None = None, *, rngs: Rngs)

Bases: Module

Neural network-based predictive model for system dynamics.

predict_step ¶

predict_step(state: ndarray, control: ndarray) -> ndarray

Predict next state given current state and control.

predict_trajectory ¶

predict_trajectory(initial_state: ndarray, control_sequence: ndarray) -> ndarray

Predict state trajectory given control sequence.

ConstraintProjector ¶

ConstraintProjector(state_dim: int, control_dim: int, state_bounds: dict[str, list[float]] | None = None, control_bounds: dict[str, list[float]] | None = None, safety_constraints: bool = False, *, rngs: Rngs)

Bases: Module

Neural network-based constraint projection.

add_custom_constraint ¶

add_custom_constraint(constraint_fn: Callable)

Add custom constraint function.

project_state ¶

project_state(state: ndarray) -> ndarray

Project state to satisfy constraints.

project_control ¶

project_control(control: ndarray) -> ndarray

Project control to satisfy constraints.

ControlBarrier ¶

ControlBarrier(constraint: Callable, alpha: float = 1.0)

Control barrier function for safety.

is_safe_control ¶

is_safe_control(state: ndarray, control: ndarray) -> bool

Check if control is safe given current state.

RealTimeOptimizer ¶

RealTimeOptimizer(max_iterations: int = 50, tolerance: float = 0.0001, learning_rate: float = 0.01, warm_start: bool = True, time_limit: float = 0.01)

Bases: Module

Real-time optimizer for MPC problems.

optimize ¶

optimize(objective: Callable, constraints: Callable | None, initial_guess: ndarray, warm_start_solution: ndarray | None = None) -> OptimizationResult

Optimize objective subject to constraints.

Note: This is not JIT-compatible due to time limits.

Parameters:

Name	Type	Description	Default
`objective`	`Callable`	Objective function to optimize.	required
`constraints`	`Callable \| None`	Constraints to enforce.	required
`initial_guess`	`ndarray`	Initial guess for the solution.	required
`warm_start_solution`	`ndarray \| None`	Solution from previous iteration for warm start.	`None`

Returns:

Name	Type	Description
`OptimizationResult`	`OptimizationResult`	Result of the optimization.

optimize_with_time_limit ¶

optimize_with_time_limit(objective: Callable, constraints: Callable | None, initial_guess: ndarray, warm_start_solution: ndarray | None = None) -> OptimizationResult

Optimize with real-time constraints (not JIT-compatible due to time limits).

This method includes time limit enforcement and therefore cannot be JIT-compiled. Use optimize() for JIT-compatible optimization without time limits.

MPCObjective ¶

MPCObjective(weights: dict[str, float])

MPC objective function.

DifferentiableMPC ¶

DifferentiableMPC(config: MPCConfig, dynamics_model: PredictiveModel | None = None, constraint_projector: ConstraintProjector | None = None)

Bases: Module

Differentiable Model Predictive Control implementation.

set_dynamics ¶

set_dynamics(dynamics_fn: Callable)

Set custom dynamics function.

compute_objective ¶

compute_objective(states: ndarray, controls: ndarray, reference: ndarray) -> Array

Compute MPC objective function.

compute_control ¶

compute_control(current_state: ndarray, reference_trajectory: ndarray) -> MPCResult

Compute optimal control action.

compute_control_batch ¶

compute_control_batch(batch_states: ndarray, batch_references: ndarray) -> BatchMPCResult

Compute control for batch of states.

SafetyCriticalMPC ¶

SafetyCriticalMPC(horizon: int = 10, control_dim: int = 2, state_dim: int = 4, safety_barriers: bool = True, emergency_control: bool = True, backup_policy: bool = True, **kwargs)

Bases: DifferentiableMPC

Safety-critical MPC with emergency control and backup policies.

add_barrier ¶

add_barrier(barrier: ControlBarrier)

Add control barrier function.

compute_safe_control ¶

compute_safe_control(current_state: ndarray, reference_trajectory: ndarray) -> MPCResult

Compute safe control action with emergency and backup policies.

RecedingHorizonController ¶

RecedingHorizonController(mpc_horizon: int = 10, control_horizon: int = 5, state_dim: int = 4, control_dim: int = 2, sampling_time: float = 0.1, safety_critical: bool = False)

Bases: Module

Receding horizon controller implementation.

compute_control ¶

compute_control(current_state: ndarray, reference_trajectory: ndarray) -> MPCResult

Compute control using receding horizon.

simulate_tracking ¶

simulate_tracking(initial_state: ndarray, reference_trajectory: ndarray, simulation_steps: int | None = None) -> ndarray

Simulate reference tracking.

Module Overview¶

The optimization module is organized into several key components:

Core Components¶

meta_optimization/: Meta-optimization framework with L2O algorithms (modular package)
production.py: Production optimization and deployment systems
performance_monitoring.py: AI-powered performance monitoring
adaptive_deployment.py: Adaptive deployment with rollback automation
../deployment/resource_management.py: Global resource management and cost optimization (imported from deployment module)
edge_network.py: Intelligent edge network optimization
scientific_integration.py: Physics-aware optimization integration

L2O Submodule (`l2o/`)¶

l2o_engine.py: Core L2O engine and parametric solvers
advanced_meta_learning.py: MAML, Reptile, and gradient-based methods
adaptive_schedulers.py: Bayesian and performance-aware schedulers
multi_objective.py: Multi-objective optimization algorithms
parametric_solver.py: Parametric programming solvers
constraint_learning.py: Constraint satisfaction learning
rl_optimization.py: Reinforcement learning optimization

Control Submodule (`control/`)¶

system_id.py: System identification networks
mpc.py: Model predictive control frameworks

Key Features¶

Meta-Optimization Features¶

Learn-to-Optimize (L2O) algorithms with >100x speedup
Adaptive learning rate scheduling
Warm-starting strategies for related problems
Performance monitoring and analytics

Production Optimization Features¶

Hybrid performance platform with adaptive JIT
Intelligent GPU memory management
AI-powered deployment strategies
Global resource management across cloud providers

Control Systems Features¶

Differentiable model predictive control
Physics-constrained system identification
Safety-critical control with barrier functions
Real-time optimization capabilities

Scientific Integration Features¶

Physics-informed optimization
Conservation law enforcement
Numerical validation and stability checks
Domain-specific profiling and benchmarking

Usage Examples¶

Basic Meta-Optimization¶

from opifex.optimization.meta_optimization import LearnToOptimize, MetaOptimizerConfig

config = MetaOptimizerConfig(
    meta_learning_rate=1e-4,
    adaptation_steps=5,
    warm_start_strategy="previous_params"
)

l2o = LearnToOptimize(config=config, rngs=nnx.Rngs(42))
optimized_params = l2o.optimize(params, objective_fn, num_steps=1000)

Production Deployment¶

from opifex.optimization.production import HybridPerformancePlatform
from opifex.optimization.adaptive_deployment import AdaptiveDeploymentSystem

platform = HybridPerformancePlatform(
    gpu_memory_optimization=True,
    adaptive_jit=True
)

deployment = AdaptiveDeploymentSystem(
    canary_percentage=10,
    ai_driven_strategies=True
)

Control System¶

from opifex.optimization.control import DifferentiableMPC, SystemIdentifier

# Learn system dynamics
system_id = SystemIdentifier(model=dynamics_model)
trained_model = system_id.fit(state_data, input_data)

# Create MPC controller
mpc = DifferentiableMPC(system_model=trained_model, config=mpc_config)
control_action = mpc.solve(current_state, reference_trajectory)

Performance Characteristics¶

L2O Speedup: >100x on learned problem families
Meta-Optimization: 10-50x faster convergence on related problems
Production Optimization: 40-60% reduction in computational costs
Edge Network: Sub-millisecond latency optimization
Memory Efficiency: Up to 80% memory usage reduction

Integration¶

The optimization module integrates seamlessly with:

Training: Meta-optimization for training workflows
Neural Networks: Compatible with all neural architectures
Physics: Physics-informed optimization constraints
Deployment: Production-ready optimization systems

Second-Order Optimization¶

Curvature-based optimization methods including L-BFGS and hybrid optimizers.

Configuration Classes¶

opifex.optimization.second_order.config ¶

Configuration classes for second-order optimization methods.

This module provides unified configuration dataclasses for all second-order optimization methods supported by the Opifex framework.

Design Principles

All configs are frozen dataclasses (immutable)
Validation happens at construction time via post_init
Sensible defaults based on literature recommendations
Clear separation between method-specific and shared configs

References

Survey: arXiv:2601.10222v1 Section 7
L-BFGS memory size: typically 3-20 (Liu & Nocedal, 1989)
Hybrid switching: Section 7.4 of the survey

LBFGSConfig `dataclass` ¶

LBFGSConfig(memory_size: int = 10, scale_init_precond: bool = True, linesearch: LinesearchType = ZOOM, max_linesearch_steps: int = 20, max_iterations: int = 100, tolerance: float = 1e-06)

Configuration for L-BFGS optimizer.

L-BFGS (Limited-memory BFGS) approximates the inverse Hessian using a limited history of gradient differences. This makes it suitable for large-scale optimization where storing the full Hessian is infeasible.

Attributes:

Name	Type	Description
`memory_size`	`int`	Number of gradient pairs to store (typically 3-20)
`scale_init_precond`	`bool`	Whether to scale initial preconditioner
`linesearch`	`LinesearchType`	Line search algorithm to use
`max_linesearch_steps`	`int`	Maximum steps for line search
`max_iterations`	`int`	Maximum L-BFGS iterations
`tolerance`	`float`	Convergence tolerance

References

Liu & Nocedal (1989): On the limited memory BFGS method
optax.lbfgs documentation

GaussNewtonConfig `dataclass` ¶

GaussNewtonConfig(damping_factor: float = 0.001, damping_increase_factor: float = 10.0, damping_decrease_factor: float = 0.1, min_damping: float = 1e-10, max_damping: float = 10000000000.0, max_iterations: int = 100, rtol: float = 1e-06, atol: float = 1e-06)

Configuration for Gauss-Newton and Levenberg-Marquardt solvers.

Gauss-Newton is effective for nonlinear least-squares problems where the residual Jacobian can be computed efficiently. Levenberg-Marquardt adds damping for improved robustness.

Attributes:

Name	Type	Description
`damping_factor`	`float`	Initial damping factor (λ) for LM
`damping_increase_factor`	`float`	Factor to increase damping on failure (> 1)
`damping_decrease_factor`	`float`	Factor to decrease damping on success (< 1)
`min_damping`	`float`	Minimum allowed damping value
`max_damping`	`float`	Maximum allowed damping value
`max_iterations`	`int`	Maximum solver iterations
`rtol`	`float`	Relative tolerance for convergence
`atol`	`float`	Absolute tolerance for convergence

References

optimistix.LevenbergMarquardt documentation
Survey Section 7.3

HybridOptimizerConfig `dataclass` ¶

HybridOptimizerConfig(first_order_steps: int = 1000, switch_criterion: SwitchCriterion = LOSS_VARIANCE, loss_variance_threshold: float = 0.0001, loss_history_window: int = 50, gradient_norm_threshold: float = 0.001, relative_improvement_threshold: float = 0.0001, adam_learning_rate: float = 0.001, adam_b1: float = 0.9, adam_b2: float = 0.999, lbfgs_config: LBFGSConfig = LBFGSConfig())

Configuration for hybrid Adam→L-BFGS optimizer.

This optimizer starts with Adam for initial exploration and switches to L-BFGS for efficient convergence once the loss landscape becomes smooth. This follows recommendations from Survey Section 7.4.

The switch can be triggered by various criteria

EPOCH: Switch after fixed number of steps
LOSS_VARIANCE: Switch when loss variance drops below threshold
GRADIENT_NORM: Switch when gradient norm drops below threshold
RELATIVE_IMPROVEMENT: Switch when relative improvement slows

Attributes:

Name	Type	Description
`first_order_steps`	`int`	Steps to run Adam before considering switch
`switch_criterion`	`SwitchCriterion`	Criterion for switching to L-BFGS
`loss_variance_threshold`	`float`	Threshold for loss variance criterion
`loss_history_window`	`int`	Window size for computing loss statistics
`gradient_norm_threshold`	`float`	Threshold for gradient norm criterion
`relative_improvement_threshold`	`float`	Threshold for relative improvement
`adam_learning_rate`	`float`	Learning rate for Adam phase
`adam_b1`	`float`	Adam beta1 parameter
`adam_b2`	`float`	Adam beta2 parameter
`lbfgs_config`	`LBFGSConfig`	Configuration for L-BFGS phase

References

Survey Section 7.4: "L-BFGS is more effective in later stages when loss varies smoothly"

L-BFGS and Gauss-Newton Wrappers¶

opifex.optimization.second_order.wrappers ¶

Wrappers for external second-order optimization libraries.

This module provides thin wrappers around optax and optimistix to create second-order optimizers with our unified configuration interface.

Design Philosophy

Wrap existing robust implementations (optax, optimistix)
Don't reinvent the wheel
Provide consistent interface through our config classes

References

optax.lbfgs: Pure JAX L-BFGS with line search
optimistix: Gauss-Newton, Levenberg-Marquardt, BFGS

create_lbfgs_optimizer ¶

create_lbfgs_optimizer(config: LBFGSConfig | None = None) -> GradientTransformation

Create L-BFGS optimizer using optax.

L-BFGS is a quasi-Newton method that approximates the inverse Hessian using a limited history of gradient differences. This is the recommended second-order optimizer for large-scale optimization.

Parameters:

Name	Type	Description	Default
`config`	`LBFGSConfig \| None`	L-BFGS configuration. Uses defaults if None.	`None`

Returns:

Type	Description
`GradientTransformation`	Optax L-BFGS gradient transformation.

Example

config = LBFGSConfig(memory_size=20) optimizer = create_lbfgs_optimizer(config)

Use with optax training loop¶

create_gauss_newton_solver ¶

create_gauss_newton_solver(config: GaussNewtonConfig | None = None) -> AbstractLeastSquaresSolver

Create Gauss-Newton solver using optimistix.

Gauss-Newton is effective for nonlinear least-squares problems. Note that this creates a solver for root-finding/minimization, not a gradient transformation like L-BFGS.

Parameters:

Name	Type	Description	Default
`config`	`GaussNewtonConfig \| None`	Gauss-Newton configuration. Uses defaults if None.	`None`

Returns:

Type	Description
`AbstractLeastSquaresSolver`	Optimistix Gauss-Newton solver.

Example

solver = create_gauss_newton_solver()

Use with optimistix.least_squares¶

Hybrid Adam → L-BFGS Optimizer¶

opifex.optimization.second_order.hybrid_optimizer ¶

Hybrid Adam→L-BFGS optimizer for physics-informed training.

This module implements a hybrid optimization strategy that starts with Adam for initial exploration and switches to L-BFGS for efficient convergence once the loss landscape becomes smooth.

Design Rationale (from Survey Section 7.4): "L-BFGS is more effective in later stages when loss varies smoothly."

The hybrid approach combines

Adam's robustness in noisy, high-curvature early optimization
L-BFGS's superior convergence in smooth regions near optima

Key Features

Multiple switching criteria (epoch, loss variance, gradient norm)
Loss history tracking for variance-based switching
Full JAX/JIT compatibility
Works with FLAX NNX models

References

Survey: arXiv:2601.10222v1 Section 7.4

HybridOptimizer `dataclass` ¶

HybridOptimizer(config: HybridOptimizerConfig)

Hybrid Adam→L-BFGS optimizer.

This optimizer starts with Adam and switches to L-BFGS based on configurable criteria. The transition is designed to leverage Adam's robustness in early training and L-BFGS's efficiency for final convergence.

Attributes:

Name	Type	Description
`config`	`HybridOptimizerConfig`	Hybrid optimizer configuration
`adam`	`HybridOptimizerConfig`	Adam optimizer instance
`lbfgs`	`HybridOptimizerConfig`	L-BFGS optimizer instance

Example

config = HybridOptimizerConfig(first_order_steps=1000) optimizer = HybridOptimizer(config) state = optimizer.init(params)

Training loop¶

for step in range(num_steps): ... loss, grads = loss_and_grad_fn(params) ... updates, state = optimizer.update(grads, state, params, loss=loss) ... params = optax.apply_updates(params, updates)

is_using_lbfgs `property` ¶

is_using_lbfgs: bool

Check if optimizer is currently using L-BFGS.

Note: This is a convenience property. For actual state, check the HybridOptimizerState.using_lbfgs field.

init ¶

init(params: PyTree) -> HybridOptimizerState

Initialize optimizer state.

Parameters:

Name	Type	Description	Default
`params`	`PyTree`	Model parameters (PyTree)	required

Returns:

Type	Description
`HybridOptimizerState`	Initial optimizer state

update ¶

update(grads: PyTree, state: HybridOptimizerState, params: PyTree, *, loss: Float[Array, ''] | None = None, value: Float[Array, ''] | None = None, grad: PyTree | None = None, value_fn: Callable[[PyTree], Float[Array, '']] | None = None) -> tuple[PyTree, HybridOptimizerState]

Compute parameter updates.

This method handles the switching logic and delegates to either Adam or L-BFGS depending on the current state.

Parameters:

Name	Type	Description	Default
`grads`	`PyTree`	Parameter gradients	required
`state`	`HybridOptimizerState`	Current optimizer state	required
`params`	`PyTree`	Current parameters (needed for L-BFGS)	required
`loss`	`Float[Array, ''] \| None`	Current loss value (for variance-based switching)	`None`
`value`	`Float[Array, ''] \| None`	Alias for loss (for optax L-BFGS compatibility)	`None`
`grad`	`PyTree \| None`	Alias for grads (for optax L-BFGS compatibility)	`None`
`value_fn`	`Callable[[PyTree], Float[Array, '']] \| None`	Loss function (needed for L-BFGS line search)	`None`

Returns:

Type	Description
`tuple[PyTree, HybridOptimizerState]`	Tuple of (updates, new_state)

NNX Integration¶

opifex.optimization.second_order.nnx_integration ¶

FLAX NNX integration for second-order optimizers.

This module provides wrapper classes that make it easy to use second-order optimizers (L-BFGS, hybrid Adam→L-BFGS) with FLAX NNX models.

Design Philosophy

Hide the complexity of nnx.split/merge from users
Provide familiar step() interface similar to nnx.Optimizer
Support both pure L-BFGS and hybrid optimization strategies

Key Classes

NNXSecondOrderOptimizer: L-BFGS optimizer for NNX models
NNXHybridOptimizer: Hybrid Adam→L-BFGS for NNX models
create_nnx_lbfgs_optimizer: Factory function for L-BFGS

References

FLAX NNX documentation: https://flax.readthedocs.io/en/latest/nnx/
optax L-BFGS requires functional API with value_and_grad_from_state

create_nnx_lbfgs_optimizer ¶

create_nnx_lbfgs_optimizer(model: Module, config: LBFGSConfig | None = None) -> NNXSecondOrderOptimizer

Create L-BFGS optimizer for NNX model.

Factory function that creates an NNXSecondOrderOptimizer configured with L-BFGS.

Parameters:

Name	Type	Description	Default
`model`	`Module`	FLAX NNX model to optimize	required
`config`	`LBFGSConfig \| None`	L-BFGS configuration. Uses defaults if None.	`None`

Returns:

Type	Description
`NNXSecondOrderOptimizer`	Configured NNXSecondOrderOptimizer instance.

Example

model = MyModel(rngs=nnx.Rngs(0)) optimizer = create_nnx_lbfgs_optimizer(model) for _ in range(100): ... loss = optimizer.step(loss_fn)

For detailed algorithms and best practices, see the Second-Order Optimization Guide.

Optimization API Reference¶

Overview¶

Core Optimization Components¶

Meta-Optimization Framework¶

MetaOptimizerConfig dataclass ¶

MetaOptimizer ¶

init_optimizer_state ¶

step ¶

store_optimization_result ¶

get_warm_start_params ¶

quantum_step ¶

PerformanceMonitor ¶

update_metrics ¶

get_metric_history ¶

check_convergence ¶

get_performance_analytics ¶

get_quantum_analytics ¶

LearnToOptimize ¶

compute_update ¶

compute_meta_gradients ¶

compute_adaptive_update ¶

compute_quantum_update ¶

AdaptiveLearningRateScheduler ¶

get_learning_rate ¶

adapt_from_performance ¶

adapt_from_quantum_metrics ¶

WarmStartingStrategy ¶

get_warm_start_params ¶

adapt_optimizer_state ¶

get_molecular_warm_start ¶

Production Optimization¶

CallableModule ¶

OptimizationStrategy ¶

WorkloadProfile dataclass ¶

PerformanceMetrics dataclass ¶

OptimizedModel dataclass ¶

AdaptiveJAXOptimizer ¶

analyze_workload_patterns ¶

apply_aggressive_kernel_fusion ¶

apply_memory_optimization ¶

apply_latency_optimization ¶

apply_balanced_optimization ¶

benchmark_model_performance ¶

optimize_neural_operator ¶

IntelligentGPUMemoryManager ¶

select_memory_pool ¶

estimate_model_memory_usage ¶

optimize_multi_model_allocation ¶

HybridPerformancePlatform ¶

optimize_for_production ¶

start_continuous_monitoring async ¶

stop_continuous_monitoring async ¶

get_comprehensive_status ¶

get_model_input_features ¶

Performance Monitoring¶

AnomalySeverity ¶

Anomaly dataclass ¶

PerformanceMetrics dataclass ¶

PredictionResult dataclass ¶

PerformanceMonitorProtocol ¶

collect_metrics async ¶

analyze_trends async ¶

PerformancePredictor ¶

AIAnomalyDetector ¶

detect_anomalies ¶

PerformanceMonitor ¶

start_monitoring async ¶

stop_monitoring async ¶

collect_current_metrics async ¶

detect_performance_anomalies async ¶

predict_future_performance async ¶

handle_anomalies async ¶

handle_predictions async ¶

PredictiveScaler ¶

evaluate_scaling_decision async ¶

execute_scaling_action async ¶

Adaptive Deployment¶

DeploymentStrategy ¶

DeploymentStatus ¶

RollbackTrigger ¶

MetaOptimizerConfig `dataclass` ¶

WorkloadProfile `dataclass` ¶

PerformanceMetrics `dataclass` ¶

OptimizedModel `dataclass` ¶

start_continuous_monitoring `async` ¶

stop_continuous_monitoring `async` ¶

Anomaly `dataclass` ¶

PerformanceMetrics `dataclass` ¶

PredictionResult `dataclass` ¶

collect_metrics `async` ¶

analyze_trends `async` ¶

start_monitoring `async` ¶

stop_monitoring `async` ¶

collect_current_metrics `async` ¶

detect_performance_anomalies `async` ¶

predict_future_performance `async` ¶

handle_anomalies `async` ¶

handle_predictions `async` ¶

evaluate_scaling_decision `async` ¶

execute_scaling_action `async` ¶

DeploymentConfig `dataclass` ¶

DeploymentMetrics `dataclass` ¶

DeploymentState `dataclass` ¶

RollbackDecision `dataclass` ¶

start_canary_deployment `async` ¶

optimize_traffic_distribution `async` ¶

evaluate_rollback_decision `async` ¶

deploy_model `async` ¶

monitor_deployments `async` ¶

allocate_resources_with_intelligence `async` ¶

start_resource_monitoring `async` ¶

stop_resource_monitoring `async` ¶

optimize_memory_layout `async` ¶

CostOptimization `dataclass` ¶

ResourceAllocation `dataclass` ¶

ResourcePool `dataclass` ¶

SustainabilityMetrics `dataclass` ¶

EdgeNodeMetrics `dataclass` ¶

LatencyProfile `dataclass` ¶

CacheEntry `dataclass` ¶

FailoverResult `dataclass` ¶