Ocean Wave Energy Flux Forecasting: Probabilistic Forecasting (Part 3)

Series: Wave Energy

Part: 4 (Flux Forecasting: Probabilistic Forecasting)

Introduction

Deterministic point forecasts (like the LightGBM models in Part 1 or the Stacking Ensemble in Part 2) predict a single expected value. However, wave energy operations depend heavily on understanding extreme scenarios (storms and calms). Underestimating wave heights can result in structural damage to wave energy converters, while overestimating them leads to costly, unnecessary maintenance delays.

This part transitions from single-point forecasts to probabilistic forecasts. We evaluate three techniques:

1. Quantile Regression (QR): Linear quantile estimation per quantile.
2. Quantile Regression Forest (QRF): Non-parametric random forests that extract leaf distribution values.
3. Bootstrapped Residuals: A point model combined with resampled validation errors (using in-sample, out-of-sample, and binned/conditional residuals).

1. Methodology Summary

The testing setup mirrors our previous walk-forward validation framework:

• 6 walk-forward folds: 1-month validation and 1-month test windows, with an expanding training window.
• Target Variables: Significant Wave Height ($H_s$ / swh) and Mean Wave Period ($T_e$ / mwp) at $1h, 3h, 6h, 12h, 24h,$ and $48h$ lead times.
• Quantiles Estimated: 13 quantile levels: $\tau \in {0.025, 0.05, 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, 0.90, 0.95, 0.975}$.
• Evaluation Metrics: Pinball Loss, Continuous Ranked Probability Score (CRPS), Empirical Coverage ($\text{Cov}{\beta}$), and Mean Interval Width ($\text{Width}{\beta}$) for $80\%$, $90\%$, and $95\%$ nominal confidence intervals.

graph TD
    Data[Input Data & Features] --> QR[Quantile Regression]
    Data --> QRF[Quantile Regression Forest]
    Data --> Boot[Bootstrapped Residuals]
    QR --> Eval[Score: Pinball Loss, CRPS, Coverage]
    QRF --> Eval
    Boot --> Eval

2. Comparative Results

2.1 Probabilistic Metrics Comparison

The table below aggregates the mean cross-validation performance across folds for select lead times ($1h, 12h, 24h, 48h$).

Significant Wave Height ($H_s$ / swh)

Mean Wave Period ($T_e$ / mwp)

2.2 Key Takeaways

1. QR dominates overall accuracy: Quantile Regression achieves the lowest Pinball Loss and CRPS at almost all lead times for both wave height and wave period.
2. QRF performs well at short horizons: QRF is competitive at 1h–3h but degrades significantly at longer lead times (e.g., 48h). This is because the random forest’s ensemble leaf averaging struggles to extrapolate to extreme conditions.
3. Bootstrapped Residuals are highly cost-efficient: While QRF requires extensive memory to keep forests on disk, and QR requires fitting 13 independent regressions, Bootstrapped Residuals require fitting just one LightGBM point model and adjusting it with validation residuals. Despite this simplicity, it achieves CRPS metrics that are remarkably close to QR.

2.3 Forecast Visualizations

To visualize the prediction intervals across different lead times, we compare the short-term and long-term forecasts for Significant Wave Height ($H_s$):

3. Comparison with Stacking Ensemble (Part 2)

Evaluating the median ($q_{0.50}$) forecast of the probabilistic models against Part 2’s Ridge-Stacking point-forecaster helps us assess whether training with quantile-specific objectives affects central accuracy.

Significant Wave Height ($H_s$ / swh) Point Metrics (MAE / RMSE)

Mean Wave Period ($T_e$ / mwp) Point Metrics (MAE / RMSE)

Observation: The Stacking Ensemble consistently outperforms the median forecasts of the probabilistic estimators. This is expected: the stacking ensemble is optimized explicitly for mean squared error (MSE) using regularized base predictions, whereas the median forecasts ($q_{0.50}$) are optimized for mean absolute error (MAE) under the pinball loss. This highlight the classical trade-off: to obtain reliable probabilistic intervals, we must accept a slight penalty in raw point accuracy.

4. Deep Dive: Bootstrapped Residual Strategies

A key limitation of the basic out-of-sample residual bootstrap is that it is unconditional: it applies the same error distribution offset across the entire test set. In practice, prediction uncertainty is heteroscedastic; we expect wider intervals during storms and narrower intervals during calm weather.

We compared three residual strategies on the final fold for $24h$ SWH forecasting:

1. In-sample residuals: Residuals computed on training data. Usually overoptimistic and too narrow due to overfitting.
2. Out-sample residuals: Validation set residuals. Provides correct overall coverage, but has a constant interval width regardless of conditions.
3. Binned (Conditional) residuals: Out-of-sample residuals grouped into 15 bins based on the predicted value. A test prediction uses residuals from the corresponding bin.

Why Binned Residuals Win

By grouping residuals based on predicted wave heights, the model builds intervals that scale with predicted conditions.

• During calm periods, the model uses validation errors from other calm predictions, producing narrow intervals.
• During storms, it scales the intervals up.

This conditional approach achieves a much smaller cumulative interval area ($455.34$ vs $671.57$ unconditional) while maintaining reliable empirical coverage ($87.1\%$).

Conclusion

This concludes our wave energy forecasting series:

• Part 1 established baseline deterministic modeling using LightGBM.
• Part 2 recovered long-lead prediction capabilities by building a Stacking Meta-model.
• Part 3 added probabilistic bounds, comparing QR, QRF, and Bootstrapped Residuals.

Quantile Regression yields the highest overall probabilistic accuracy, while conditional (binned) Bootstrapped Residuals offer an elegant, computationally inexpensive alternative that scales uncertainty intervals dynamically with sea conditions.

References

• “Probabilistic Forecasting: Bootstrapped Residuals User Guide”, skforecast documentation.
• Koenker, R. (2005). Quantile Regression. Cambridge University Press.
• Meinshausen, N. (2006). “Quantile Regression Forests”. Journal of Machine Learning Research.

Code

• qr_qrf_wave_pipeline.py: Implements walk-forward cross-validation for QR and QRF.
• bootstrap_residual_wave_pipeline.py: Generates probabilistic distributions using LightGBM validation residuals.
• bootstrap_residuals_deep_dive.py: Compares in-sample, out-sample, and conditional binned bootstrapping methods.