enh(SES): improve implementation

[Christopher Titchen (christophertitchen)]

Summary

• Amend README.md to note support for in-sample fitted values for SimpleExponentialSmoothing, SimpleExponentialSmoothingOptimized, SeasonalExponentialSmoothing, and SeasonalExponentialSmoothingOptimized.
• Set the first in-sample fitted value in _ses_fcst_mse to the first observation, rather than np.nan, to make it explicitly clear that the initial state is the first observation.
• Improve the efficiency and performance of _ses_fcst_mse by, for example, pre-calculating the smoothing complement, $1 - \alpha$, and removing the unnecessary $n - 1$ copies of fitted values before assignment.
• Improve the performance of _optimized_ses_forecast by changing from scipy.optimize.minimize to scipy.optimize.minimize_scalar. This uses Brent's method which is derivative-free and optimised for local one-dimensional optimisation problems like this, with guaranteed convergence within a reasonable number of evaluations. The convergence will also be superlinear for our strictly convex $C^2$ objective function when the minimum is at an interior point of the interval for $\alpha$, which will be the case particularly in SimpleExponentialSmoothingOptimized and SeasonalExponentialSmoothingOptimized, where $0 \lt \alpha \lt 1$. It should also be quite a bit faster even for the $0.1 \lt \alpha \lt 0.3$ bounds of ADIDA, Croston, and IMAPA. It also does not have the initialisation sensitivity that L-BFGS-B has (the initialisation of $\alpha$ was also out-of-bounds before for some reason).
• Add a temporary benchmark to test_efficiency.py for CodSpeed.
• Improve the efficiency and performance of the objective function and forecast function.

Benchmarks

Memory Profile

Base

Head

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[codspeed]

CodSpeed Performance Report

Merging #973 will not alter performance

_{Comparing christophertitchen:fix/ses-fitted (686cf6d) with main (078f0e5)}

Summary

✅ 6 untouched benchmarks

[José Morales (jmoralez)]

Thanks. Can you just elaborate on the following?

Set the first in-sample fitted value in _ses_fcst_mse to the first observation, rather than np.nan, to make it explicitly clear that the initial state is the first observation.

The fitted values are supposed to be the one-step ahead forecasts with an expanding training set, so the first value should be NaN.

[Christopher Titchen (christophertitchen)]

Thanks. Can you just elaborate on the following?

Set the first in-sample fitted value in _ses_fcst_mse to the first observation, rather than np.nan, to make it explicitly clear that the initial state is the first observation.

The fitted values are supposed to be the one-step ahead forecasts with an expanding training set, so the first value should be NaN.

Hi José. I reverted the change to the initial fitted value as requested to align with the one-step forecast with an expanding training set definition and made a few more performance improvements.

I originally changed it as I thought that as our implementation uses $x_{0}$ as $\ell_{0}$ to initialise the smoothing process, it should be the initial fitted value. It works this way in other libraries, like the example from tsintermittent below which uses the same approach as us to set $\ell_{0}$, and of course also as expected in the SSOE state space approaches in forecast and smooth.

[José Morales (jmoralez)]

Thanks but I think the big win here is just using brent and we can avoid complicating things.

[Christopher Titchen (christophertitchen)]

José Morales (@jmoralez) tests are failing, see my PR Nixtla/coreforecast#79 for fix to C++ code for EMA, which should fix the issue. 🤞

[José Morales (jmoralez)]

Christopher Titchen (@christophertitchen) can you restore your numba function? It'll take me a bit longer to do the coreforecast release because I want to refactor all the single array functions. Sorry for the troubles.

[Christopher Titchen (christophertitchen)]

Christopher Titchen (@christophertitchen) can you restore your numba function? It'll take me a bit longer to do the coreforecast release because I want to refactor all the single array functions. Sorry for the troubles.

No worries!

[José Morales (jmoralez)]

Thanks!