Thanks for the kind words. ❤️

bayesian optimization

Sounds useful. We should certainly look into using that package as a library:

https://github.com/fmfn/BayesianOptimization#1-specifying-the-function-to-be-optimized

In most cases probably even something as simple as random grid search should work much faster, while still giving sufficient ballpark results:

I would suggest using scikit-optimize? The random search is not guaranteed to converge faster to the optimum point because it does not learn from previous trial points.

We could use skopt as our constraints do match. 👍 The question is, will you implement it? Random search should be roughly a 5-line change.

Sure, if nothing comes up I'll work on this in the next few days. I'll also look into skopt. So, we can just add a parameter method to optimize() method, and that way, different techniques of optimization can be chosen based on the input for method param.

Exactly. For random search I also envisioned an argument: max_tries: Union[int, float] = 200. Integer for absolute; float for relative to exhaustive search. This should then match n_calls= for skopt.

I have implemented skopt's forest_minimize() method with encouraging results. Just need to validate the things and I also think it would be helpful to add the partial dependency plots. I am looking at how you've used bokeh and will add a function to plot this after the optimization. Hopefully will have something for your review next weekend.

it would be helpful to add the partial dependency plots

Think you can just return the raw scipy.optimize.OptimizeResult that skopt.plots.plot_objective() (and possibly other utils) already knows how to work with? No need to reinvent the wheel. Instead, we should document the workflow.

We should, however, look into adopting the lower-triangular subplot layout for the backtesting.lib.plot_heatmaps(). That makes it all much clearer!

Yes, plotting the optimization results in a jupyter notebook is straightforward with plot_objective (and that's how I plotted the charts above for the Quick Start example). Though I am not sure if it would be as straightforward to integrate that with bokeh. The only thing that might be possible if we can export the plot_objective plots as PNG and bokeh can use them as image tag in their html file? It just won't be interactive like native bokeh charts.

I quickly looked at the heatmap plots from backtesting.lib.plot_heatmaps() and they are not as nice and clear with contour lines and test points etc.

Though I am not sure if it would be as straightforward to integrate that with bokeh.

I don't fully understand why we'd want it to integrate it with Bokeh. plot_objective() will output into notebooks just as well, and the returned matplotlib.Axes can be savefig'd for an image. Sure, the plot won't be interactive, but that's a miniature price to pay for not needing to write/maintain any extra code? What I'm suggesting, therefore, is that we just return the OptimizeResult and mark a note in the docs that the user might want to run skopt.plots.plot_objective() upon it. It's perfectly ok to mix best of breed utilities from the ecosystem. 😁

I quickly looked at the heatmap plots from backtesting.lib.plot_heatmaps() and they are not as nice and clear with contour lines and test points etc.

They are not. It's just an unrelated note-to-self proposal that we lay them out sorted in a more immediately clear lower-triangular instead of the simple grid layout.

this should be on the top priority! big win if we have this feature!