Dask and Optuna for Hyper Parameter Optimization

[@UmmadikTas]

Thank you so much. This is very helpful with my research.

[@UmmadikTas]

I had an issue with parallelization and the random sampler for hyperparameter search. When I submit optimize function in parallel, optuna keeps repeating the same hyper-paremeters across all processes. I could not figure out how to reseed the sampler for different processes.

[@Coiled]

Are the different processes communcating hyperparameters with a central Optuna Storage object? This video shows using the DaskStorage, which helps all of the Optuna search functions coordinate and share results between each other using Dask. Other ways to do this include using things like a database (although we think that Dask is easier).