{"action":"created","issue":{"url":"https://api.github.com/repos/facebookresearch/nevergrad/issues/4","repository_url":"https://api.github.com/repos/facebookresearch/nevergrad","labels_url":"https://api.github.com/repos/facebookresearch/nevergrad/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/facebookresearch/nevergrad/issues/4/comments","events_url":"https://api.github.com/repos/facebookresearch/nevergrad/issues/4/events","html_url":"https://github.com/facebookresearch/nevergrad/issues/4","id":393305293,"node_id":"MDU6SXNzdWUzOTMzMDUyOTM=","number":4,"title":"How to handle randomness in optimizer","user":{"login":"se4u","id":2301364,"node_id":"MDQ6VXNlcjIzMDEzNjQ=","avatar_url":"https://avatars3.githubusercontent.com/u/2301364?v=4","gravatar_id":"","url":"https://api.github.com/users/se4u","html_url":"https://github.com/se4u","followers_url":"https://api.github.com/users/se4u/followers","following_url":"https://api.github.com/users/se4u/following{/other_user}","gists_url":"https://api.github.com/users/se4u/gists{/gist_id}","starred_url":"https://api.github.com/users/se4u/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/se4u/subscriptions","organizations_url":"https://api.github.com/users/se4u/orgs","repos_url":"https://api.github.com/users/se4u/repos","events_url":"https://api.github.com/users/se4u/events{/privacy}","received_events_url":"https://api.github.com/users/se4u/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-12-21T04:02:44Z","updated_at":"2018-12-21T10:13:47Z","closed_at":null,"author_association":"NONE","body":"Hi, I have prevously worked¹ on a gradient-free optimization algorithm called SPSA²-³, and I have Matlab/mex code⁴ that I can port to python easily. I am interested in benchmarking SPSA against other zero-order optimization algorithms using nevergrad. \r\n\r\nI am following the instructions for benchmarking a new optimizer given in [adding-your-own-experiments-andor-optimizers-andor-function](https://github.com/facebookresearch/nevergrad#adding-your-own-experiments-andor-optimizers-andor-functions). My understanding is that I can just add a new `SPSA` class in `nevergrad/optimization/optimizerlib.py` and implement \r\n\r\n* __init\r\n* _internal_ask\r\n* _internal_provide_recommendation\r\n* _internal_tell\r\n\r\nfunctions and then add `SPSA` to the `optims` variable in the right experiment function in the `nevergrad/benchmark/experiments.py` module and then I should be able to generate graphs like \r\n[docs/resources/noise_r400s12_xpresults_namecigar,rotationTrue.png](https://github.com/facebookresearch/nevergrad/blob/master/docs/resources/noise_r400s12_xpresults_namecigar,rotationTrue.png)\r\n\r\nHowever, SPSA itself uses an rng in the `_internal_ask` function. But the `optimizer` `base` class does not take any seed in the `__init__` function. What will be a good way to make the experiments reproducible in such situation? \r\n\r\n[1] Pushpendre Rastogi, Jingyi Zhu, James C. Spall (2016). Efficient implementation of Enhanced Adaptive Simultaneous Perturbation Algorithms. CISS 2016, pdf\r\n[2] https://en.wikipedia.org/wiki/Simultaneous_perturbation_stochastic_approximation\r\n[3] https://www.chessprogramming.org/SPSA\r\n[4] https://github.com/se4u/FASPSA/blob/master/src/SPSA.m"},"comment":{"url":"https://api.github.com/repos/facebookresearch/nevergrad/issues/comments/449345752","html_url":"https://github.com/facebookresearch/nevergrad/issues/4#issuecomment-449345752","issue_url":"https://api.github.com/repos/facebookresearch/nevergrad/issues/4","id":449345752,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0OTM0NTc1Mg==","user":{"login":"tomjorquera","id":4150873,"node_id":"MDQ6VXNlcjQxNTA4NzM=","avatar_url":"https://avatars3.githubusercontent.com/u/4150873?v=4","gravatar_id":"","url":"https://api.github.com/users/tomjorquera","html_url":"https://github.com/tomjorquera","followers_url":"https://api.github.com/users/tomjorquera/followers","following_url":"https://api.github.com/users/tomjorquera/following{/other_user}","gists_url":"https://api.github.com/users/tomjorquera/gists{/gist_id}","starred_url":"https://api.github.com/users/tomjorquera/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tomjorquera/subscriptions","organizations_url":"https://api.github.com/users/tomjorquera/orgs","repos_url":"https://api.github.com/users/tomjorquera/repos","events_url":"https://api.github.com/users/tomjorquera/events{/privacy}","received_events_url":"https://api.github.com/users/tomjorquera/received_events","type":"User","site_admin":false},"created_at":"2018-12-21T10:13:47Z","updated_at":"2018-12-21T10:13:47Z","author_association":"NONE","body":"@se4u not one of the authors here, but it seems the lib is relying on `random` and `numpy.random` for these matter. In both cases, you can set the seed globally by doing `random.seed(1234) or `np.random.seed(1234)` to make subsequent calls to rngs reproducibles.\r\n\r\nYou can see how the authors use this here https://github.com/facebookresearch/nevergrad/blob/7fb29affd443330d8b75936856f43182a2d3b233/nevergrad/benchmark/xpbase.py#L119-L124\r\n\r\nSo I guess if you are able to use base python or numpy random, you should be able to get reproducible experiments the same way."}}
{"actor":{"display_login":"tomjorquera"}}