{"action":"created","issue":{"url":"https://api.github.com/repos/deeplearning4j/Arbiter/issues/45","repository_url":"https://api.github.com/repos/deeplearning4j/Arbiter","labels_url":"https://api.github.com/repos/deeplearning4j/Arbiter/issues/45/labels{/name}","comments_url":"https://api.github.com/repos/deeplearning4j/Arbiter/issues/45/comments","events_url":"https://api.github.com/repos/deeplearning4j/Arbiter/issues/45/events","html_url":"https://github.com/deeplearning4j/Arbiter/issues/45","id":184538169,"number":45,"title":"Adding New hyperparameter optimization tools","user":{"login":"ianpjohnson","id":19364563,"avatar_url":"https://avatars.githubusercontent.com/u/19364563?v=3","gravatar_id":"","url":"https://api.github.com/users/ianpjohnson","html_url":"https://github.com/ianpjohnson","followers_url":"https://api.github.com/users/ianpjohnson/followers","following_url":"https://api.github.com/users/ianpjohnson/following{/other_user}","gists_url":"https://api.github.com/users/ianpjohnson/gists{/gist_id}","starred_url":"https://api.github.com/users/ianpjohnson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ianpjohnson/subscriptions","organizations_url":"https://api.github.com/users/ianpjohnson/orgs","repos_url":"https://api.github.com/users/ianpjohnson/repos","events_url":"https://api.github.com/users/ianpjohnson/events{/privacy}","received_events_url":"https://api.github.com/users/ianpjohnson/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2016-10-21T17:56:40Z","updated_at":"2016-10-22T13:07:15Z","closed_at":null,"body":"The current Arbiter has a couple of starter hyperparameter  strategies: Grid & Random Search (and apparently a closed Spearmint) . It would be nice to enhance these with existing \"best of breed\" tools out there (e.g. Spearmint. SMAC, TPE, BayesOpt etc and/or derivates)\r\n\r\nOne problem is that many of these are implemented in Python/C++ or other languages, so it would be good if we had a way to allow their use without necessarily porting them to Java. Usually it is possible to invoke the DL4J runs from the command line but it is clumsy and non trivial to integrate with the existing DL4J optimization  mechanisms. If external tools can be integrated with limited engineering effort they can potential be part of future Arbiter mechanisms perhaps including adversarial, parallel etc where we might run several  hyperparameter tools concurrently and/or in competition\r\n\r\nTo this end i am proposing to \"bridge\" into the existing hyperparameter  mechanism by implementing \"external\" strategies. They might be connected, for example, by sockets or other mechanisms.\r\n\r\nA few things this bridging mechanism might be able to do would be:\r\n* To export the hyperparameter  ranges/types/constraints to the external tool - each has its own format for defining parameters and it would be nice to keep these in 1 place (Arbiter or another common format) and translate automatically for each tool preferably on the fly\r\n* To pool the experiment results and allow each tool to see the results from other tools to save them repeating the same or similar experiments, or to add existing results to their own space for further analysis"},"comment":{"url":"https://api.github.com/repos/deeplearning4j/Arbiter/issues/comments/255527389","html_url":"https://github.com/deeplearning4j/Arbiter/issues/45#issuecomment-255527389","issue_url":"https://api.github.com/repos/deeplearning4j/Arbiter/issues/45","id":255527389,"user":{"login":"ianpjohnson","id":19364563,"avatar_url":"https://avatars.githubusercontent.com/u/19364563?v=3","gravatar_id":"","url":"https://api.github.com/users/ianpjohnson","html_url":"https://github.com/ianpjohnson","followers_url":"https://api.github.com/users/ianpjohnson/followers","following_url":"https://api.github.com/users/ianpjohnson/following{/other_user}","gists_url":"https://api.github.com/users/ianpjohnson/gists{/gist_id}","starred_url":"https://api.github.com/users/ianpjohnson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ianpjohnson/subscriptions","organizations_url":"https://api.github.com/users/ianpjohnson/orgs","repos_url":"https://api.github.com/users/ianpjohnson/repos","events_url":"https://api.github.com/users/ianpjohnson/events{/privacy}","received_events_url":"https://api.github.com/users/ianpjohnson/received_events","type":"User","site_admin":false},"created_at":"2016-10-22T13:07:14Z","updated_at":"2016-10-22T13:07:14Z","body":"Further to my comments on the earlyadopters channel:\r\nAm looking at plugging a \"concrete candidate\" into the candidate generation framework - from the CandidateSearch aspect, which seems a semantically appropriate place. I see a graph of Parameters in a space - each is a mapping/generation mechanism. I would like to pass a concrete value from an external optimisation tool without any distribution  mapping as the external tool might already be applying distribution mapping \r\n\r\nMaking each an FixedValue would be fine (with mutator) but is then not suited to switch in alternate search methods and i no longer have the meta data required to setup the external tool\r\n\r\nMy thought was simply to add a setValue to each leaf ParameterSpace type which, when used, overrides the getValue implementation.  This avoid issues with discretization and multiple distribution mappings (though my jury is still out of retaining DL4J distribution mapping in some cases)"}}
{"actor":{"display_login":"ianpjohnson"}}