{"action":"created","comment":{"url":"https://api.github.com/repos/scipy/scipy/pulls/comments/214628899","pull_request_review_id":151724740,"id":214628899,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIxNDYyODg5OQ==","diff_hunk":"@@ -0,0 +1,557 @@\n+# Simulated Dual Annealing implementation.\n+# Copyright (c) 2018 Sylvain Gubian <sylvain.gubian@pmi.com>,\n+# Yang Xiang <yang.xiang@pmi.com>\n+# Author: Sylvain Gubian, PMP S.A.\n+\"\"\"\n+SDA: A Simulated Dual Annealing global optimization algorithm\n+\"\"\"\n+from __future__ import division, print_function, absolute_import\n+\n+import numpy as np\n+from scipy.optimize import OptimizeResult\n+from scipy.optimize import minimize\n+from scipy.special import gammaln\n+from scipy._lib._util import check_random_state\n+\n+__all__ = ['simulated_annealing']\n+BIG_VALUE = 1e16\n+\n+\n+class VisitingDistribution(object):\n+    \"\"\"\n+    Class used to generate new coordinates based on the distorted\n+    Cauchy-Lorentz distribution. Depending on the steps within the Markov\n+    chain, the class implements the strategy for generating new location\n+    changes.\n+    \"\"\"\n+    TAIL_LIMIT = 1.e8\n+    MIN_VISIT_BOUND = 1.e-10\n+\n+    def __init__(self, lb, ub, visiting_param, rand_state):\n+        self.visiting_param = visiting_param\n+        self.rand_state = rand_state\n+        self.lower = lb\n+        self.upper = ub\n+        self.bound_range = ub - lb\n+        self.x_gaussian_sample = None\n+        self.square_gaussian = 0\n+        self.square_root_gaussian = None\n+\n+    def visiting(self, x, step, temperature):","path":"scipy/optimize/_simulated_annealing.py","position":40,"original_position":40,"commit_id":"be1b7d08a37a7191797efcb11bbf6d7e086d2c6f","original_commit_id":"be1b7d08a37a7191797efcb11bbf6d7e086d2c6f","user":{"login":"sgubianpm","id":17434537,"node_id":"MDQ6VXNlcjE3NDM0NTM3","avatar_url":"https://avatars0.githubusercontent.com/u/17434537?v=4","gravatar_id":"","url":"https://api.github.com/users/sgubianpm","html_url":"https://github.com/sgubianpm","followers_url":"https://api.github.com/users/sgubianpm/followers","following_url":"https://api.github.com/users/sgubianpm/following{/other_user}","gists_url":"https://api.github.com/users/sgubianpm/gists{/gist_id}","starred_url":"https://api.github.com/users/sgubianpm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sgubianpm/subscriptions","organizations_url":"https://api.github.com/users/sgubianpm/orgs","repos_url":"https://api.github.com/users/sgubianpm/repos","events_url":"https://api.github.com/users/sgubianpm/events{/privacy}","received_events_url":"https://api.github.com/users/sgubianpm/received_events","type":"User","site_admin":false},"body":"Docstring added. Thanks for that.","created_at":"2018-09-03T09:35:23Z","updated_at":"2018-09-03T09:35:23Z","html_url":"https://github.com/scipy/scipy/pull/8203#discussion_r214628899","pull_request_url":"https://api.github.com/repos/scipy/scipy/pulls/8203","author_association":"NONE","_links":{"self":{"href":"https://api.github.com/repos/scipy/scipy/pulls/comments/214628899"},"html":{"href":"https://github.com/scipy/scipy/pull/8203#discussion_r214628899"},"pull_request":{"href":"https://api.github.com/repos/scipy/scipy/pulls/8203"}},"in_reply_to_id":176964561},"pull_request":{"url":"https://api.github.com/repos/scipy/scipy/pulls/8203","id":156956080,"node_id":"MDExOlB1bGxSZXF1ZXN0MTU2OTU2MDgw","html_url":"https://github.com/scipy/scipy/pull/8203","diff_url":"https://github.com/scipy/scipy/pull/8203.diff","patch_url":"https://github.com/scipy/scipy/pull/8203.patch","issue_url":"https://api.github.com/repos/scipy/scipy/issues/8203","number":8203,"state":"open","locked":false,"title":"ENH: adding simulated dual annealing to optimize","user":{"login":"sgubianpm","id":17434537,"node_id":"MDQ6VXNlcjE3NDM0NTM3","avatar_url":"https://avatars0.githubusercontent.com/u/17434537?v=4","gravatar_id":"","url":"https://api.github.com/users/sgubianpm","html_url":"https://github.com/sgubianpm","followers_url":"https://api.github.com/users/sgubianpm/followers","following_url":"https://api.github.com/users/sgubianpm/following{/other_user}","gists_url":"https://api.github.com/users/sgubianpm/gists{/gist_id}","starred_url":"https://api.github.com/users/sgubianpm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sgubianpm/subscriptions","organizations_url":"https://api.github.com/users/sgubianpm/orgs","repos_url":"https://api.github.com/users/sgubianpm/repos","events_url":"https://api.github.com/users/sgubianpm/events{/privacy}","received_events_url":"https://api.github.com/users/sgubianpm/received_events","type":"User","site_admin":false},"body":"Dear All,\r\n\r\nOld related PRs:\r\n#6569\r\n#6197\r\n\r\nThanks a lot for the time you spent on reviewing these PRs that you did quite long time ago now. Last @rgommers comment about adding a needs-work yellow label and @js850 commented that refactoring was needed as well. So I did the recent weeks.\r\n\r\nFirst of all, the code has been refactored and cut into smaller classes pieces so that it gained a lot in readability while addressing as much as possible all of your comments.\r\n\r\nSecondly, in the meantime, the method has evolved internally and relies now on Simulated Dual Annealing (and therefore has been renamed into SDA). It is showing even better results than the previous implementation in terms of successful rate and number of objective function calls.\r\n\r\nIn addition, as suggested by @js850, the possibility to use any of the SciPy minimizer to operate the local search has been added.\r\n\r\nMaybe this changes could go towards a foreseeable merge.\r\n\r\nBest regards,\r\n\r\nSylvain.\r\n\r\nAs mentioned by @rgommers, I haven't attached the  benchmarks results in this PR (thanks for reminding me). So, below is the summary for the benchmarks.\r\nThe benchmark has been done by using the SciPy global optimization testing functions implemented by SciPy contributors plus some of these functions using bigger dimensions (from 2 to 100 components). The SDA method has been tested against methods available in SciPy but also other Python available global optimizers like the [NLOpt](https://nlopt.readthedocs.io/en/latest/NLopt_Python_Reference/) suite, [PSO](https://pythonhosted.org/pyswarm/) (particle swarm) and [CMAES](https://github.com/CMA-ES/pycma) which is the is the Covariance Matrix Adaptation Evolution Strategy implementation. A cluster infrastructure has been used to execute 100 runs with different random initial points for the 250 testing functions over the 14 tested methods.\r\n(The internal SciPy benchmark has not been used as it not meant to be run on a cluster, it does use the default testing function dimensions and it is dedicated to SciPy global optimization methods). \r\nBelow figure are showing the overall results of the benchmark.\r\nESCH, MLSL, ISRES and CRS are NLOpt implementations, TSA stands for traditional simulated annealing. BF, BH and DE are the available SciPy methods (brute_force, basin_hopping, differential evolution). \r\n\r\n![sda_bench](https://user-images.githubusercontent.com/17434537/33834032-e39ebd9e-de81-11e7-9138-c1ab4d61376d.png)\r\n\r\n(ANFC: Average number of objective function call used to reach the global optimum value (with a 1.e-8 absolute tolerance value).","created_at":"2017-12-07T10:28:49Z","updated_at":"2018-09-03T09:35:23Z","closed_at":null,"merged_at":null,"merge_commit_sha":"bd4244ea83249e4f29e7fa4e68d2381f6dcb4c18","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":37340735,"node_id":"MDU6TGFiZWwzNzM0MDczNQ==","url":"https://api.github.com/repos/scipy/scipy/labels/enhancement","name":"enhancement","color":"bfe5bf","default":true},{"id":37341011,"node_id":"MDU6TGFiZWwzNzM0MTAxMQ==","url":"https://api.github.com/repos/scipy/scipy/labels/scipy.optimize","name":"scipy.optimize","color":"207de5","default":false}],"milestone":{"url":"https://api.github.com/repos/scipy/scipy/milestones/36","html_url":"https://github.com/scipy/scipy/milestone/36","labels_url":"https://api.github.com/repos/scipy/scipy/milestones/36/labels","id":3257163,"node_id":"MDk6TWlsZXN0b25lMzI1NzE2Mw==","number":36,"title":"1.2.0","description":"","creator":{"login":"pv","id":35046,"node_id":"MDQ6VXNlcjM1MDQ2","avatar_url":"https://avatars1.githubusercontent.com/u/35046?v=4","gravatar_id":"","url":"https://api.github.com/users/pv","html_url":"https://github.com/pv","followers_url":"https://api.github.com/users/pv/followers","following_url":"https://api.github.com/users/pv/following{/other_user}","gists_url":"https://api.github.com/users/pv/gists{/gist_id}","starred_url":"https://api.github.com/users/pv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pv/subscriptions","organizations_url":"https://api.github.com/users/pv/orgs","repos_url":"https://api.github.com/users/pv/repos","events_url":"https://api.github.com/users/pv/events{/privacy}","received_events_url":"https://api.github.com/users/pv/received_events","type":"User","site_admin":false},"open_issues":36,"closed_issues":162,"state":"open","created_at":"2018-04-10T20:26:02Z","updated_at":"2018-09-03T05:13:09Z","due_on":"2018-11-09T08:00:00Z","closed_at":null},"commits_url":"https://api.github.com/repos/scipy/scipy/pulls/8203/commits","review_comments_url":"https://api.github.com/repos/scipy/scipy/pulls/8203/comments","review_comment_url":"https://api.github.com/repos/scipy/scipy/pulls/comments{/number}","comments_url":"https://api.github.com/repos/scipy/scipy/issues/8203/comments","statuses_url":"https://api.github.com/repos/scipy/scipy/statuses/be1b7d08a37a7191797efcb11bbf6d7e086d2c6f","head":{"label":"sgubianpm:ENH-adding-sda-optimization","ref":"ENH-adding-sda-optimization","sha":"be1b7d08a37a7191797efcb11bbf6d7e086d2c6f","user":{"login":"sgubianpm","id":17434537,"node_id":"MDQ6VXNlcjE3NDM0NTM3","avatar_url":"https://avatars0.githubusercontent.com/u/17434537?v=4","gravatar_id":"","url":"https://api.github.com/users/sgubianpm","html_url":"https://github.com/sgubianpm","followers_url":"https://api.github.com/users/sgubianpm/followers","following_url":"https://api.github.com/users/sgubianpm/following{/other_user}","gists_url":"https://api.github.com/users/sgubianpm/gists{/gist_id}","starred_url":"https://api.github.com/users/sgubianpm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sgubianpm/subscriptions","organizations_url":"https://api.github.com/users/sgubianpm/orgs","repos_url":"https://api.github.com/users/sgubianpm/repos","events_url":"https://api.github.com/users/sgubianpm/events{/privacy}","received_events_url":"https://api.github.com/users/sgubianpm/received_events","type":"User","site_admin":false},"repo":{"id":52374389,"node_id":"MDEwOlJlcG9zaXRvcnk1MjM3NDM4OQ==","name":"scipy","full_name":"sgubianpm/scipy","owner":{"login":"sgubianpm","id":17434537,"node_id":"MDQ6VXNlcjE3NDM0NTM3","avatar_url":"https://avatars0.githubusercontent.com/u/17434537?v=4","gravatar_id":"","url":"https://api.github.com/users/sgubianpm","html_url":"https://github.com/sgubianpm","followers_url":"https://api.github.com/users/sgubianpm/followers","following_url":"https://api.github.com/users/sgubianpm/following{/other_user}","gists_url":"https://api.github.com/users/sgubianpm/gists{/gist_id}","starred_url":"https://api.github.com/users/sgubianpm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sgubianpm/subscriptions","organizations_url":"https://api.github.com/users/sgubianpm/orgs","repos_url":"https://api.github.com/users/sgubianpm/repos","events_url":"https://api.github.com/users/sgubianpm/events{/privacy}","received_events_url":"https://api.github.com/users/sgubianpm/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/sgubianpm/scipy","description":"Scipy library main repository","fork":true,"url":"https://api.github.com/repos/sgubianpm/scipy","forks_url":"https://api.github.com/repos/sgubianpm/scipy/forks","keys_url":"https://api.github.com/repos/sgubianpm/scipy/keys{/key_id}","collaborators_url":"https://api.github.com/repos/sgubianpm/scipy/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/sgubianpm/scipy/teams","hooks_url":"https://api.github.com/repos/sgubianpm/scipy/hooks","issue_events_url":"https://api.github.com/repos/sgubianpm/scipy/issues/events{/number}","events_url":"https://api.github.com/repos/sgubianpm/scipy/events","assignees_url":"https://api.github.com/repos/sgubianpm/scipy/assignees{/user}","branches_url":"https://api.github.com/repos/sgubianpm/scipy/branches{/branch}","tags_url":"https://api.github.com/repos/sgubianpm/scipy/tags","blobs_url":"https://api.github.com/repos/sgubianpm/scipy/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/sgubianpm/scipy/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/sgubianpm/scipy/git/refs{/sha}","trees_url":"https://api.github.com/repos/sgubianpm/scipy/git/trees{/sha}","statuses_url":"https://api.github.com/repos/sgubianpm/scipy/statuses/{sha}","languages_url":"https://api.github.com/repos/sgubianpm/scipy/languages","stargazers_url":"https://api.github.com/repos/sgubianpm/scipy/stargazers","contributors_url":"https://api.github.com/repos/sgubianpm/scipy/contributors","subscribers_url":"https://api.github.com/repos/sgubianpm/scipy/subscribers","subscription_url":"https://api.github.com/repos/sgubianpm/scipy/subscription","commits_url":"https://api.github.com/repos/sgubianpm/scipy/commits{/sha}","git_commits_url":"https://api.github.com/repos/sgubianpm/scipy/git/commits{/sha}","comments_url":"https://api.github.com/repos/sgubianpm/scipy/comments{/number}","issue_comment_url":"https://api.github.com/repos/sgubianpm/scipy/issues/comments{/number}","contents_url":"https://api.github.com/repos/sgubianpm/scipy/contents/{+path}","compare_url":"https://api.github.com/repos/sgubianpm/scipy/compare/{base}...{head}","merges_url":"https://api.github.com/repos/sgubianpm/scipy/merges","archive_url":"https://api.github.com/repos/sgubianpm/scipy/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/sgubianpm/scipy/downloads","issues_url":"https://api.github.com/repos/sgubianpm/scipy/issues{/number}","pulls_url":"https://api.github.com/repos/sgubianpm/scipy/pulls{/number}","milestones_url":"https://api.github.com/repos/sgubianpm/scipy/milestones{/number}","notifications_url":"https://api.github.com/repos/sgubianpm/scipy/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/sgubianpm/scipy/labels{/name}","releases_url":"https://api.github.com/repos/sgubianpm/scipy/releases{/id}","deployments_url":"https://api.github.com/repos/sgubianpm/scipy/deployments","created_at":"2016-02-23T16:39:07Z","updated_at":"2016-02-23T16:39:17Z","pushed_at":"2018-03-24T15:55:49Z","git_url":"git://github.com/sgubianpm/scipy.git","ssh_url":"git@github.com:sgubianpm/scipy.git","clone_url":"https://github.com/sgubianpm/scipy.git","svn_url":"https://github.com/sgubianpm/scipy","homepage":"http://scipy.org/scipylib/","size":75688,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"scipy:master","ref":"master","sha":"70e61dee181de23fdd8d893eaa9491100e2218d7","user":{"login":"scipy","id":288277,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI4ODI3Nw==","avatar_url":"https://avatars2.githubusercontent.com/u/288277?v=4","gravatar_id":"","url":"https://api.github.com/users/scipy","html_url":"https://github.com/scipy","followers_url":"https://api.github.com/users/scipy/followers","following_url":"https://api.github.com/users/scipy/following{/other_user}","gists_url":"https://api.github.com/users/scipy/gists{/gist_id}","starred_url":"https://api.github.com/users/scipy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scipy/subscriptions","organizations_url":"https://api.github.com/users/scipy/orgs","repos_url":"https://api.github.com/users/scipy/repos","events_url":"https://api.github.com/users/scipy/events{/privacy}","received_events_url":"https://api.github.com/users/scipy/received_events","type":"Organization","site_admin":false},"repo":{"id":1460385,"node_id":"MDEwOlJlcG9zaXRvcnkxNDYwMzg1","name":"scipy","full_name":"scipy/scipy","owner":{"login":"scipy","id":288277,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI4ODI3Nw==","avatar_url":"https://avatars2.githubusercontent.com/u/288277?v=4","gravatar_id":"","url":"https://api.github.com/users/scipy","html_url":"https://github.com/scipy","followers_url":"https://api.github.com/users/scipy/followers","following_url":"https://api.github.com/users/scipy/following{/other_user}","gists_url":"https://api.github.com/users/scipy/gists{/gist_id}","starred_url":"https://api.github.com/users/scipy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scipy/subscriptions","organizations_url":"https://api.github.com/users/scipy/orgs","repos_url":"https://api.github.com/users/scipy/repos","events_url":"https://api.github.com/users/scipy/events{/privacy}","received_events_url":"https://api.github.com/users/scipy/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/scipy/scipy","description":"Scipy library main repository","fork":false,"url":"https://api.github.com/repos/scipy/scipy","forks_url":"https://api.github.com/repos/scipy/scipy/forks","keys_url":"https://api.github.com/repos/scipy/scipy/keys{/key_id}","collaborators_url":"https://api.github.com/repos/scipy/scipy/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/scipy/scipy/teams","hooks_url":"https://api.github.com/repos/scipy/scipy/hooks","issue_events_url":"https://api.github.com/repos/scipy/scipy/issues/events{/number}","events_url":"https://api.github.com/repos/scipy/scipy/events","assignees_url":"https://api.github.com/repos/scipy/scipy/assignees{/user}","branches_url":"https://api.github.com/repos/scipy/scipy/branches{/branch}","tags_url":"https://api.github.com/repos/scipy/scipy/tags","blobs_url":"https://api.github.com/repos/scipy/scipy/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/scipy/scipy/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/scipy/scipy/git/refs{/sha}","trees_url":"https://api.github.com/repos/scipy/scipy/git/trees{/sha}","statuses_url":"https://api.github.com/repos/scipy/scipy/statuses/{sha}","languages_url":"https://api.github.com/repos/scipy/scipy/languages","stargazers_url":"https://api.github.com/repos/scipy/scipy/stargazers","contributors_url":"https://api.github.com/repos/scipy/scipy/contributors","subscribers_url":"https://api.github.com/repos/scipy/scipy/subscribers","subscription_url":"https://api.github.com/repos/scipy/scipy/subscription","commits_url":"https://api.github.com/repos/scipy/scipy/commits{/sha}","git_commits_url":"https://api.github.com/repos/scipy/scipy/git/commits{/sha}","comments_url":"https://api.github.com/repos/scipy/scipy/comments{/number}","issue_comment_url":"https://api.github.com/repos/scipy/scipy/issues/comments{/number}","contents_url":"https://api.github.com/repos/scipy/scipy/contents/{+path}","compare_url":"https://api.github.com/repos/scipy/scipy/compare/{base}...{head}","merges_url":"https://api.github.com/repos/scipy/scipy/merges","archive_url":"https://api.github.com/repos/scipy/scipy/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/scipy/scipy/downloads","issues_url":"https://api.github.com/repos/scipy/scipy/issues{/number}","pulls_url":"https://api.github.com/repos/scipy/scipy/pulls{/number}","milestones_url":"https://api.github.com/repos/scipy/scipy/milestones{/number}","notifications_url":"https://api.github.com/repos/scipy/scipy/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/scipy/scipy/labels{/name}","releases_url":"https://api.github.com/repos/scipy/scipy/releases{/id}","deployments_url":"https://api.github.com/repos/scipy/scipy/deployments","created_at":"2011-03-09T18:52:03Z","updated_at":"2018-09-03T04:35:49Z","pushed_at":"2018-09-03T06:03:08Z","git_url":"git://github.com/scipy/scipy.git","ssh_url":"git@github.com:scipy/scipy.git","clone_url":"https://github.com/scipy/scipy.git","svn_url":"https://github.com/scipy/scipy","homepage":"https://scipy.org/scipylib/","size":81590,"stargazers_count":4856,"watchers_count":4856,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":2466,"mirror_url":null,"archived":false,"open_issues_count":1272,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":2466,"open_issues":1272,"watchers":4856,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/scipy/scipy/pulls/8203"},"html":{"href":"https://github.com/scipy/scipy/pull/8203"},"issue":{"href":"https://api.github.com/repos/scipy/scipy/issues/8203"},"comments":{"href":"https://api.github.com/repos/scipy/scipy/issues/8203/comments"},"review_comments":{"href":"https://api.github.com/repos/scipy/scipy/pulls/8203/comments"},"review_comment":{"href":"https://api.github.com/repos/scipy/scipy/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/scipy/scipy/pulls/8203/commits"},"statuses":{"href":"https://api.github.com/repos/scipy/scipy/statuses/be1b7d08a37a7191797efcb11bbf6d7e086d2c6f"}},"author_association":"NONE"}}
{"actor":{"display_login":"sgubianpm"}}