{"action":"created","comment":{"url":"https://api.github.com/repos/scipy/scipy-articles/pulls/comments/177299840","pull_request_review_id":107136174,"id":177299840,"diff_hunk":"@@ -0,0 +1,118 @@\n+\\subsubsection*{Solving nonlinear optimization problems}\n+\n+\\begin{table}[]\n+  \\centering\n+  \\begin{tabular}{cccccccccc}\n+      & \\rotatebox{60}{\\texttt{Nelder-Mead}} & \\rotatebox{60}{\\texttt{Powell}} & \\rotatebox{60}{\\texttt{CG}} & \\rotatebox{60}{\\texttt{BFGS}}& \\rotatebox{60}{\\texttt{Newton-CG}} & \\rotatebox{60}{\\texttt{dogleg}} & \\rotatebox{60}{\\texttt{trust-ncg}} & \\rotatebox{60}{\\texttt{trust-exact}} & \\rotatebox{60}{\\texttt{trust-krylov}} \\\\\n+    \\hline\n+    Version added & $<$0.7 & $<$0.7 & $<$0.7 & $<$0.7 & $<$0.7 & 0.13 & 0.13 & 0.19 & 1.0 \\\\\n+    \\hline\n+    It is a wrapper? &  &  & & &  &  & & & \\cmark \\\\\n+    \\hline\n+    \\nth{1} derivatives &  & & \\cmark & \\cmark & \\cmark & \\cmark & \\cmark & \\cmark & \\cmark \\\\\n+    \\hline\n+    \\nth{2} derivatives &  &  &  & Approx. & \\cmark & \\cmark & \\cmark & \\cmark & \\cmark \\\\\n+    \\hline\n+    \\makecell{Iterative Hessian \\\\\n+    factorization} & & & & & \\cmark &  & \\cmark &  & \\cmark \\\\\n+    \\hline\n+    Local convergence& & & L & S & S & Q & S & Q & ?  \\\\\n+    \\hline\n+    Global convergence & & &  & \\cmark & \\cmark & \\cmark & \\cmark & \\cmark & ?  \\\\\n+    \\hline\n+    \\makecell{Line-search (LS) or\\\\ trust-region (TR)} & Neither  & LS & LS & LS & LS & TR & TR & TR & TR \\\\\n+    \\hline\n+    References & \\cite{nelder_simplex_1965, wright_direct_1996} & \\cite{powell_efficient_1964} &\n+      \\cite{polak_note_1969, nocedal_numerical_2006} & \\cite{nocedal_numerical_2006} & \\cite{nocedal_numerical_2006}  &\n+      \\cite{powell_new_1970, nocedal_numerical_2006} &  \\cite{steihaug_conjugate_1983, nocedal_numerical_2006} &\n+      \\cite{conn_trust_2000, more_computing_1983} & \\cite{gould_solving_1999, lenders_trlib:_2016} \\\\\n+    \\hline\n+  \\end{tabular}\n+  \\caption{Optimization methods from \\texttt{minimize} for unconstrained minimization.  The field \\textit{It is a wrapper?}\n+    informs if the implementation available in SciPy is a wrapper to a function written in other programming language\n+    (e.g. C or FORTRAN). The fields \\textit{\\nth{1}} and \\textit{\\nth{2} derivatives}\n+    informs if the first or second order derivatives are required. When \\textit{\\nth{2} derivatives} is flagged\n+    with the word \\textit{Approx.} it means that, while the algorithm does not requires the second-order derivatives from\n+    the user, it computes an approximation internally and uses it to accelerate the method convergence.\n+    \\textit{Local convergence} gives\n+    a lower bound to the rate of convergence of the iterations sequence once the iterate is close enough to the\n+    objective function solution. The convergence rates mentioned at the table\n+    are, from the slowest to the fastest:  linear (L), superlinear (S) and quadratic (Q). The \\textit{global convergence}\n+    is marked for the algorithms for which there are some proof it always converge to some stationary\n+    point (i.e. a point $x^*$ for which $\\nabla f(x^*) = 0$).}\n+  \\label{tab:minimize-unconstrained}\n+\\end{table}\n+\n+\n+The \\texttt{minimize} function provides methods for solving general non-linear optimization\n+problems. It unifies several methods under the same general interface in order to facilitate\n+interchanging between solvers with different characteristics.  Table~\\ref{tab:minimize-unconstrained} summarizes\n+all the methods available for solving unconstrained minimization problems of the type:\n+\\begin{equation}\n+  \\label{eq:unconstrained-minimization-prob}\n+  \\min_x f(x)\n+\\end{equation}\n+where $f$ is a multivariate function $f: \\mathbb{R}^n \\rightarrow \\mathbb{R}$ and $x \\in \\mathbb{R}^n$.\n+\n+There are a sufficient variety of methods to enable the user to solve problems with the diverse properties.\n+Derivative-free methods (e.g. \\texttt{Nelder-Mead} e \\texttt{Powell}) are appropriate for minimizing noisy objective\n+functions. When the derivatives are available, on the other hand, methods with faster convergence can also be used.\n+\n+There are algorithms implementing two competing strategies: line-search and trust-region. The so-called line-search (LS)\n+methods choose a direction and then find the step length that gives the (approximately) optimal point in the given\n+direction. Trust-region (TR) methods, on the other hand, fixes the maximum step length for which the local approximation\n+is valid and finds approximately the best point inside this \\textit{trust-region}.\n+\n+An example of trust-region method introduced in the latest releases is \\texttt{trust-exact}, which a  provides a fast convergence rate by solving \n+ almost exactly the trust-region subproblem. Its downside is it also require the Hessian\n+to be stored and factored every iteration, which may renders impossible the solution of\n+very large problems.\n+\n+Some available unconstrained methods are appropriate for large-scale problems\n+(i.e. problems with more than 10000 variables).\n+\\texttt{CG} method does not use second derivatives and is adequate to large-scale problems due to the low storage requirements.\n+\\texttt{Newton-CG}, \\texttt{trust-ncg} and \\texttt{trust-krylov} use the second order derivatives in order to get faster\n+convergence but are still appropriate for large-scale problems because they factorate the Hessian in a inexact and iterative way,","path":"scipy-1.0/minimize.tex","position":75,"original_position":75,"commit_id":"f37a3967f9f704dbc228fead393b2a6704ec2bb0","original_commit_id":"f37a3967f9f704dbc228fead393b2a6704ec2bb0","user":{"login":"mdhaber","id":6570539,"avatar_url":"https://avatars1.githubusercontent.com/u/6570539?v=4","gravatar_id":"","url":"https://api.github.com/users/mdhaber","html_url":"https://github.com/mdhaber","followers_url":"https://api.github.com/users/mdhaber/followers","following_url":"https://api.github.com/users/mdhaber/following{/other_user}","gists_url":"https://api.github.com/users/mdhaber/gists{/gist_id}","starred_url":"https://api.github.com/users/mdhaber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mdhaber/subscriptions","organizations_url":"https://api.github.com/users/mdhaber/orgs","repos_url":"https://api.github.com/users/mdhaber/repos","events_url":"https://api.github.com/users/mdhaber/events{/privacy}","received_events_url":"https://api.github.com/users/mdhaber/received_events","type":"User","site_admin":false},"body":"\"factorate\"->\"factorize\"","created_at":"2018-03-27T03:38:26Z","updated_at":"2018-03-27T03:50:22Z","html_url":"https://github.com/scipy/scipy-articles/pull/17#discussion_r177299840","pull_request_url":"https://api.github.com/repos/scipy/scipy-articles/pulls/17","author_association":"NONE","_links":{"self":{"href":"https://api.github.com/repos/scipy/scipy-articles/pulls/comments/177299840"},"html":{"href":"https://github.com/scipy/scipy-articles/pull/17#discussion_r177299840"},"pull_request":{"href":"https://api.github.com/repos/scipy/scipy-articles/pulls/17"}}},"pull_request":{"url":"https://api.github.com/repos/scipy/scipy-articles/pulls/17","id":177614754,"html_url":"https://github.com/scipy/scipy-articles/pull/17","diff_url":"https://github.com/scipy/scipy-articles/pull/17.diff","patch_url":"https://github.com/scipy/scipy-articles/pull/17.patch","issue_url":"https://api.github.com/repos/scipy/scipy-articles/issues/17","number":17,"state":"open","locked":false,"title":"Add subsection about `minimize` function","user":{"login":"antonior92","id":16557411,"avatar_url":"https://avatars0.githubusercontent.com/u/16557411?v=4","gravatar_id":"","url":"https://api.github.com/users/antonior92","html_url":"https://github.com/antonior92","followers_url":"https://api.github.com/users/antonior92/followers","following_url":"https://api.github.com/users/antonior92/following{/other_user}","gists_url":"https://api.github.com/users/antonior92/gists{/gist_id}","starred_url":"https://api.github.com/users/antonior92/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/antonior92/subscriptions","organizations_url":"https://api.github.com/users/antonior92/orgs","repos_url":"https://api.github.com/users/antonior92/repos","events_url":"https://api.github.com/users/antonior92/events{/privacy}","received_events_url":"https://api.github.com/users/antonior92/received_events","type":"User","site_admin":false},"body":"Hi all,\r\n\r\nI just wrote a first draft of the table+text for ``minimize`` function. I wrote a full paragraph about ``trust-exact`` and another about ``trust-krylov`` (which are the methods added more recently). And I tried to give a general overview of all methods in comparative terms.\r\n\r\n@felixlen can you help me fiding out the local convergence rate of ``trust-krylov`` and if there are guarantees of **global convergence** for it? I was not able to find this information by myself for this method. For the other unconstrained minimisation methods Nocedal book or original papers usually contained this kind of information.\r\n\r\n@rgommers I added a table entry about the version each optimization method was added to scipy. I was not able to find out when some of the early methods were added, so I just wrote ``<0.7``. Where can I look for this information?\r\n\r\nIf we have space, I think it would be nice to include some benchmark results in this section latter on.","created_at":"2018-03-27T02:19:50Z","updated_at":"2018-03-27T03:50:22Z","closed_at":null,"merged_at":null,"merge_commit_sha":"ef1d49d90b183a001f8b42b0f17ddcc060190e93","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/scipy/scipy-articles/pulls/17/commits","review_comments_url":"https://api.github.com/repos/scipy/scipy-articles/pulls/17/comments","review_comment_url":"https://api.github.com/repos/scipy/scipy-articles/pulls/comments{/number}","comments_url":"https://api.github.com/repos/scipy/scipy-articles/issues/17/comments","statuses_url":"https://api.github.com/repos/scipy/scipy-articles/statuses/f37a3967f9f704dbc228fead393b2a6704ec2bb0","head":{"label":"antonior92:minimize","ref":"minimize","sha":"f37a3967f9f704dbc228fead393b2a6704ec2bb0","user":{"login":"antonior92","id":16557411,"avatar_url":"https://avatars0.githubusercontent.com/u/16557411?v=4","gravatar_id":"","url":"https://api.github.com/users/antonior92","html_url":"https://github.com/antonior92","followers_url":"https://api.github.com/users/antonior92/followers","following_url":"https://api.github.com/users/antonior92/following{/other_user}","gists_url":"https://api.github.com/users/antonior92/gists{/gist_id}","starred_url":"https://api.github.com/users/antonior92/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/antonior92/subscriptions","organizations_url":"https://api.github.com/users/antonior92/orgs","repos_url":"https://api.github.com/users/antonior92/repos","events_url":"https://api.github.com/users/antonior92/events{/privacy}","received_events_url":"https://api.github.com/users/antonior92/received_events","type":"User","site_admin":false},"repo":{"id":126898221,"name":"scipy-articles","full_name":"antonior92/scipy-articles","owner":{"login":"antonior92","id":16557411,"avatar_url":"https://avatars0.githubusercontent.com/u/16557411?v=4","gravatar_id":"","url":"https://api.github.com/users/antonior92","html_url":"https://github.com/antonior92","followers_url":"https://api.github.com/users/antonior92/followers","following_url":"https://api.github.com/users/antonior92/following{/other_user}","gists_url":"https://api.github.com/users/antonior92/gists{/gist_id}","starred_url":"https://api.github.com/users/antonior92/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/antonior92/subscriptions","organizations_url":"https://api.github.com/users/antonior92/orgs","repos_url":"https://api.github.com/users/antonior92/repos","events_url":"https://api.github.com/users/antonior92/events{/privacy}","received_events_url":"https://api.github.com/users/antonior92/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/antonior92/scipy-articles","description":"Publications about Scipy","fork":true,"url":"https://api.github.com/repos/antonior92/scipy-articles","forks_url":"https://api.github.com/repos/antonior92/scipy-articles/forks","keys_url":"https://api.github.com/repos/antonior92/scipy-articles/keys{/key_id}","collaborators_url":"https://api.github.com/repos/antonior92/scipy-articles/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/antonior92/scipy-articles/teams","hooks_url":"https://api.github.com/repos/antonior92/scipy-articles/hooks","issue_events_url":"https://api.github.com/repos/antonior92/scipy-articles/issues/events{/number}","events_url":"https://api.github.com/repos/antonior92/scipy-articles/events","assignees_url":"https://api.github.com/repos/antonior92/scipy-articles/assignees{/user}","branches_url":"https://api.github.com/repos/antonior92/scipy-articles/branches{/branch}","tags_url":"https://api.github.com/repos/antonior92/scipy-articles/tags","blobs_url":"https://api.github.com/repos/antonior92/scipy-articles/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/antonior92/scipy-articles/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/antonior92/scipy-articles/git/refs{/sha}","trees_url":"https://api.github.com/repos/antonior92/scipy-articles/git/trees{/sha}","statuses_url":"https://api.github.com/repos/antonior92/scipy-articles/statuses/{sha}","languages_url":"https://api.github.com/repos/antonior92/scipy-articles/languages","stargazers_url":"https://api.github.com/repos/antonior92/scipy-articles/stargazers","contributors_url":"https://api.github.com/repos/antonior92/scipy-articles/contributors","subscribers_url":"https://api.github.com/repos/antonior92/scipy-articles/subscribers","subscription_url":"https://api.github.com/repos/antonior92/scipy-articles/subscription","commits_url":"https://api.github.com/repos/antonior92/scipy-articles/commits{/sha}","git_commits_url":"https://api.github.com/repos/antonior92/scipy-articles/git/commits{/sha}","comments_url":"https://api.github.com/repos/antonior92/scipy-articles/comments{/number}","issue_comment_url":"https://api.github.com/repos/antonior92/scipy-articles/issues/comments{/number}","contents_url":"https://api.github.com/repos/antonior92/scipy-articles/contents/{+path}","compare_url":"https://api.github.com/repos/antonior92/scipy-articles/compare/{base}...{head}","merges_url":"https://api.github.com/repos/antonior92/scipy-articles/merges","archive_url":"https://api.github.com/repos/antonior92/scipy-articles/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/antonior92/scipy-articles/downloads","issues_url":"https://api.github.com/repos/antonior92/scipy-articles/issues{/number}","pulls_url":"https://api.github.com/repos/antonior92/scipy-articles/pulls{/number}","milestones_url":"https://api.github.com/repos/antonior92/scipy-articles/milestones{/number}","notifications_url":"https://api.github.com/repos/antonior92/scipy-articles/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/antonior92/scipy-articles/labels{/name}","releases_url":"https://api.github.com/repos/antonior92/scipy-articles/releases{/id}","deployments_url":"https://api.github.com/repos/antonior92/scipy-articles/deployments","created_at":"2018-03-26T22:39:38Z","updated_at":"2018-03-26T22:39:40Z","pushed_at":"2018-03-27T02:15:02Z","git_url":"git://github.com/antonior92/scipy-articles.git","ssh_url":"git@github.com:antonior92/scipy-articles.git","clone_url":"https://github.com/antonior92/scipy-articles.git","svn_url":"https://github.com/antonior92/scipy-articles","homepage":null,"size":76,"stargazers_count":0,"watchers_count":0,"language":"TeX","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"scipy:master","ref":"master","sha":"44ecd4c196ab2016ad49e47193e632d5869b5c3a","user":{"login":"scipy","id":288277,"avatar_url":"https://avatars2.githubusercontent.com/u/288277?v=4","gravatar_id":"","url":"https://api.github.com/users/scipy","html_url":"https://github.com/scipy","followers_url":"https://api.github.com/users/scipy/followers","following_url":"https://api.github.com/users/scipy/following{/other_user}","gists_url":"https://api.github.com/users/scipy/gists{/gist_id}","starred_url":"https://api.github.com/users/scipy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scipy/subscriptions","organizations_url":"https://api.github.com/users/scipy/orgs","repos_url":"https://api.github.com/users/scipy/repos","events_url":"https://api.github.com/users/scipy/events{/privacy}","received_events_url":"https://api.github.com/users/scipy/received_events","type":"Organization","site_admin":false},"repo":{"id":66140682,"name":"scipy-articles","full_name":"scipy/scipy-articles","owner":{"login":"scipy","id":288277,"avatar_url":"https://avatars2.githubusercontent.com/u/288277?v=4","gravatar_id":"","url":"https://api.github.com/users/scipy","html_url":"https://github.com/scipy","followers_url":"https://api.github.com/users/scipy/followers","following_url":"https://api.github.com/users/scipy/following{/other_user}","gists_url":"https://api.github.com/users/scipy/gists{/gist_id}","starred_url":"https://api.github.com/users/scipy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scipy/subscriptions","organizations_url":"https://api.github.com/users/scipy/orgs","repos_url":"https://api.github.com/users/scipy/repos","events_url":"https://api.github.com/users/scipy/events{/privacy}","received_events_url":"https://api.github.com/users/scipy/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/scipy/scipy-articles","description":"Publications about Scipy","fork":false,"url":"https://api.github.com/repos/scipy/scipy-articles","forks_url":"https://api.github.com/repos/scipy/scipy-articles/forks","keys_url":"https://api.github.com/repos/scipy/scipy-articles/keys{/key_id}","collaborators_url":"https://api.github.com/repos/scipy/scipy-articles/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/scipy/scipy-articles/teams","hooks_url":"https://api.github.com/repos/scipy/scipy-articles/hooks","issue_events_url":"https://api.github.com/repos/scipy/scipy-articles/issues/events{/number}","events_url":"https://api.github.com/repos/scipy/scipy-articles/events","assignees_url":"https://api.github.com/repos/scipy/scipy-articles/assignees{/user}","branches_url":"https://api.github.com/repos/scipy/scipy-articles/branches{/branch}","tags_url":"https://api.github.com/repos/scipy/scipy-articles/tags","blobs_url":"https://api.github.com/repos/scipy/scipy-articles/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/scipy/scipy-articles/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/scipy/scipy-articles/git/refs{/sha}","trees_url":"https://api.github.com/repos/scipy/scipy-articles/git/trees{/sha}","statuses_url":"https://api.github.com/repos/scipy/scipy-articles/statuses/{sha}","languages_url":"https://api.github.com/repos/scipy/scipy-articles/languages","stargazers_url":"https://api.github.com/repos/scipy/scipy-articles/stargazers","contributors_url":"https://api.github.com/repos/scipy/scipy-articles/contributors","subscribers_url":"https://api.github.com/repos/scipy/scipy-articles/subscribers","subscription_url":"https://api.github.com/repos/scipy/scipy-articles/subscription","commits_url":"https://api.github.com/repos/scipy/scipy-articles/commits{/sha}","git_commits_url":"https://api.github.com/repos/scipy/scipy-articles/git/commits{/sha}","comments_url":"https://api.github.com/repos/scipy/scipy-articles/comments{/number}","issue_comment_url":"https://api.github.com/repos/scipy/scipy-articles/issues/comments{/number}","contents_url":"https://api.github.com/repos/scipy/scipy-articles/contents/{+path}","compare_url":"https://api.github.com/repos/scipy/scipy-articles/compare/{base}...{head}","merges_url":"https://api.github.com/repos/scipy/scipy-articles/merges","archive_url":"https://api.github.com/repos/scipy/scipy-articles/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/scipy/scipy-articles/downloads","issues_url":"https://api.github.com/repos/scipy/scipy-articles/issues{/number}","pulls_url":"https://api.github.com/repos/scipy/scipy-articles/pulls{/number}","milestones_url":"https://api.github.com/repos/scipy/scipy-articles/milestones{/number}","notifications_url":"https://api.github.com/repos/scipy/scipy-articles/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/scipy/scipy-articles/labels{/name}","releases_url":"https://api.github.com/repos/scipy/scipy-articles/releases{/id}","deployments_url":"https://api.github.com/repos/scipy/scipy-articles/deployments","created_at":"2016-08-20T10:12:32Z","updated_at":"2018-03-21T00:18:34Z","pushed_at":"2018-03-27T02:19:50Z","git_url":"git://github.com/scipy/scipy-articles.git","ssh_url":"git@github.com:scipy/scipy-articles.git","clone_url":"https://github.com/scipy/scipy-articles.git","svn_url":"https://github.com/scipy/scipy-articles","homepage":null,"size":71,"stargazers_count":3,"watchers_count":3,"language":"TeX","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":9,"mirror_url":null,"archived":false,"open_issues_count":8,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":9,"open_issues":8,"watchers":3,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/scipy/scipy-articles/pulls/17"},"html":{"href":"https://github.com/scipy/scipy-articles/pull/17"},"issue":{"href":"https://api.github.com/repos/scipy/scipy-articles/issues/17"},"comments":{"href":"https://api.github.com/repos/scipy/scipy-articles/issues/17/comments"},"review_comments":{"href":"https://api.github.com/repos/scipy/scipy-articles/pulls/17/comments"},"review_comment":{"href":"https://api.github.com/repos/scipy/scipy-articles/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/scipy/scipy-articles/pulls/17/commits"},"statuses":{"href":"https://api.github.com/repos/scipy/scipy-articles/statuses/f37a3967f9f704dbc228fead393b2a6704ec2bb0"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"mdhaber"}}