{"action":"created","issue":{"url":"https://api.github.com/repos/pycalphad/pycalphad/issues/176","repository_url":"https://api.github.com/repos/pycalphad/pycalphad","labels_url":"https://api.github.com/repos/pycalphad/pycalphad/issues/176/labels{/name}","comments_url":"https://api.github.com/repos/pycalphad/pycalphad/issues/176/comments","events_url":"https://api.github.com/repos/pycalphad/pycalphad/issues/176/events","html_url":"https://github.com/pycalphad/pycalphad/pull/176","id":329568246,"node_id":"MDExOlB1bGxSZXF1ZXN0MTkyNzkyODI5","number":176,"title":"ENH: Parallelize calculation of objective function","user":{"login":"bocklund","id":7681751,"node_id":"MDQ6VXNlcjc2ODE3NTE=","avatar_url":"https://avatars0.githubusercontent.com/u/7681751?v=4","gravatar_id":"","url":"https://api.github.com/users/bocklund","html_url":"https://github.com/bocklund","followers_url":"https://api.github.com/users/bocklund/followers","following_url":"https://api.github.com/users/bocklund/following{/other_user}","gists_url":"https://api.github.com/users/bocklund/gists{/gist_id}","starred_url":"https://api.github.com/users/bocklund/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bocklund/subscriptions","organizations_url":"https://api.github.com/users/bocklund/orgs","repos_url":"https://api.github.com/users/bocklund/repos","events_url":"https://api.github.com/users/bocklund/events{/privacy}","received_events_url":"https://api.github.com/users/bocklund/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-06-05T18:12:10Z","updated_at":"2018-06-05T22:06:54Z","closed_at":null,"author_association":"COLLABORATOR","pull_request":{"url":"https://api.github.com/repos/pycalphad/pycalphad/pulls/176","html_url":"https://github.com/pycalphad/pycalphad/pull/176","diff_url":"https://github.com/pycalphad/pycalphad/pull/176.diff","patch_url":"https://github.com/pycalphad/pycalphad/pull/176.patch"},"body":"These changes define a parallel wrapper around PhaseRecord.obj that uses OpenMP to parallelize calculation of the objective function and uses that by default in `_compute_phase_values`.\r\n\r\nI tested two main cases and some scaling between them\r\n\r\n1. Large case (same as #175, ~30M points): gained a 2x speedup of the same benchmark from #175 (on an Intel i7)\r\n2. Small case: running calculate with a 1 sublattice phase and `pdens=1`, giving 4 total points. The parallel and serial versions of `calculate` are within 2x for small cases . As the number of points grows and sampling becomes more expensive, they eventually switch and parallel seems to be about 2x faster. Note that on the curve there is some constant time that grows due to copying arrays, so the speedup of the objective is actually more exaggerated than shown here.\r\n\r\n![scaling](https://user-images.githubusercontent.com/7681751/40994368-03e0b896-68ca-11e8-9c60-0752615d40d6.png)\r\n\r\n\r\nI chose to have parallelism on by default, but I also put in an option to turn off parallelization. My reasoning for default is that the time difference is only a few ms. The use case with the biggest potential pitfall is if users are running `calculate` for small point densities in tight loops (instead of using broadcasting features). This does not seem too likely to me. There's also an option to have some heuristic so that having parallelism turned on still uses serial mode unless there are more than 100k point sets (for example).\r\n\r\nI added an option for parallelization to be turned off because uses may not want it if they are performing equilibrium calculations in parallel, such as in ESPEI or using a dask scheduler, or for other special cases like the above.\r\n\r\nThis also knocks off a chunk of time for large calculate calls and can go a long way depending on the machine without having to fully Cythonize `_compute_phaes_values` yet. That still might be worth doing in the future.\r\n\r\n@richardotis what do you think? I have some reservations about parallelism as a default. "},"comment":{"url":"https://api.github.com/repos/pycalphad/pycalphad/issues/comments/394875918","html_url":"https://github.com/pycalphad/pycalphad/pull/176#issuecomment-394875918","issue_url":"https://api.github.com/repos/pycalphad/pycalphad/issues/176","id":394875918,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NDg3NTkxOA==","user":{"login":"bocklund","id":7681751,"node_id":"MDQ6VXNlcjc2ODE3NTE=","avatar_url":"https://avatars0.githubusercontent.com/u/7681751?v=4","gravatar_id":"","url":"https://api.github.com/users/bocklund","html_url":"https://github.com/bocklund","followers_url":"https://api.github.com/users/bocklund/followers","following_url":"https://api.github.com/users/bocklund/following{/other_user}","gists_url":"https://api.github.com/users/bocklund/gists{/gist_id}","starred_url":"https://api.github.com/users/bocklund/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bocklund/subscriptions","organizations_url":"https://api.github.com/users/bocklund/orgs","repos_url":"https://api.github.com/users/bocklund/repos","events_url":"https://api.github.com/users/bocklund/events{/privacy}","received_events_url":"https://api.github.com/users/bocklund/received_events","type":"User","site_admin":false},"created_at":"2018-06-05T22:06:53Z","updated_at":"2018-06-05T22:06:53Z","author_association":"COLLABORATOR","body":"FWIW: line profiling indicates that the execution of the objective calls for the large test case went from 35.9s to 12.2s, so about 3x speedup."}}
{"actor":{"display_login":"bocklund"}}