{"action":"created","issue":{"url":"https://api.github.com/repos/ContinuumIO/anaconda-issues/issues/8803","repository_url":"https://api.github.com/repos/ContinuumIO/anaconda-issues","labels_url":"https://api.github.com/repos/ContinuumIO/anaconda-issues/issues/8803/labels{/name}","comments_url":"https://api.github.com/repos/ContinuumIO/anaconda-issues/issues/8803/comments","events_url":"https://api.github.com/repos/ContinuumIO/anaconda-issues/issues/8803/events","html_url":"https://github.com/ContinuumIO/anaconda-issues/issues/8803","id":303463351,"number":8803,"title":"MKL+Cython+OpenMP linear algebra bugs on OSX","user":{"login":"jackkamm","id":9089216,"avatar_url":"https://avatars1.githubusercontent.com/u/9089216?v=4","gravatar_id":"","url":"https://api.github.com/users/jackkamm","html_url":"https://github.com/jackkamm","followers_url":"https://api.github.com/users/jackkamm/followers","following_url":"https://api.github.com/users/jackkamm/following{/other_user}","gists_url":"https://api.github.com/users/jackkamm/gists{/gist_id}","starred_url":"https://api.github.com/users/jackkamm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jackkamm/subscriptions","organizations_url":"https://api.github.com/users/jackkamm/orgs","repos_url":"https://api.github.com/users/jackkamm/repos","events_url":"https://api.github.com/users/jackkamm/events{/privacy}","received_events_url":"https://api.github.com/users/jackkamm/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-03-08T12:30:20Z","updated_at":"2018-03-08T14:31:31Z","closed_at":null,"author_association":"NONE","body":"### Actual Behavior\r\n\r\nOn OSX with MKL-enabled scipy, importing a Cython module that uses OpenMP will cause strange linear algebra behavior. For example, the SVD of a 50x50 identity matrix will have negative values. More specifically, if we do\r\n```\r\neye = np.eye(51, 51)\r\nu, s, vh = scipy.linalg.svd(eye)\r\n```\r\nThen `u`,`vh`, and `u @ np.diag(s) @ vh` will all contain negative values, as shown here:\r\n![image](https://user-images.githubusercontent.com/9089216/37150697-91640920-22ca-11e8-863f-9135ea2e99a7.png)\r\n\r\nHowever setting `MKL_NUM_THREADS=1` will yield the desired behavior:\r\n![image](https://user-images.githubusercontent.com/9089216/37150735-ba787436-22ca-11e8-89ee-1c6e33b004c3.png)\r\n\r\nNote the Cython module only needs to be imported to cause the error -- it does not need to be used in any way.\r\n\r\n### Expected Behavior\r\n\r\nThe singular value decomposition should be correct.\r\n\r\n### Steps to Reproduce\r\n\r\nPlease see https://github.com/jackkamm/osx-mkl-omp-cython-bug for the full details. In brief:\r\n\r\n1. Create a clean conda environment. Install python3, cython, gcc, and MKL-enabled numpy/scipy.\r\n2. Create a small Cython module named `ext.pyx` that uses OpenMP:\r\n```\r\ncimport numpy as np\r\nimport numpy as np\r\nimport cython\r\nfrom cython.parallel import prange\r\n\r\n@cython.boundscheck(False)\r\n@cython.wraparound(False)\r\n@cython.cdivision(True)\r\ndef p_arange(int n):\r\n    '''\r\n    A useless function only for demonstration purposes\r\n    '''\r\n    cdef np.ndarray[np.int_t, ndim=1] ret = np.zeros(n)\r\n    for i in range(n, schedule='guided', nogil=True):\r\n        ret[i] = i\r\n    return ret\r\n```\r\n3. Build the Cython module. gcc is required for the `-fopenmp` flag.\r\n4. In Python, import the Cython module, then use scipy to compute the SVD of the identity matrix:\r\n```\r\nimport numpy as np\r\nimport scipy.linalg\r\n\r\n# import the extension but don't do anything with it\r\nimport ext\r\n\r\nn = 50\r\neye = np.eye(n+1, n+1)\r\nu, s, vh = scipy.linalg.svd(eye)\r\neye2 = u @ np.diag(s) @ vh\r\n\r\nprint(\"All close: \", np.allclose(eye, eye2)) \r\nprint(\"Min(eye2): \", np.min(eye2))\r\n```\r\n\r\n\r\n##### Anaconda or Miniconda version:\r\nAnaconda 3, conda 4.4.11\r\n\r\n##### Operating System:\r\n\r\nmacOS 10.12.6\r\n\r\n##### `conda info`\r\n<!-- between the ticks below, paste the output of 'conda info' -->\r\n```\r\n     active environment : osx-mkl-omp-cython-bug\r\n    active env location : /Users/jack/anaconda3/envs/osx-mkl-omp-cython-bug\r\n            shell level : 1\r\n       user config file : /Users/jack/.condarc\r\n populated config files :\r\n          conda version : 4.4.11\r\n    conda-build version : 3.7.0\r\n         python version : 3.6.4.final.0\r\n       base environment : /Users/jack/anaconda3  (writable)\r\n           channel URLs : https://repo.continuum.io/pkgs/main/osx-64\r\n                          https://repo.continuum.io/pkgs/main/noarch\r\n                          https://repo.continuum.io/pkgs/free/osx-64\r\n                          https://repo.continuum.io/pkgs/free/noarch\r\n                          https://repo.continuum.io/pkgs/r/osx-64\r\n                          https://repo.continuum.io/pkgs/r/noarch\r\n                          https://repo.continuum.io/pkgs/pro/osx-64\r\n                          https://repo.continuum.io/pkgs/pro/noarch\r\n          package cache : /Users/jack/anaconda3/pkgs\r\n                          /Users/jack/.conda/pkgs\r\n       envs directories : /Users/jack/anaconda3/envs\r\n                          /Users/jack/.conda/envs\r\n               platform : osx-64\r\n             user-agent : conda/4.4.11 requests/2.18.4 CPython/3.6.4 Darwin/16.7.0 OSX/10.12.6\r\n                UID:GID : 501:20\r\n             netrc file : None\r\n           offline mode : False\r\n```\r\n\r\n##### `conda list --show-channel-urls`\r\n<!-- between the ticks below, paste the output of 'conda list --show-channel-urls' -->\r\n```\r\n# packages in environment at /Users/jack/anaconda3/envs/osx-mkl-omp-cython-bug:\r\n#\r\n# Name                    Version                   Build  Channel\r\nca-certificates           2017.08.26           ha1e5d58_0    defaults\r\ncertifi                   2018.1.18                py36_0    defaults\r\ncloog                     0.18.0                        0    defaults\r\ncycler                    0.10.0           py36hfc81398_0    defaults\r\ncython                    0.27.3           py36h6ecb376_0    defaults\r\nfreetype                  2.8                  h12048fb_1    defaults\r\ngcc                       4.8.5                         8    defaults\r\ngmp                       6.1.2                hb37e062_1    defaults\r\nintel-openmp              2018.0.0             h8158457_8    defaults\r\nisl                       0.12.2                        1    defaults\r\nkiwisolver                1.0.1            py36h792292d_0    defaults\r\nlibcxx                    4.0.1                h579ed51_0    defaults\r\nlibcxxabi                 4.0.1                hebd6815_0    defaults\r\nlibedit                   3.1                  hb4e282d_0    defaults\r\nlibffi                    3.2.1                h475c297_4    defaults\r\nlibgfortran               3.0.1                h93005f0_2    defaults\r\nlibpng                    1.6.34               he12f830_0    defaults\r\nmatplotlib                2.2.0            py36hfa7797c_0    defaults\r\nmkl                       2018.0.1             hfbd8650_4    defaults\r\nmpc                       1.0.3                h7a72875_5    defaults\r\nmpfr                      3.1.5                h711e7fd_2    defaults\r\nncurses                   6.0                  hd04f020_2    defaults\r\nnumpy                     1.14.1           py36ha726252_2    defaults\r\nopenssl                   1.0.2n               hdbc3d79_0    defaults\r\npandas                    0.22.0           py36h0a44026_0    defaults\r\npatsy                     0.5.0                    py36_0    defaults\r\npip                       9.0.1                    py36_5    defaults\r\npyparsing                 2.2.0            py36hb281f35_0    defaults\r\npython                    3.6.4                hc167b69_1    defaults\r\npython-dateutil           2.6.1            py36h86d2abb_1    defaults\r\npytz                      2018.3                   py36_0    defaults\r\nreadline                  7.0                  hc1231fa_4    defaults\r\nscipy                     1.0.0            py36h1de22e9_0    defaults\r\nseaborn                   0.8.1            py36h595ecd9_0    defaults\r\nsetuptools                38.5.1                   py36_0    defaults\r\nsix                       1.11.0           py36h0e22d5e_1    defaults\r\nsqlite                    3.22.0               h3efe00b_0    defaults\r\nstatsmodels               0.8.0            py36h9c68fc9_0    defaults\r\ntk                        8.6.7                h35a86e2_3    defaults\r\ntornado                   5.0                      py36_0    defaults\r\nwheel                     0.30.0           py36h5eb2c71_1    defaults\r\nxz                        5.2.3                h0278029_2    defaults\r\nzlib                      1.2.11               hf3cbc9b_2    defaults\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/ContinuumIO/anaconda-issues/issues/comments/371503266","html_url":"https://github.com/ContinuumIO/anaconda-issues/issues/8803#issuecomment-371503266","issue_url":"https://api.github.com/repos/ContinuumIO/anaconda-issues/issues/8803","id":371503266,"user":{"login":"msarahan","id":38393,"avatar_url":"https://avatars3.githubusercontent.com/u/38393?v=4","gravatar_id":"","url":"https://api.github.com/users/msarahan","html_url":"https://github.com/msarahan","followers_url":"https://api.github.com/users/msarahan/followers","following_url":"https://api.github.com/users/msarahan/following{/other_user}","gists_url":"https://api.github.com/users/msarahan/gists{/gist_id}","starred_url":"https://api.github.com/users/msarahan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/msarahan/subscriptions","organizations_url":"https://api.github.com/users/msarahan/orgs","repos_url":"https://api.github.com/users/msarahan/repos","events_url":"https://api.github.com/users/msarahan/events{/privacy}","received_events_url":"https://api.github.com/users/msarahan/received_events","type":"User","site_admin":false},"created_at":"2018-03-08T14:31:31Z","updated_at":"2018-03-08T14:31:31Z","author_association":"MEMBER","body":"In private discussions, @oleksandr-pavlyk suggested that you are probably hitting https://github.com/numpy/numpy/issues/9758, not the issue with OpenMP that I pointed out.  The sinister thing here is that if ANY GUI stuff is loaded before numpy, Mac's Accelerate library gets found preferentially for MKL, which has the strange result that you point to.  Sasha says:\r\n\r\n> It would be best if he tried to confirm that by loading NumPy before loading Qt5.  He would be need to do it from a terminal, using Python (not IPython and not jupyter)"}}
{"actor":{"display_login":"msarahan"}}