{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/12259","repository_url":"https://api.github.com/repos/scikit-learn/scikit-learn","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/12259/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/12259/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/12259/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/12259","id":366366665,"node_id":"MDU6SXNzdWUzNjYzNjY2NjU=","number":12259,"title":"DecisionTreeClassifier behaviour when there are 2 or more best splitter (a tie among splitters) ","user":{"login":"GMarzinotto","id":5233985,"node_id":"MDQ6VXNlcjUyMzM5ODU=","avatar_url":"https://avatars0.githubusercontent.com/u/5233985?v=4","gravatar_id":"","url":"https://api.github.com/users/GMarzinotto","html_url":"https://github.com/GMarzinotto","followers_url":"https://api.github.com/users/GMarzinotto/followers","following_url":"https://api.github.com/users/GMarzinotto/following{/other_user}","gists_url":"https://api.github.com/users/GMarzinotto/gists{/gist_id}","starred_url":"https://api.github.com/users/GMarzinotto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GMarzinotto/subscriptions","organizations_url":"https://api.github.com/users/GMarzinotto/orgs","repos_url":"https://api.github.com/users/GMarzinotto/repos","events_url":"https://api.github.com/users/GMarzinotto/events{/privacy}","received_events_url":"https://api.github.com/users/GMarzinotto/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-10-03T14:25:36Z","updated_at":"2018-10-04T07:45:09Z","closed_at":null,"author_association":"NONE","body":"#### Description\r\n\r\nMy issue has been discussed in the past issues #2386  and #8443\r\n\r\nIt concerns `DecisionTreeClassifier` and the `splitter` parameter\r\nIn the [docs](http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier.decision_path) this parameter has two possible values  `splitter=\"best\"` and `splitter=\"random\"`\r\n\r\n\r\nThe current behaviour when  `splitter=\"best\"` is to shuffle the features at each step and take the best feature to split. In case there is a tie, we take a random one.\r\n\r\nIn some cases, there is a prior on the feature importance or in the ease of the interpretation. \r\n\r\nFor example, some features can be noisy (due to possible data errors or any other cause) and shouldn't be used for splitting unless there is no 'good quality' feature available that does the job. Similarly, some features can be easy to understand and some others can be obscure and shouldn't be used for splitting when possible.\r\n\r\n\r\nThe current implementation doesn't allow having this kind of prior. So when two features tie as the best splitter one get chosen at random. I believe this could be an important improvement. Specially when the trees do not have a `max_depth`, because they tend to over-fit on random features while there may be some \"better\" features (in terms of prior) that do the work.\r\n\r\nHaving an option such as `splitter=\"best_no_shuffle\"` would allow the user to provide the features in order of importance and then when two or more features tie as the best splitters, it would systemically chose the features with the lowest index. This was once proposed [here](https://github.com/scikit-learn/scikit-learn/issues/2386#issuecomment-23148454) \r\n\r\n\r\nIf you believe that this can be a valuable improvement I can try to implement it.\r\nOr maybe you know a better workaround that solves my problem ! \r\n\r\n\r\n\r\n\r\n\r\n\r\n#### Steps/Code to Reproduce\r\n\r\nI have recovered the code from  #8443 because it is simple and explains the issue\r\n\r\n\r\n```python\r\nimport numpy as np\r\nfrom sklearn.datasets import load_iris\r\nfrom sklearn.tree import DecisionTreeClassifier\r\n\r\niris = load_iris()\r\nx,y  = iris.data, iris.target\r\n\r\ndtc1 = DecisionTreeClassifier(random_state=1)\r\ndtc2 = DecisionTreeClassifier(random_state=2)\r\n\r\nrs = np.random.RandomState(1234)\r\nitr = rs.rand(x.shape[0]) < 0.75\r\n\r\ndtc1.fit(x[itr],y[itr])\r\ndtc2.fit(x[itr],y[itr])\r\n\r\nprint(  (dtc1.predict(x[~itr]) != dtc2.predict(x[~itr])).sum() )\r\n```\r\n\r\n\r\n#### Expected Results\r\n`Should print 0`\r\n\r\n#### Actual Results\r\n`Prints 1`\r\n\r\n#### Versions\r\n\r\nSystem\r\n------\r\n   machine: Linux-4.4.0-135-generic-x86_64-with-Ubuntu-16.04-xenial\r\nexecutable: /usr/bin/python3\r\n    python: 3.5.2 (default, Nov 23 2017, 16:37:01)  [GCC 5.4.0 20160609]\r\n\r\nBLAS\r\n----\r\ncblas_libs: openblas, openblas\r\n    macros: HAVE_CBLAS=None\r\n  lib_dirs: /usr/lib\r\n\r\nPython deps\r\n-----------\r\n     scipy: 0.19.0\r\n   sklearn: 0.21.dev0\r\n    pandas: 0.20.3\r\n     numpy: 1.13.3\r\n    Cython: 0.26\r\nsetuptools: 36.7.2\r\n       pip: 9.0.1\r\n\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/426918071","html_url":"https://github.com/scikit-learn/scikit-learn/issues/12259#issuecomment-426918071","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/12259","id":426918071,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNjkxODA3MQ==","user":{"login":"jnothman","id":78827,"node_id":"MDQ6VXNlcjc4ODI3","avatar_url":"https://avatars2.githubusercontent.com/u/78827?v=4","gravatar_id":"","url":"https://api.github.com/users/jnothman","html_url":"https://github.com/jnothman","followers_url":"https://api.github.com/users/jnothman/followers","following_url":"https://api.github.com/users/jnothman/following{/other_user}","gists_url":"https://api.github.com/users/jnothman/gists{/gist_id}","starred_url":"https://api.github.com/users/jnothman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jnothman/subscriptions","organizations_url":"https://api.github.com/users/jnothman/orgs","repos_url":"https://api.github.com/users/jnothman/repos","events_url":"https://api.github.com/users/jnothman/events{/privacy}","received_events_url":"https://api.github.com/users/jnothman/received_events","type":"User","site_admin":false},"created_at":"2018-10-04T07:45:09Z","updated_at":"2018-10-04T07:45:09Z","author_association":"MEMBER","body":"Intuitively this makes sense. Are there other ways to do this sort of\nthing, e.g. with feature weights? Is there literature on it?\n"}}
{"actor":{"display_login":"jnothman"}}