{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10798","repository_url":"https://api.github.com/repos/scikit-learn/scikit-learn","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10798/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10798/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10798/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/10798","id":304380137,"number":10798,"title":"Decision Tree split looks wrong","user":{"login":"juan-barragan","id":9152124,"avatar_url":"https://avatars1.githubusercontent.com/u/9152124?v=4","gravatar_id":"","url":"https://api.github.com/users/juan-barragan","html_url":"https://github.com/juan-barragan","followers_url":"https://api.github.com/users/juan-barragan/followers","following_url":"https://api.github.com/users/juan-barragan/following{/other_user}","gists_url":"https://api.github.com/users/juan-barragan/gists{/gist_id}","starred_url":"https://api.github.com/users/juan-barragan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/juan-barragan/subscriptions","organizations_url":"https://api.github.com/users/juan-barragan/orgs","repos_url":"https://api.github.com/users/juan-barragan/repos","events_url":"https://api.github.com/users/juan-barragan/events{/privacy}","received_events_url":"https://api.github.com/users/juan-barragan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2018-03-12T13:53:19Z","updated_at":"2018-03-13T12:22:22Z","closed_at":null,"author_association":"NONE","body":"<!--\r\nIf your issue is a usage question, submit it here instead:\r\n- StackOverflow with the scikit-learn tag: http://stackoverflow.com/questions/tagged/scikit-learn\r\n- Mailing List: https://mail.python.org/mailman/listinfo/scikit-learn\r\nFor more information, see User Questions: http://scikit-learn.org/stable/support.html#user-questions\r\n-->\r\n\r\n<!-- Instructions For Filing a Bug: https://github.com/scikit-learn/scikit-learn/blob/master/CONTRIBUTING.md#filing-bugs -->\r\n\r\n#### Description\r\nWhen looking for an optimal split on a decision tree, DecisionTreeClassifier gives a seemingly wrong answer\r\n\r\n#### Steps/Code to Reproduce\r\n#### raw dataset used : https://github.com/juan-barragan/titanic/blob/master/titanic_ds.csv\r\n````python\r\nimport pandas as pd\r\nimport seaborn\r\nimport matplotlib.pyplot as plt\r\nfrom sklearn import tree\r\nimport graphviz\r\nimport numpy as np\r\n\r\ndef gini_by_age(df, t):\r\n    df['age_group'] = df['age'].apply(lambda row : 0 if row <= t else 1)\r\n    kids = df[df['age_group'] == 0]\r\n    kids0 = kids[kids['survived'] == 0]\r\n    kids1 = kids[kids['survived'] == 1]\r\n    adults = df[df['age_group'] == 1]\r\n    adults0 = adults[adults['survived'] == 0]\r\n    adults1 = adults[adults['survived'] == 1] \r\n    gk = 1 - (len(kids0)**2 + len(kids1)**2)/float(len(kids))**2\r\n    ga = 1 - (len(adults0)**2 + len(adults1)**2)/float(len(adults))**2\r\n    return gk + ga\r\n\r\ndef plot_gini_by_age(df):    \r\n    ages = range(6,25)\r\n    y = [gini_by_age(df, a) for a in ages]\r\n    plt.plot(ages, y)\r\n    plt.show()\r\n\r\ndef use_tree(df):\r\n    X = np.array(df['age']).reshape((len(df['age']),1))\r\n    y = df['survived']\r\n    clf = tree.DecisionTreeClassifier(max_depth=1).fit(X,y)    \r\n    dot_data = tree.export_graphviz(clf, out_file=None)\r\n    graph = graphviz.Source(dot_data)\r\n    graph.render(\"age\")\r\n\r\ntitanic_df = pd.read_csv(\"titanic_ds.csv\")\r\nages_cov = titanic_df[['age', 'survived']].dropna()\r\nplot_gini_by_age(ages_cov)\r\nuse_tree(ages_cov)\r\nprint gini_by_age(ages_cov, 5)\r\nprint gini_by_age(ages_cov, 6)\r\nprint gini_by_age(ages_cov, 8.5)\r\nprint gini_by_age(ages_cov, 15)\r\n````\r\n#### Expected Results\r\ngini split at around 5\r\n\r\n#### Actual Results\r\nGot 8.5 instead\r\n0.925844132419\r\n0.93508349655\r\n0.937732003001\r\n0.963875889772\r\n#### Versions\r\nLinux-4.4.0-21-generic-x86_64-with-LinuxMint-18-sarah\r\n('Python', '2.7.12 (default, Nov 20 2017, 18:23:56) \\n[GCC 5.4.0 20160609]')\r\n('NumPy', '1.11.2')\r\n('SciPy', '0.18.1')\r\n('Scikit-Learn', '0.18')\r\n\r\n\r\n<!-- Thanks for contributing! -->\r\n"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/372646937","html_url":"https://github.com/scikit-learn/scikit-learn/issues/10798#issuecomment-372646937","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10798","id":372646937,"user":{"login":"glouppe","id":477771,"avatar_url":"https://avatars3.githubusercontent.com/u/477771?v=4","gravatar_id":"","url":"https://api.github.com/users/glouppe","html_url":"https://github.com/glouppe","followers_url":"https://api.github.com/users/glouppe/followers","following_url":"https://api.github.com/users/glouppe/following{/other_user}","gists_url":"https://api.github.com/users/glouppe/gists{/gist_id}","starred_url":"https://api.github.com/users/glouppe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/glouppe/subscriptions","organizations_url":"https://api.github.com/users/glouppe/orgs","repos_url":"https://api.github.com/users/glouppe/repos","events_url":"https://api.github.com/users/glouppe/events{/privacy}","received_events_url":"https://api.github.com/users/glouppe/received_events","type":"User","site_admin":false},"created_at":"2018-03-13T12:22:22Z","updated_at":"2018-03-13T12:22:22Z","author_association":"OWNER","body":"The implementation follows the CART description, using weighting. \r\n\r\nWe want to split a split that maximize the reduction of impurity I(n) - p_L * I(n_L) - p_R * I(n_R), where\r\n- I(n) is the impurity of node n \r\n- p_L is the proportion of examples going left \r\n- I(n_L) is the impurity of the resulting left node \r\n- p_R is the proportion of examples going right \r\n- I(n_R) is the impurity of the resulting right node \r\n\r\nNot reweighting might lead to degenerate splits, where one node is very pure and small but the other remains large and with more or less the same level of impurity. "}}
{"actor":{"display_login":"glouppe"}}