{"action":"closed","issue":{"url":"https://api.github.com/repos/rhiever/tpot/issues/201","repository_url":"https://api.github.com/repos/rhiever/tpot","labels_url":"https://api.github.com/repos/rhiever/tpot/issues/201/labels{/name}","comments_url":"https://api.github.com/repos/rhiever/tpot/issues/201/comments","events_url":"https://api.github.com/repos/rhiever/tpot/issues/201/events","html_url":"https://github.com/rhiever/tpot/issues/201","id":168098926,"number":201,"title":"Titanic example broken, np.isnan(titanic_new) generates error and training results in CV of 0.5","user":{"login":"MichaelMarkieta","id":3111752,"avatar_url":"https://avatars.githubusercontent.com/u/3111752?v=3","gravatar_id":"","url":"https://api.github.com/users/MichaelMarkieta","html_url":"https://github.com/MichaelMarkieta","followers_url":"https://api.github.com/users/MichaelMarkieta/followers","following_url":"https://api.github.com/users/MichaelMarkieta/following{/other_user}","gists_url":"https://api.github.com/users/MichaelMarkieta/gists{/gist_id}","starred_url":"https://api.github.com/users/MichaelMarkieta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MichaelMarkieta/subscriptions","organizations_url":"https://api.github.com/users/MichaelMarkieta/orgs","repos_url":"https://api.github.com/users/MichaelMarkieta/repos","events_url":"https://api.github.com/users/MichaelMarkieta/events{/privacy}","received_events_url":"https://api.github.com/users/MichaelMarkieta/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/rhiever/tpot/labels/question","name":"question","color":"cc317c"}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-07-28T13:44:10Z","updated_at":"2016-08-13T18:28:54Z","closed_at":"2016-08-13T18:28:54Z","body":"No luck with the Titanic tutorial on the latest installation versions of TPOT and its dependencies.\r\n\r\n```python\r\nPython 3.5.1 (default, Dec 26 2015, 18:08:53) \r\n[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin\r\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\r\n>>> from tpot import TPOT\r\n>>> from sklearn.cross_validation import train_test_split\r\n>>> import pandas as pd \r\n>>> import numpy as np\r\n>>> titanic = pd.read_csv('train.csv')\r\n>>> titanic.head(5)\r\n   PassengerId  Survived  Pclass  \\\r\n0            1         0       3   \r\n1            2         1       1   \r\n2            3         1       3   \r\n3            4         1       1   \r\n4            5         0       3   \r\n\r\n                                                Name     Sex   Age  SibSp  \\\r\n0                            Braund, Mr. Owen Harris    male  22.0      1   \r\n1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \r\n2                             Heikkinen, Miss. Laina  female  26.0      0   \r\n3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \r\n4                           Allen, Mr. William Henry    male  35.0      0   \r\n\r\n   Parch            Ticket     Fare Cabin Embarked  \r\n0      0         A/5 21171   7.2500   NaN        S  \r\n1      0          PC 17599  71.2833   C85        C  \r\n2      0  STON/O2. 3101282   7.9250   NaN        S  \r\n3      0            113803  53.1000  C123        S  \r\n4      0            373450   8.0500   NaN        S  \r\n>>> titanic.groupby('Sex').Survived.value_counts()\r\nSex     Survived\r\nfemale  1           233\r\n        0            81\r\nmale    0           468\r\n        1           109\r\nName: Survived, dtype: int64\r\n>>> titanic.groupby(['Pclass','Sex']).Survived.value_counts()\r\nPclass  Sex     Survived\r\n1       female  1            91\r\n                0             3\r\n        male    0            77\r\n                1            45\r\n2       female  1            70\r\n                0             6\r\n        male    0            91\r\n                1            17\r\n3       female  0            72\r\n                1            72\r\n        male    0           300\r\n                1            47\r\nName: Survived, dtype: int64\r\n>>> id = pd.crosstab([titanic.Pclass, titanic.Sex], titanic.Survived.astype(float))\r\n>>> id.div(id.sum(1).astype(float), 0)\r\nSurvived            0.0       1.0\r\nPclass Sex                       \r\n1      female  0.031915  0.968085\r\n       male    0.631148  0.368852\r\n2      female  0.078947  0.921053\r\n       male    0.842593  0.157407\r\n3      female  0.500000  0.500000\r\n       male    0.864553  0.135447\r\n>>> titanic.rename(columns={'Survived': 'class'}, inplace=True)\r\n>>> titanic.dtypes\r\nPassengerId      int64\r\nclass            int64\r\nPclass           int64\r\nName            object\r\nSex             object\r\nAge            float64\r\nSibSp            int64\r\nParch            int64\r\nTicket          object\r\nFare           float64\r\nCabin           object\r\nEmbarked        object\r\ndtype: object\r\n>>> for cat in ['Name', 'Sex', 'Ticket', 'Cabin', 'Embarked']:\r\n...     print(\"Number of levels in category '{0}': \\b {1:2.2f} \".format(cat, titanic[cat].unique().size))\r\n... \r\nNumber of levels in category 'Name': 891.00 \r\nNumber of levels in category 'Sex': 2.00 \r\nNumber of levels in category 'Ticket': 681.00 \r\nNumber of levels in category 'Cabin': 148.00 \r\nNumber of levels in category 'Embarked': 4.00 \r\n>>> for cat in ['Sex', 'Embarked']:\r\n...     print(\"Levels for catgeory '{0}': {1}\".format(cat, titanic[cat].unique()))\r\n... \r\nLevels for catgeory 'Sex': ['male' 'female']\r\nLevels for catgeory 'Embarked': ['S' 'C' 'Q' nan]\r\n>>> titanic = titanic.fillna(-999)\r\n>>> pd.isnull(titanic).any()\r\nPassengerId    False\r\nclass          False\r\nPclass         False\r\nName           False\r\nSex            False\r\nAge            False\r\nSibSp          False\r\nParch          False\r\nTicket         False\r\nFare           False\r\nCabin          False\r\nEmbarked       False\r\ndtype: bool\r\n>>> from sklearn.preprocessing import MultiLabelBinarizer\r\n>>> mlb = MultiLabelBinarizer()\r\n>>> CabinTrans = mlb.fit_transform([{str(val)} for val in titanic['Cabin'].values])\r\n>>> CabinTrans\r\narray([[1, 0, 0, ..., 0, 0, 0],\r\n       [0, 0, 0, ..., 0, 0, 0],\r\n       [1, 0, 0, ..., 0, 0, 0],\r\n       ..., \r\n       [1, 0, 0, ..., 0, 0, 0],\r\n       [0, 0, 0, ..., 0, 0, 0],\r\n       [1, 0, 0, ..., 0, 0, 0]])\r\n>>> titanic_new = titanic.drop(['Name','Ticket','Cabin','class'], axis=1)\r\n>>> assert (len(titanic['Cabin'].unique()) == len(mlb.classes_)), \"Not Equal\"\r\n>>> titanic_new = np.hstack((titanic_new.values,CabinTrans))\r\n>>> np.isnan(titanic_new).any()\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\nTypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''\r\n>>> titanic_new[0].size\r\n156\r\n>>> titanic_class = titanic['class'].values\r\n>>> training_indices, validation_indices = training_indices, testing_indices = train_test_split(titanic.index, stratify = titanic_class, train_size=0.75, test_size=0.25)\r\n>>> training_indices.size, validation_indices.size\r\n(668, 223)\r\n>>> tpot = TPOT(generations=5, verbosity=2)\r\n>>> tpot.fit(titanic_new[training_indices], titanic_class[training_indices])\r\nGeneration 1 - Current best internal CV score: 0.50000                                                                                                                                                                                      \r\nGeneration 2 - Current best internal CV score: 0.50000                                                                                                                                                                                      \r\nGeneration 3 - Current best internal CV score: 0.50000                                                                                                                                                                                      \r\nGeneration 4 - Current best internal CV score: 0.50000                                                                                                                                                                                      \r\nGeneration 5 - Current best internal CV score: 0.50000\r\n                                                                                                                                                                                                                                            \r\nBest pipeline: _variance_threshold(input_df, 0.10000000000000001)\r\n```"}}
{"actor":{"display_login":"rhiever"}}