{"action":"closed","issue":{"url":"https://api.github.com/repos/EpistasisLab/tpot/issues/667","repository_url":"https://api.github.com/repos/EpistasisLab/tpot","labels_url":"https://api.github.com/repos/EpistasisLab/tpot/issues/667/labels{/name}","comments_url":"https://api.github.com/repos/EpistasisLab/tpot/issues/667/comments","events_url":"https://api.github.com/repos/EpistasisLab/tpot/issues/667/events","html_url":"https://github.com/EpistasisLab/tpot/issues/667","id":296811271,"node_id":"MDU6SXNzdWUyOTY4MTEyNzE=","number":667,"title":"Run Arbitrary Code Every Generation","user":{"login":"phillip-martin","id":5428143,"node_id":"MDQ6VXNlcjU0MjgxNDM=","avatar_url":"https://avatars2.githubusercontent.com/u/5428143?v=4","gravatar_id":"","url":"https://api.github.com/users/phillip-martin","html_url":"https://github.com/phillip-martin","followers_url":"https://api.github.com/users/phillip-martin/followers","following_url":"https://api.github.com/users/phillip-martin/following{/other_user}","gists_url":"https://api.github.com/users/phillip-martin/gists{/gist_id}","starred_url":"https://api.github.com/users/phillip-martin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/phillip-martin/subscriptions","organizations_url":"https://api.github.com/users/phillip-martin/orgs","repos_url":"https://api.github.com/users/phillip-martin/repos","events_url":"https://api.github.com/users/phillip-martin/events{/privacy}","received_events_url":"https://api.github.com/users/phillip-martin/received_events","type":"User","site_admin":false},"labels":[{"id":281818167,"node_id":"MDU6TGFiZWwyODE4MTgxNjc=","url":"https://api.github.com/repos/EpistasisLab/tpot/labels/question","name":"question","color":"cc317c","default":true}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-02-13T16:56:05Z","updated_at":"2018-07-02T19:34:53Z","closed_at":"2018-07-02T19:34:53Z","author_association":"NONE","body":"I couldn't find this in the api or [documentation](http://epistasislab.github.io), so please excuse me if this is trivial. At the end of training a model using tpot, I score the generated model based on mean absolute error and mean squared error. The following demonstrates an example.\r\n\r\n```python\r\nx_train, x_test, y_train, y_test = train_test_split(\r\n    x, y, test_size=test_size, random_state=seed\r\n)\r\n\r\ntpot = TPOTRegressor(generations=50, population_size=20, verbosity=2)\r\ntpot.fit(x_train, y_train)\r\n\r\n# score at the end of training\r\ny_predicted = tpot.predict(scaler_x.transform(x_test))\r\nprint('me: ', mean_absolute_error(y_test, y_predicted))\r\nprint('mse: ', mean_squared_error(y_test, y_predicted))\r\n```\r\n\r\nI would like to be able to run these scores every generation, ideally through a function passed to either `TPOTRegressor` or `tpot.fit`. This might look like the following.\r\n\r\n```python\r\nx_train, x_test, y_train, y_test = train_test_split(\r\n    x, y, test_size=test_size, random_state=seed\r\n)\r\n\r\ndef score_tpot(_tpot):\r\n    y_predicted = _tpot.predict(scaler_x.transform(x_test))\r\n    print('me: ', mean_absolute_error(y_test, y_predicted))\r\n    print('mse: ', mean_squared_error(y_test, y_predicted))\r\n\r\ntpot = TPOTRegressor(\r\n    generations=50, population_size=20, verbosity=2, each_generation=score_tpot\r\n)\r\ntpot.fit(x_train, y_train)\r\n```\r\n\r\nIs there currently a way to do something like this that I could not find in the documentation?\r\n\r\nThank you for any time you put into reviewing this question."}}
{"actor":{"display_login":"weixuanfu"}}