{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-optimize/scikit-optimize/issues/653","repository_url":"https://api.github.com/repos/scikit-optimize/scikit-optimize","labels_url":"https://api.github.com/repos/scikit-optimize/scikit-optimize/issues/653/labels{/name}","comments_url":"https://api.github.com/repos/scikit-optimize/scikit-optimize/issues/653/comments","events_url":"https://api.github.com/repos/scikit-optimize/scikit-optimize/issues/653/events","html_url":"https://github.com/scikit-optimize/scikit-optimize/issues/653","id":310764915,"node_id":"MDU6SXNzdWUzMTA3NjQ5MTU=","number":653,"title":"TypeError: %d format: a number is required, not tuple - when changing n_iter","user":{"login":"Atarust","id":15972036,"node_id":"MDQ6VXNlcjE1OTcyMDM2","avatar_url":"https://avatars1.githubusercontent.com/u/15972036?v=4","gravatar_id":"","url":"https://api.github.com/users/Atarust","html_url":"https://github.com/Atarust","followers_url":"https://api.github.com/users/Atarust/followers","following_url":"https://api.github.com/users/Atarust/following{/other_user}","gists_url":"https://api.github.com/users/Atarust/gists{/gist_id}","starred_url":"https://api.github.com/users/Atarust/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Atarust/subscriptions","organizations_url":"https://api.github.com/users/Atarust/orgs","repos_url":"https://api.github.com/users/Atarust/repos","events_url":"https://api.github.com/users/Atarust/events{/privacy}","received_events_url":"https://api.github.com/users/Atarust/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-04-03T09:43:24Z","updated_at":"2018-06-09T11:42:58Z","closed_at":"2018-06-09T11:42:58Z","author_association":"NONE","body":"When running `opt.fit(X_train, y_train)` with n_iter=10 (or higher) I get the error `TypeError: %d format: a number is required, not tuple`. With n_iter=9 or less, it works (with only a few convergence warnings).\r\nEDIT: seen with version 0.5.2\r\n```\r\n# minimal example\r\nfrom skopt import BayesSearchCV\r\nfrom sklearn.model_selection import train_test_split\r\nfrom sklearn.neural_network.multilayer_perceptron import MLPRegressor\r\nimport itertools\r\n\r\nXS = [[1], [2], [3], [4], [5], [6]]\r\nys = [2, 3, 4, 5, 6, 7]\r\nX_train, X_test, y_train, y_test = train_test_split(XS, ys, train_size=0.75, random_state=1)\r\n\r\nopt = BayesSearchCV(\r\n    MLPRegressor(),\r\n    {\r\n        'hidden_layer_sizes': [x for x in itertools.product((10, 100), repeat=2)]\r\n    },\r\n    n_iter=10\r\n)\r\n\r\nopt.fit(X_train, y_train)\r\n\r\n```\r\n# Full error log\r\n\r\n```\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/model_selection/_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\r\n  FutureWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\n/home/atarust/.local/lib/python3.5/site-packages/sklearn/neural_network/multilayer_perceptron.py:564: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\r\n  % self.max_iter, ConvergenceWarning)\r\nTraceback (most recent call last):\r\n  File \"/home/atarust/eclipse-workspace/learnSkiKit/src/minExample.py\", line 19, in <module>\r\n    opt.fit(X_train, y_train)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/skopt/searchcv.py\", line 654, in fit\r\n    groups=groups, n_points=n_points_adjusted\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/skopt/searchcv.py\", line 541, in _step\r\n    params = optimizer.ask(n_points=n_points)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/skopt/optimizer/optimizer.py\", line 372, in ask\r\n    opt._tell(x, y_lie)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/skopt/optimizer/optimizer.py\", line 486, in _tell\r\n    est.fit(self.space.transform(self.Xi), self.yi)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/skopt/learning/gaussian_process/gpr.py\", line 195, in fit\r\n    super(GaussianProcessRegressor, self).fit(X, y)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py\", line 217, in fit\r\n    self.kernel_.bounds))]\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py\", line 454, in _constrained_optimization\r\n    fmin_l_bfgs_b(obj_func, initial_theta, bounds=bounds)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/scipy/optimize/lbfgsb.py\", line 199, in fmin_l_bfgs_b\r\n    **opts)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/scipy/optimize/lbfgsb.py\", line 335, in _minimize_lbfgsb\r\n    f, g = func_and_grad(x)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/scipy/optimize/lbfgsb.py\", line 285, in func_and_grad\r\n    f = fun(x, *args)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/scipy/optimize/optimize.py\", line 292, in function_wrapper\r\n    return function(*(wrapper_args + args))\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/scipy/optimize/optimize.py\", line 63, in __call__\r\n    fg = self.fun(x, *args)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py\", line 209, in obj_func\r\n    theta, eval_gradient=True)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/sklearn/gaussian_process/gpr.py\", line 412, in log_marginal_likelihood\r\n    K, K_gradient = kernel(self.X_train_, eval_gradient=True)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/sklearn/gaussian_process/kernels.py\", line 679, in __call__\r\n    K1, K1_gradient = self.k1(X, Y, eval_gradient=True)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/sklearn/gaussian_process/kernels.py\", line 754, in __call__\r\n    K2, K2_gradient = self.k2(X, Y, eval_gradient=True)\r\n  File \"/home/atarust/.local/lib/python3.5/site-packages/skopt/learning/gaussian_process/kernels.py\", line 391, in __call__\r\n    (X.shape, len(length_scale)))\r\nTypeError: %d format: a number is required, not tuple\r\n```"},"comment":{"url":"https://api.github.com/repos/scikit-optimize/scikit-optimize/issues/comments/395962712","html_url":"https://github.com/scikit-optimize/scikit-optimize/issues/653#issuecomment-395962712","issue_url":"https://api.github.com/repos/scikit-optimize/scikit-optimize/issues/653","id":395962712,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NTk2MjcxMg==","user":{"login":"iaroslav-ai","id":14038787,"node_id":"MDQ6VXNlcjE0MDM4Nzg3","avatar_url":"https://avatars2.githubusercontent.com/u/14038787?v=4","gravatar_id":"","url":"https://api.github.com/users/iaroslav-ai","html_url":"https://github.com/iaroslav-ai","followers_url":"https://api.github.com/users/iaroslav-ai/followers","following_url":"https://api.github.com/users/iaroslav-ai/following{/other_user}","gists_url":"https://api.github.com/users/iaroslav-ai/gists{/gist_id}","starred_url":"https://api.github.com/users/iaroslav-ai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/iaroslav-ai/subscriptions","organizations_url":"https://api.github.com/users/iaroslav-ai/orgs","repos_url":"https://api.github.com/users/iaroslav-ai/repos","events_url":"https://api.github.com/users/iaroslav-ai/events{/privacy}","received_events_url":"https://api.github.com/users/iaroslav-ai/received_events","type":"User","site_admin":false},"created_at":"2018-06-09T11:42:58Z","updated_at":"2018-06-09T11:42:58Z","author_association":"MEMBER","body":"Hi @Atarust ,\r\nthe reason you see an exception is because as bounds for space `hidden_layer_sizes`, you provide `[(10, 10), (10, 100), (100, 10), (100, 100)]`.\r\n\r\nBayesSearchCV expects that any parameter of the underlying estimator is of a Categorical, Integer or Real type. The values that you specify in search space are typically either left and right bound of dimensions, or a list of categorical values to choose from. `hidden_layer_sizes` is a rare exception in that it should be an array of values, which is currently not supported directly by BayesSearchCV.\r\n\r\nOne way to work around this is to wrap MLPRegressor, e.g.:\r\n```python\r\n# minimal example\r\nfrom skopt import BayesSearchCV\r\nfrom skopt.space import Integer\r\nfrom sklearn.model_selection import train_test_split\r\nfrom sklearn.neural_network import MLPRegressor\r\nfrom sklearn.base import BaseEstimator, RegressorMixin\r\nimport itertools\r\n\r\nXS = [[1], [2], [3], [4], [5], [6]]\r\nys = [2, 3, 4, 5, 6, 7]\r\nX_train, X_test, y_train, y_test = train_test_split(XS, ys, train_size=0.75, random_state=1)\r\n\r\n# Ugly wrapper around MLPRegressor; The ugliness is to make parameters of the estimator be recognized properly\r\nclass MLPWrapper(BaseEstimator, RegressorMixin):\r\n    def __init__(self, layer1=10, layer2=10, layer3=10):\r\n        self.layer1 = layer1\r\n        self.layer2 = layer2\r\n        self.layer3 = layer3\r\n\r\n    def fit(self, X, y):\r\n        model = MLPRegressor(\r\n            hidden_layer_sizes=[self.layer1, self.layer2, self.layer3]\r\n        )\r\n        model.fit(X, y)\r\n        self.model = model\r\n        return self\r\n\r\n    def predict(self, X):\r\n        return self.model.predict(X)\r\n\r\n    def score(self, X, y):\r\n        return self.model.score(X, y)\r\n\r\n\r\nopt = BayesSearchCV(\r\n    estimator=MLPWrapper(),\r\n    search_spaces={\r\n        'layer1': Integer(10, 100),\r\n        'layer2': Integer(10, 100),\r\n        'layer3': Integer(10, 100)\r\n    },\r\n    n_iter=11\r\n)\r\n\r\nopt.fit(X_train, y_train)\r\n```\r\nIf you have a better solution let us know. I am closing this for now - feel free to reopen."}}
{"actor":{"display_login":"iaroslav-ai"}}