{"action":"created","issue":{"url":"https://api.github.com/repos/SheffieldML/GPy/issues/381","repository_url":"https://api.github.com/repos/SheffieldML/GPy","labels_url":"https://api.github.com/repos/SheffieldML/GPy/issues/381/labels{/name}","comments_url":"https://api.github.com/repos/SheffieldML/GPy/issues/381/comments","events_url":"https://api.github.com/repos/SheffieldML/GPy/issues/381/events","html_url":"https://github.com/SheffieldML/GPy/issues/381","id":151368984,"number":381,"title":"Understanding Optimization of parameters in GPy.core.SVGP","user":{"login":"Homesh01","id":17414083,"avatar_url":"https://avatars.githubusercontent.com/u/17414083?v=3","gravatar_id":"","url":"https://api.github.com/users/Homesh01","html_url":"https://github.com/Homesh01","followers_url":"https://api.github.com/users/Homesh01/followers","following_url":"https://api.github.com/users/Homesh01/following{/other_user}","gists_url":"https://api.github.com/users/Homesh01/gists{/gist_id}","starred_url":"https://api.github.com/users/Homesh01/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Homesh01/subscriptions","organizations_url":"https://api.github.com/users/Homesh01/orgs","repos_url":"https://api.github.com/users/Homesh01/repos","events_url":"https://api.github.com/users/Homesh01/events{/privacy}","received_events_url":"https://api.github.com/users/Homesh01/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-04-27T12:05:56Z","updated_at":"2016-04-28T15:27:59Z","closed_at":null,"body":"Hi, \r\n\r\nI'm using GPy.core.SVGP for large datasets; I'm trying to understand the implementation better. Here is code from your tutorials;\r\n```\r\nN=5000\r\nX = np.random.rand(N)[:, None]\r\nY1 = np.sin(6*X) + 0.1*np.random.randn(N,1)\r\nY2 = np.sin(3*X) + 0.1*np.random.randn(N,1)\r\nY = np.hstack((Y1, Y2))\r\nZ = np.random.rand(20,1)\r\n\r\nbatchsize = 100\r\nm = GPy.core.SVGP(X, Y, Z, GPy.kern.RBF(1) + GPy.kern.White(1), GPy.likelihoods.Gaussian(), batchsize=batchsize)\r\nm.kern.white.variance = 1e-5\r\n#m.kern.white.fix()\r\n```\r\n\r\nThis makes sense. However when I investigate the number of hyperparameters which will be optimized in the above model I use the following:\r\n```\r\nprint(m.optimizer_array.shape)\r\nprint(m.kern.param_array.shape)\r\n```\r\nand get:\r\n```\r\n(484,)\r\n(3,)\r\n```\r\n\r\nThere are 3 parameters (2 from the rbf kernel and one from the white noise). Combine that with the 20 inducing points to give 23. So I expected 23 hyperparameters would be optimized over. Can someone explain why 484 hyperparameters are been optimized?\r\n\r\nKind Regards,\r\n\r\nHomesh\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/SheffieldML/GPy/issues/comments/215466679","html_url":"https://github.com/SheffieldML/GPy/issues/381#issuecomment-215466679","issue_url":"https://api.github.com/repos/SheffieldML/GPy/issues/381","id":215466679,"user":{"login":"Homesh01","id":17414083,"avatar_url":"https://avatars.githubusercontent.com/u/17414083?v=3","gravatar_id":"","url":"https://api.github.com/users/Homesh01","html_url":"https://github.com/Homesh01","followers_url":"https://api.github.com/users/Homesh01/followers","following_url":"https://api.github.com/users/Homesh01/following{/other_user}","gists_url":"https://api.github.com/users/Homesh01/gists{/gist_id}","starred_url":"https://api.github.com/users/Homesh01/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Homesh01/subscriptions","organizations_url":"https://api.github.com/users/Homesh01/orgs","repos_url":"https://api.github.com/users/Homesh01/repos","events_url":"https://api.github.com/users/Homesh01/events{/privacy}","received_events_url":"https://api.github.com/users/Homesh01/received_events","type":"User","site_admin":false},"created_at":"2016-04-28T15:27:59Z","updated_at":"2016-04-28T15:27:59Z","body":"Of course, thanks for clarifying that.\n\nOn Wed, Apr 27, 2016 at 1:13 PM, Neil Lawrence <notifications@github.com>\nwrote:\n\n> In stochastic variational inference you optimise over the distribution q(u)\n> also (which has a mean and a covariance)\n>\n> Neil\n>\n> On Wed, Apr 27, 2016 at 8:05 AM, IrishBeast1 <notifications@github.com>\n> wrote:\n>\n> > Hi,\n> >\n> > I'm using GPy.core.SVGP for large datasets; I'm trying to understand the\n> > implementation better. Here is code from your tutorials;\n> > `import numpy as np\n> > import GPy\n> > from matplotlib import pyplot as plt\n> > import climin\n> >\n> > N=5000\n> > X = np.random.rand(N)[:, None]\n> > Y1 = np.sin(6*X) + 0.1*np.random.randn(N,1)\n> > Y2 = np.sin(3*X) + 0.1*np.random.randn(N,1)\n> > Y = np.hstack((Y1, Y2))\n> >\n> > Z = np.random.rand(20,1)\n> >\n> > batchsize = 100\n> > m = GPy.core.SVGP(X, Y, Z, GPy.kern.RBF(1) + GPy.kern.White(1),\n> > GPy.likelihoods.Gaussian(), batchsize=batchsize)\n> > m.kern.white.variance = 1e-5\n> > #m.kern.white.fix()`\n> >\n> > This makes sense. However When I investigate the number of\n> hyperparameters\n> > which will be optimized in the above model I use the following:\n> > print(m.optimizer_array.shape\n> > print(m.kern.param_array.shape)\n> > and get:\n> > (484,)\n> > (3,)\n> >\n> > There are 3 parameters (2 from the rbf kernel and one from the white\n> > noise). Combine that with the 20 inducing points to give 23. So I\n> expected\n> > 23 hyperparameters would be optimized over. Can someone explain why 484\n> > hyperparameters are been optimized?\n> >\n> > Kind Regards,\n> >\n> > Homesh\n> >\n> > —\n> > You are receiving this because you are subscribed to this thread.\n> > Reply to this email directly or view it on GitHub\n> > <https://github.com/SheffieldML/GPy/issues/381>\n> >\n>\n> —\n> You are receiving this because you authored the thread.\n> Reply to this email directly or view it on GitHub\n> <https://github.com/SheffieldML/GPy/issues/381#issuecomment-215063729>\n>\n"}}