Take the 2-minute tour ×

I am taking a machine learning course and today we were given an example of regression, with two attributes $x_,x_2$ and $y$ being the real valued outcome.
$y$ is a quadratic function of $x_1,x_2$, given as
$$y=\alpha_1 x_1^2+\alpha_2 x_2^2+\alpha_3 x_1x_2+\alpha_4x_1+\alpha_5x_2+c$$ Now my teacher told that we can view this function as a linear function of $\lt x_1^2,x_2^2,x_1x_2,x_1,x_2 \gt$.
But I can't really visualize how can a curved function look like a straight line.
Please some one explain it or give some good reference.

share|improve this question
It is a strange way to look at it. You can define the linear (actually affine) function $f(z_1,z_2,z_3,z_4,z_5) = c+\sum_{i=1}^5 \alpha_i z_i$, and then you have $y = f(x_1^2,x_2^2,x_1 x_2 , x_1, x_2)$, but its not clear why you want to do that. –  copper.hat Jul 27 '12 at 7:19
Perhaps the best idea is not to think of it geometrically but algebraically. We need to pretend $x_1^2,x_2^2,x_1x_2,x_1,x_2$ are five independent variables, and I don't think attempting to visualize five-dimensional hyperplanes in relation to a quadratic surface in three dimensions is the way; just work formally. BTW, in what context are you supposed to view it as a linear function? The details of the specific application might lend to a more narrowly tailored explanation behind why such an interpretation works. –  anon Jul 27 '12 at 7:19
@anon It is related to support vector machines. Although he hadn't taught about SVMs but he just gave an idea that we can view every function as a straight line in higher dimensions. –  Happy Mittal Jul 27 '12 at 7:24
I haven't time for a complete answer, but Figures 93 and 94 on this page show how the simpler case of how $f(x,y) = ax + by + c(x^2+y^2)$ can be visualized as a linear function of $(x,y,x^2+y^2)$. The point is that a nonlinear function of the original data is turned into a linear function of a curved warping of the data. –  Rahul Jul 27 '12 at 7:52
add comment

1 Answer

It would seem to me that your teacher has completely botched his/her explanation. What s/he probably intended to say is that your model function ($y$ as a function of the parameters $\alpha_k$) is what's termed as a linear model (and thus, the kind of regression you need to do here is called a linear regression).

Here, you can think of the adjective "linear" like so: if you take the partial derivative of $y$ with respect to any of the $\alpha_j$, you will obtain an expression that is free of any of the $\alpha_j$ (in fact, the result will be whatever was multiplied with the $\alpha_j$ under consideration). As an example,

$$\frac{\partial y}{\partial \alpha_4}=x_1$$

See also this answer in CV.

share|improve this answer
add comment

Your Answer


