Take the 2-minute tour ×

I have searched stack overflow and have not found any question that really is the same as mine because none really have more than one independent variable. Basically I have an array of datapoints and I want to be able to find a regression equation for those data points. The code I have so far looks like this: (w,x,z are the independent variables and y is the dependent variable)

var dataPoints = [{
 "w" : 1, "x" : 2, "z" : 1, "y" : 7
}, {
 "w" : 2, "x" : 1, "z" : 4, "y" : 5
}, {
 "w" : 1, "x" : 5, "z" : 3, "y" : 2
}, {
 "w" : 4, "x" : 3, "z" : 5, "y" : 15

I would like a function that would return a formula object like this:

var regressionEquation = [{
 "var" : "w", "power" : 1, "coeff" : "1.5"
}, {
 "var" : "x", "power" : 1, "coeff" : "2"
}, {
 "var" : "z", "power" : 1, "coeff" : "1"

Is there a way to come up with a regression equation like this without using a loop to step and plug in the values? Is there a way to come up with the regression equation for powers that are more than 1? Thanks in advance.


Many people have suggested solving a system of equations made by plugging in the powers. The problem I have with this is when there is more than enough data points to solve for a system of equations. In the examples in the question, I have 3 variables in order to solve the system of equations that people are suggesting, I would need 3 datapoints but I have 4. This leads to a problem because there is more than one solution. There are 4 possible solutions because there are 4 ways to combine the 4 equations into different groups of 3. This would leave me with 4 answers with possibly none of them the best fit to all 4 points.

share|improve this question
Are you thinking about something like Polynomial Interpolation? (en.wikipedia.org/wiki/Polynomial_interpolation) I've written some js to do that before through operations on a Vandermonde matrix. –  sgcharlie Nov 13 '12 at 21:22
No I would like to find a regression equation and then extrapolate –  Vedaant Arora Nov 13 '12 at 22:27
so you have function w = ax^k1 + by^k2 + cz^k3 where you want to find a, b, c, k1, k2 and k3 that fit your data? –  Teemu Ikonen Nov 18 '12 at 14:47
I want a process that will find a, b, and c. But I would like it to work for any values k1, k2 and k3. (I will have already decided these values) –  Vedaant Arora Nov 18 '12 at 19:37
This is probably better suited with a math tag. –  Andrew Rhyne Nov 19 '12 at 20:20

3 Answers 3

The problem as you've stated it is equivalent, under transformation, to a linear regression problem. You said in comments that you have fixed exponents k_1, k_2, and k_3. The transformation takes a tuple {w, x, z ,y} to the tuple {w^k_1, x^k_2, z^k_2, y} = {w', x', z' ,y}. Use linear regression on the primed variables to get your coefficients.

For example, if k_1 = 2, k_2 = 3, and k_3 = 1, then here's a single example of the transform:

==> {"w*" : 16, "x*" : 27, "z*" : 5, "y" : 15}

This is just a special case of how you convert a polynomial regression problem into a linear regression one. In your case the polynomial forms you are considering are particularly simple.

Use any JavaScript library you like to solve the linear regression problem; there are a number of them.

share|improve this answer
please look at the edit for my problems with your answer –  Vedaant Arora Nov 22 '12 at 13:10

I think if it is the case that there are four equations and only 3 variables (As you already determined the powers, plugin and make it a linear equation), the linear equation is over complete, and there does not exist an exact answer that will satisfy all four equations.

What you can do is to minimize the residual error and get a best approximation.

Assume you have coefficients a b and c for the w x and z,

define matrix


and define the vector


define vector


Then the problem is

M*v=r solve v. 

1. If rank(M)>variable number, you have to minimize the residual error


Since this is convex, take derivative on it and make it zero:

M^T*M*v-M^T*r=0 => v=(M^T*M)\M^T*r. 

(M^T*M)\M^T is MP-inverse of M, if rank(M)>variable number, then (M^T*M) is inversible.

2. If the rank(M)<=variable number, you can get infinitely many exact solution to the equation.


Let singular value decomposition of M:




is one of the solutions.

V*S^-1*U^T is pseudo inverse of M.

If you use a linear algebra library, it is very easy to get closed form solution without iterating. http://sylvester.jcoglan.com/

share|improve this answer
If the first case sylvester.jcoglan.com/api/matrix.html#inverse can already solve. But if the second case, I didn't see any javascript library that can return a pseudo inverse. Maybe this library numericjs.com can help because it can compute svd of a matrix, and you can get pseudo inverse using U S and V. –  Min Lin Nov 25 '12 at 6:39
I don't truly understand all of the math used in the explanation what I understand is that in the first case(which is the only one I care for) in order to come up with the solution I would need to use to library you provided to find the inverse of the M and then find that * r using that library. Am I correct? –  Vedaant Arora Nov 25 '12 at 15:32
I tried this and it caused an error because M is not inversible because it is not a square –  Vedaant Arora Nov 25 '12 at 16:03
No it is not inverse of M, it is MP-inverse of M, v=inverse(transpose(M)*M)*transpose(M)*r; because transpose(M)*M is square, and in your case full rank, it is inversible. –  Min Lin Nov 26 '12 at 1:22
If your matrix M is not too big, this closed form solution would be very fast. –  Min Lin Nov 26 '12 at 1:25

I would suggest using least squares for getting a linear equation. Further, you could use non-linear least squares, given that you know in advance the function you want to fit.


There are several links for linear LS in javascript and you can probably adapt these to 3 dimensions (e.g. http://dracoblue.net/dev/linear-least-squares-in-javascript/159/ from a quick Google search). For the non-linear case it would need some more work though.

share|improve this answer
The way I understood the link that you posted it would not work for multiple variables which is a major problem –  Vedaant Arora Nov 25 '12 at 14:59
Well, it can work for any number of variables or any function (see for example en.wikipedia.org/wiki/…) -- the thing is that it is mostly used for y=ax+b, so most of the code you'll find will be targeting this type of function. For the linear case with multiple variables see en.wikipedia.org/wiki/…. But generally, for non-linear problems you'll probably have to work out some math by yourself. –  nvlass Nov 25 '12 at 15:43
Edit/Addition: Since, however, the system is over-determined, it's probably the way to go. –  nvlass Nov 25 '12 at 15:50
The problem really is that I don't know how to do this math –  Vedaant Arora Nov 25 '12 at 18:03

Your Answer


