Take the 2-minute tour ×

My algorithm is like:

data is stored as:

data = [record1, record2, ... ]
    where record1 is [1, x1, x2 ..., x_m] m feature values for that record
theta is parameter of linear regression function, theta is vector of size m+1
y is true label, again array of length, len(data). (y[0] is true value for record 0)

Linear regression Stochastic update:

while True:
    for i in range(len(data)):
        x = data[i]
        for t in range(0, m):
            theta[t] = theta[t] - my_lambda * (np.dot(theta, x) - y[i]) * x[t]
        j_theta = compute_J_of_theta(data, y, theta)
    print "Iteration #: ", iterations, " j_theta ", j_theta
    if j_theta < 5000:
        #print "******************** FINALLY CONVERGED!!!! ********************" 

compute_j_of_theta(data, y, theta):
Convergence criteria,
compute J(theta) = 1/2M sum (h_theta(x_t) - y_t)**2
temp = 0
    x = data[i]
    temp += (np.dot(theta, x) - y[i])**2
return temp/2*M

my_lambda is very small Initially theta is 0 vector of size m+1

Que: Training set err is more than testing ... WHY? what's wrong with this ? EDIT 1: It was my stupid mistake in calculating err

share|improve this question
What is your question exactly? –  DuckMaestro Jan 31 '13 at 4:41
You don't do parallel theta updates, you dot the current theta while updating it. –  Thomas Jungblut Jan 31 '13 at 10:48
@ThomasJungblut didn't get it ... I thought this is how Stochastic update looks like ... could you please make it clear? Thanks! –  code muncher Jan 31 '13 at 15:19
@codemuncher stochastic means that you are updating the weights after a single training item. Parallel means that you update the completed new theta vector after you looped over all M features so you do not manipulate the parameters while calculating with them. So either put the np.dot(theta, x) out of the loop or rewrite the loop so it doesn't use the updating theta anymore. –  Thomas Jungblut Jan 31 '13 at 15:37
add comment

Your Answer



Browse other questions tagged or ask your own question.