title: Plotting gradient descent function in R
body: <p>I have implemented a multivariate linear regression in R followed by a batch update gradient descent algorithm. I am now trying to plot the results of this gradient descent.</p>
  <p>I have found instructional links on how to create a plot <a href="http://www.machinegurning.com/rstats/gradient-descent/" rel="nofollow noreferrer">here</a> and <a href="https://www.r-bloggers.com/gradient-descent-in-r/" rel="nofollow noreferrer">here</a>.</p>
  <p>The problem with these tutorials is that in both cases, they explicitly define the linear regression equation (also it is not multivariate).</p>
  <p>How can I create similar plots to overlay the results of running the <code>gradDescent</code> function several times with different learning rates and convergence thresholds in the code listed below:</p>
  <pre><code>data &lt;- read.csv(&quot;Data/Bike-Sharing-Dataset/hour.csv&quot;)
  
  # Select the useable features
  data1 &lt;- data[, c(&quot;season&quot;, &quot;mnth&quot;, &quot;hr&quot;, &quot;holiday&quot;, &quot;weekday&quot;, &quot;workingday&quot;, &quot;weathersit&quot;, &quot;temp&quot;, &quot;atemp&quot;, &quot;hum&quot;, &quot;windspeed&quot;, &quot;cnt&quot;)]
  
  # Set seed
  set.seed(100)
  
  # Split the data
  trainingObs&lt;-sample(nrow(data1),0.70*nrow(data1),replace=FALSE)
  
  # Create the training dataset
  trainingDS&lt;-data1[trainingObs,]
  
  # Create the test dataset
  testDS&lt;-data1[-trainingObs,]
  
  # Create the variables
  y &lt;- trainingDS$cnt
  y_test &lt;- testDS$cnt
  X &lt;- as.matrix(trainingDS[-ncol(trainingDS)])
  X_test &lt;- as.matrix(testDS[-ncol(testDS)])
  
  int &lt;- rep(1, length(y))
  
  # Add intercept column to X
  X &lt;- cbind(int, X)
  X_test &lt;- cbind(int, X_test)
  
  # Solve for beta
  betas &lt;- solve(t(X) %*% X) %*% t(X) %*% y
  
  # Round the beta values
  betas &lt;- round(betas, 2)
  
  # Gradient descent 1
  gradientDesc &lt;- function(x, y, learn_rate, conv_threshold, max_iter) {
    n &lt;- nrow(x) 
    m &lt;- runif(ncol(x), 0, 1)
    yhat &lt;- x %*% m
    
    cost &lt;- sum((y - yhat) ^ 2) / (2*n)
  
    converged = F
    iterations = 0
    
    while(converged == F) {
      ## Implement the gradient descent algorithm
      m &lt;- m - learn_rate * ( 1/n * t(x) %*% (yhat - y))
      yhat &lt;- x %*% m
      new_cost &lt;- sum((y - yhat) ^ 2) / (2*n)
      
      if( abs(cost - new_cost) &lt;= conv_threshold) {
        converged = T
      }
      iterations = iterations + 1
      cost &lt;- new_cost
      
      if(iterations &gt;= max_iter) break
    }
    return(list(converged = converged, 
                num_iterations = iterations, 
                cost = cost,
                new_cost = new_cost,
                coefs = m) )
  }
  
  out &lt;- gradientDesc(X, y, 0.005, 0.0000001, 200000)
  </code></pre>
  <p><strong>Note</strong>:
  The data being used is-</p>
  <blockquote>
  <p>Bike-Sharing-Dataset</p>
  <p>UCI Machine Learning Repository</p>
  </blockquote>