title: Different gradient calculations in a logistic regression
body: <p>I am trying to find how one variable (<code>EVI</code>) may predict a binary outcome (<code>an_larv_bin</code>) using <code>glmer</code> from <code>lme4</code> package. The code I input is:</p>
  <pre><code>univ_points_evi &lt;- glmer(an_larv_bin ~ EVI + (1|grid_no), family=&quot;binomial&quot;, data=univ_points)
  </code></pre>
  <p>and I come up against the following warning message:</p>
  <pre><code>In checkConv(attr(opt, &quot;derivs&quot;), opt$par, ctrl = control$checkConv,  :
    Model failed to converge with max|grad| = 0.0331857 (tol = 0.002, component 1)
  </code></pre>
  <p>I have rescaled and centered the variable as such:</p>
  <pre><code>scale(EVI, center = TRUE, scale = TRUE)
  </code></pre>
  <p>And still receive the same warning. The next steps I took from <a href="https://rstudio-pubs-static.s3.amazonaws.com/33653_57fc7b8e5d484c909b615d8633c01d51.html" rel="nofollow noreferrer">Rstudio troubleshooting for lme4 convergence warnings</a>. I checked for singularity, but it was not a problem:</p>
  <pre><code>&gt; tt &lt;- getME(univ_points_evi,&quot;theta&quot;)
  &gt; ll &lt;- getME(univ_points_evi,&quot;lower&quot;)
  &gt; min(tt[ll==0])
  [1] 0.80632
  </code></pre>
  <p>I restarted the function from a different fit:</p>
  <pre><code>ss &lt;- getME(univ_points_evi,c(&quot;theta&quot;,&quot;fixef&quot;))
  m2 &lt;- update(univ_points_evi,start=ss,control=glmerControl(optCtrl=list(maxfun=2e4)))
  </code></pre>
  <p>And get the same warning again. I checked the gradient calculations and got the embedded results:</p>
  <pre><code>#pre-computed information 
  derivs1 &lt;- univ_points_evi@optinfo$derivs
  sc_grad1 &lt;- with(derivs1,solve(Hessian,gradient))
  max(abs(sc_grad1))
  [1] 4.991948e-05
  
  
  max(pmin(abs(sc_grad1),abs(derivs1$gradient)))
  [1] 4.991948e-05
  
  #numDeriv
  dd &lt;- update(univ_points_evi,devFunOnly=TRUE)
  pars &lt;- unlist(getME(univ_points_evi,c(&quot;theta&quot;,&quot;fixef&quot;)))
  grad2 &lt;- grad(dd,pars)
  hess2 &lt;- hessian(dd,pars)
  sc_grad2 &lt;- solve(hess2,grad2)
  max(pmin(abs(sc_grad2),abs(grad2)))
  [1] 3.308277
  </code></pre>
  <p>The third figure is obviously quite different from the first two. I am unsure what this is actually telling me, and whether it is the root of the warning?</p>
  <p>Lastly, I checked the fit of all optimisers:</p>
  <pre><code>univ_points_evi.all &lt;- allFit(univ_points_evi)
  </code></pre>
  <p>And got the following:</p>
  <pre><code>bobyqa : [OK]
  Nelder_Mead : [OK]
  nlminbwrap : [OK]
  nmkbw : [OK]
  optimx.L-BFGS-B : [OK]
  nloptwrap.NLOPT_LN_NELDERMEAD : [OK]
  nloptwrap.NLOPT_LN_BOBYQA : [OK]
  </code></pre>
  <p>And tried all available optimisers, which converge to values that are practically equivalent.</p>
  <pre><code>ss &lt;- summary(univ_points_evi.all)
  ss$ fixef
  ss$ llik
  ss$ sdcor
  ss$ theta
                                
  </code></pre>
  <p>The lme4 package guidelines state that they would in this case consider the convergence warnings to be false positives. I am wondering whether this is true in my case, or if there is an issue with the gradient calculations which is causing the issue.</p>
  <p>My data:</p>
  <pre><code>&gt; dput(EVI)
  c(0.499929994, 0.589900017, 0.593994021, 0.589900017, 0.601158023, 
  0.492922992, 0.546519995, 0.601045012, 0.536565006, 0.592272997, 
  0.592227995, 0.645565987, 0.61619997, 0.516200006, 0.516200006, 
  0.4639, 0.4639, 0.561200023, 0.5898, 0.564800024, 0.5898, 0.5898, 
  0.605099976, 0.595300019, 0.545300007, 0.572000027, 0.599600017, 
  0.585300028, 0.591700017, 0.533399999, 0.552100003, 0.569700003, 
  0.592499971, 0.596199989, 0.53490001, 0.53490001, 0.53490001, 
  0.553300023, 0.582899988, 0.545000017, 0.592100024, 0.582899988, 
  0.59009999, 0.569299996, 0.612900019, 0.533500016, 0.583299994, 
  0.772599995, 0.772599995, 0.682500005, 0.682500005, 0.682500005, 
  0.772599995, 0.628099978, 0.626299977, 0.628099978, 0.747399986, 
  0.640200019, 0.531899989, 0.680199981, 0.535099983, 0.680199981, 
  0.535099983, 0.565299988, 0.680199981, 0.703199983, 0.703199983, 
  0.541700006, 0.678200006, 0.678200006, 0.547100008, 0.634899974, 
  0.696399987, 0.688199997, 0.574899971, 0.574899971, 0.669799984, 
  0.611000001, 0.61559999, 0.639100015, 0.669799984, 0.669799984, 
  0.611000001, 0.59890002, 0.639100015, 0.604799986, 0.604799986, 
  0.604799986, 0.606599987, 0.606599987, 0.640600026, 0.624899983, 
  0.640600026, 0.624899983, 0.624899983, 0.640600026, 0.640600026, 
  0.516200006, 0.507499993, 0.507499993, 0.46540001, 0.530300021, 
  0.530300021, 0.565100014, 0.546599984, 0.546599984, 0.530399978, 
  0.530399978, 0.530399978, 0.523199975, 0.523199975, 0.546400011, 
  0.546599984, 0.496600002, 0.530799985, 0.537800014, 0.545000017, 
  0.496600002, 0.496600002, 0.514100015, 0.530799985, 0.530799985, 
  0.537800014, 0.530200005, 0.530200005, 0.546599984, 0.546599984, 
  0.576399982, 0.46540001, 0.516200006, 0.530399978, 0.655300021, 
  0.680999994, 0.660000026, 0.661499977, 0.661499977, 0.680999994, 
  0.655300021, 0.617799997, 0.647099972, 0.647099972, 0.617799997, 
  0.673300028, 0.673300028, 0.507700026, 0.507700026, 0.507700026, 
  0.651799977, 0.591799974, 0.591799974, 0.688300014, 0.661499977, 
  0.661499977, 0.661499977, 0.661499977, 0.648500025, 0.648500025, 
  0.495799989, 0.495799989, 0.495799989, 0.648899972, 0.648899972, 
  0.673300028, 0.673300028, 0.648500025, 0.647099972, 0.691999972, 
  0.647099972, 0.647099972, 0.617799997, 0.657199979, 0.706499994, 
  0.591799974, 0.661499977, 0.661499977, 0.641600013, 0.648500025, 
  0.648500025, 0.688300014, 0.495799989, 0.495799989, 0.688300014, 
  0.582000017, 0.582000017, 0.57069999, 0.582000017, 0.62559998, 
  0.565500021, 0.565500021, 0.62559998, 0.593599975, 0.604700029, 
  0.599699974, 0.536800027, 0.600300014, 0.600300014, 0.604700029, 
  0.566699982, 0.566699982, 0.626900017, 0.626900017, 0.594900012, 
  0.594900012, 0.584500015, 0.586199999, 0.605700016, 0.584699988, 
  0.553799987, 0.542900026, 0.584699988, 0.584699988, 0.575399995, 
  0.579999983, 0.579299986, 0.596899986, 0.594900012, 0.565500021, 
  0.579299986, 0.594900012, 0.565500021, 0.549499989, 0.549499989, 
  0.549499989, 0.549499989, 0.606899977, 0.539600015, 0.584699988, 
  0.571699977, 0.56129998, 0.595600009, 0.62559998, 0.565500021, 
  0.565500021, 0.620299995, 0.620299995, 0.594900012, 0.579999983, 
  0.654299974, 0.654299974, 0.627600014, 0.627600014, 0.64349997, 
  0.687699974, 0.64349997, 0.59859997, 0.59859997, 0.649999976, 
  0.518299997, 0.658299983, 0.658299983, 0.658299983, 0.627600014, 
  0.658299983, 0.658299983, 0.627600014, 0.667500019, 0.653100014, 
  0.564899981, 0.561999977, 0.629000008, 0.639999986, 0.639999986, 
  0.675100029, 0.675100029, 0.658299983, 0.659300029, 0.658299983, 
  0.659300029, 0.657400012, 0.645299971, 0.425599992, 0.425599992, 
  0.474299997, 0.598800004, 0.595200002, 0.416399986, 0.564899981, 
  0.564899981, 0.70599997, 0.70599997, 0.664699972, 0.484299988, 
  0.496199995, 0.496199995, 0.484299988, 0.517499983, 0.517499983, 
  0.517499983, 0.535899997, 0.51730001, 0.562399983, 0.540000021, 
  0.540000021, 0.501299977, 0.501299977, 0.528599977, 0.532400012, 
  0.51730001, 0.562399983, 0.501299977, 0.574299991, 0.528599977, 
  0.528599977, 0.528599977, 0.503499985, 0.568700016, 0.521799982, 
  0.503499985, 0.521799982, 0.557699978, 0.557699978, 0.545099974, 
  0.532400012, 0.563399971, 0.530700028, 0.431100011, 0.431100011, 
  0.510900021, 0.556400001, 0.501299977, 0.48120001, 0.48120001, 
  0.528800011, 0.528800011, 0.62470001, 0.62470001, 0.707899988, 
  0.707899988, 0.62529999, 0.62529999, 0.630500019, 0.646300018, 
  0.604900002, 0.62529999, 0.669799984, 0.634199977, 0.634199977, 
  0.634199977, 0.612999976, 0.662400007, 0.698700011, 0.632799983, 
  0.682099998, 0.428499997, 0.513300002, 0.569700003, 0.519500017, 
  0.519500017, 0.48120001, 0.48120001, 0.646399975, 0.559899986, 
  0.564899981, 0.564899981, 0.564899981, 0.602699995, 0.602699995, 
  0.60650003, 0.575699985, 0.5722, 0.584299982, 0.584900022, 0.584900022, 
  0.5722, 0.584299982, 0.5722, 0.560699999, 0.560699999)
  &gt; dput(an_larv_bin)
  c(1L, 0L, 1L, 1L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 
  0L, 1L, 0L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 
  0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 1L, 1L, 1L, 0L, 
  0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 
  1L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 1L, 1L, 0L, 0L, 1L, 0L, 0L, 0L, 
  0L, 1L, 1L, 1L, 1L, 1L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 
  1L, 1L, 0L, 1L, 1L, 1L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 1L, 1L, 1L, 
  0L, 1L, 0L, 0L, 0L, 0L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 0L, 
  0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 1L, 1L, 
  1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
  0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
  0L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 
  0L, 1L, 1L, 1L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 
  0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 
  0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
  0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
  0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 
  0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 
  0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 0L, 0L, 
  0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 
  0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 
  0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 
  0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 1L, 0L, 
  1L)
  
  
  
   &gt; dput(grid_no)
  c(4937L, 3270L, 2854L, 3270L, 2582L, 2584L, 2585L, 2584L, 4663L, 
  3416L, 3416L, 3979L, 2986L, 4839L, 4839L, 4937L, 4937L, 3264L, 
  2854L, 2854L, 2289L, 2289L, 2582L, 3978L, 3834L, 3416L, 3416L, 
  3547L, 3687L, 2852L, 4388L, 4388L, 4538L, 4538L, 4937L, 4937L, 
  4937L, 2854L, 2854L, 2996L, 2996L, 2289L, 2582L, 3416L, 3692L, 
  2983L, 2301L, 4937L, 3264L, 3264L, 3547L, 3547L, 3547L, 3264L, 
  3822L, 3683L, 3683L, 3678L, 2427L, 2427L, 2427L, 2289L, 2427L, 
  2289L, 4117L, 2710L, 2582L, 2303L, 2854L, 2854L, 4520L, 3692L, 
  3692L, 3416L, 4526L, 4527L, 3264L, 3685L, 3685L, 4937L, 3264L, 
  3264L, 3685L, 4801L, 4937L, 2290L, 2289L, 2289L, 2854L, 2854L, 
  2581L, 2719L, 2719L, 2578L, 2578L, 2582L, 2581L, 3416L, 3978L, 
  3978L, 3416L, 3549L, 3549L, 2986L, 2700L, 2700L, 4680L, 4680L, 
  4680L, 4670L, 4670L, 2428L, 4527L, 3264L, 2854L, 4937L, 2582L, 
  3264L, 3264L, 3264L, 2854L, 2854L, 4937L, 2289L, 2289L, 4527L, 
  4680L, 4680L, 3416L, 3416L, 4680L, 3409L, 3547L, 3685L, 3685L, 
  3685L, 3409L, 3547L, 2861L, 2581L, 2578L, 2861L, 2430L, 2430L, 
  2293L, 2293L, 2293L, 3977L, 3684L, 4523L, 4669L, 3264L, 3264L, 
  3264L, 3264L, 2854L, 2854L, 2289L, 2289L, 2289L, 2577L, 2577L, 
  4937L, 4937L, 2577L, 2582L, 2582L, 2578L, 2578L, 3416L, 3416L, 
  4527L, 4801L, 3685L, 3822L, 2302L, 2855L, 2855L, 4669L, 2287L, 
  2287L, 4669L, 3549L, 3549L, 4798L, 3549L, 4680L, 4680L, 4680L, 
  4822L, 4258L, 4948L, 3273L, 4677L, 4677L, 4677L, 4948L, 2854L, 
  2854L, 3264L, 3264L, 3264L, 4937L, 4937L, 2582L, 2582L, 2578L, 
  2578L, 2289L, 2289L, 2289L, 3416L, 2573L, 3416L, 4527L, 3685L, 
  3547L, 4801L, 3685L, 3547L, 2287L, 2287L, 2287L, 2287L, 2436L, 
  2291L, 2718L, 2718L, 4099L, 3131L, 4680L, 4680L, 4680L, 3260L, 
  3260L, 3977L, 2571L, 2578L, 2578L, 2854L, 2854L, 3264L, 3264L, 
  3264L, 4937L, 4937L, 2582L, 2582L, 2289L, 2289L, 2289L, 2573L, 
  2573L, 2573L, 2573L, 3132L, 3407L, 3416L, 3416L, 3685L, 3685L, 
  3685L, 4527L, 4801L, 2991L, 2287L, 2287L, 2426L, 3399L, 2301L, 
  4680L, 4680L, 4680L, 4541L, 4390L, 3277L, 3277L, 3277L, 3978L, 
  3978L, 3978L, 4937L, 4801L, 4801L, 4937L, 2289L, 2289L, 2289L, 
  2573L, 2854L, 3264L, 3264L, 3264L, 3684L, 3684L, 2582L, 2582L, 
  2854L, 3264L, 3684L, 4527L, 2578L, 2578L, 2718L, 2718L, 2296L, 
  4665L, 4665L, 4665L, 3416L, 3416L, 3277L, 3277L, 2443L, 2300L, 
  2302L, 4680L, 4680L, 4680L, 3546L, 3546L, 4937L, 4937L, 4801L, 
  4801L, 2854L, 2854L, 3264L, 3264L, 2289L, 2289L, 2582L, 2582L, 
  2578L, 2289L, 3416L, 3416L, 3416L, 3556L, 3277L, 3685L, 3978L, 
  4680L, 4110L, 4237L, 4527L, 4237L, 4937L, 4937L, 4801L, 4801L, 
  3264L, 3685L, 3416L, 3416L, 3416L, 2289L, 2289L, 2289L, 2582L, 
  2578L, 2582L, 2293L, 2857L, 2721L, 2443L, 4680L, 4680L)
  </code></pre>
  <p>Thanks in advance!</p>