title: Gradient Boost Decision Tree (GBDT) or Multiple Additive Regression Tree(MART): Calculating gradient/pseudo-response
body: <p>I'm implementing MART from (<a href="http://www-stat.stanford.edu/~jhf/ftp/trebst.pdf" rel="nofollow noreferrer">http://www-stat.stanford.edu/~jhf/ftp/trebst.pdf</a>) algorithm 5,
  My algorithm "works" for say less data(3000 training data file, 22 features) and J=5,10,20 (# of leaf nodes) and T = 10, 20. It gives me good result (R-Precision is 0.30 to 0.5 for training) but when I try to run on some what large training data (70K records) it gives me runtime underflow error - which I think it should be - just don't know how workaround this problem?</p>
  
  <p>Underflow err comes here, calculating gradient of cost (or pseudo-response):
  <img src="https://i.stack.imgur.com/USBV9.png" alt="enter image description here"></p>
  
  <p>here y_i are {1,-1} labels so if I just try: 2/exp(5000) its overflow in denominator!</p>
  
  <p>Just wondering if I can "normalize" this or "threshold" this, but then I'm using this pseudo-response in calculating "label" (gamma in that pdf), and then those gamma to calculate model scores. </p>