<p>The idea of <a href="http://en.wikipedia.org/wiki/Gradient_boosting">gradient boosting</a> is that an ensemble model is built from black-box weak models. You can surely use VW as the black box, but note that VW does not offer decision trees, which are the most popular choice for the black-box weak models in boosting. Boosting in general decreases bias (and increases variance), so you should make sure that the VW models have low variance (no overfitting). See <a href="http://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff">bias-variance tradeoff</a>.</p>

<p>There are some reductions related to boosting and bagging in VW:</p>

<ul>
<li><code>--autolink N</code> adds a link function with polynomial N, which can be considered a simple way of boosting.</li>
<li><code>--log_multi K</code> is an online boosting algorithm for K-class classification. See <a href="http://arxiv.org/pdf/1406.1822">the paper</a>. You can use it even for binary classification (K=2), but not for regression.</li>
<li><code>--bootstrap M</code> M-way bootstrap by online importance resampling. Use <code>--bs_type=vote</code> for classification and <code>--bs_type=mean</code> for regression. Note that this is <a href="http://en.wikipedia.org/wiki/Bootstrap_aggregating">bagging</a>, not boosting.</li>
<li><code>--boosting N</code> (added on 2015-06-17) online boosting with N weak learners, see <a href="http://arxiv.org/abs/1502.02651">a theoretic paper</a></li>
</ul>
Gradient boosting on Vowpal Wabbit
<p>Is there a way to use gradient boosting on regression using Vowpal Wabbit? I use various techniques that come with Vowpal Wabbit that are helpful. I want to try gradient boosting along with that, but I can't find a way to implement gradient boosting on VW. </p>