---
abstract: 'Non-convex optimization is ubiquitous in machine learning. The Majorization-Minimization (MM) procedure systematically optimizes non-convex functions through an iterative construction and optimization of upper bounds on the objective function. The bound at each iteration is required to *touch* the objective function at the optimizer of the previous bound. We show that this touching constraint is unnecessary and overly restrictive. We generalize MM by relaxing this constraint, and propose a new optimization framework, named Generalized Majorization-Minimization ([G-MM]{}) that is more flexible compared to MM. For instance, it can incorporate application-specific biases into the optimization procedure without changing the objective function. We derive [G-MM]{}algorithms for several latent variable models and show empirically that they consistently outperform their MM counterparts in optimizing non-convex objectives. In particular, [G-MM]{}algorithms appear to be less sensitive to initialization.'
author:
- |
    Sobhan Naderi Parizi sobhan@alumni.brown.edu\
    School of Engineering, Brown University\
    Providence, RI 02912, USA Kun He hekun@bu.edu\
    Dept. of Computer Science, Boston University\
    Boston, MA 02215, USA Stan Sclaroff sclaroff@bu.edu\
    Dept. of Computer Science, Boston University\
    Boston, MA 02215, USA Pedro Felzenszwalb pff@brown.edu\
    School of Engineering, Brown University\
    Providence, RI 02912, USA
bibliography:
- 'gmm\_bibfile.bib'
title: 'Generalized Majorization-Minimization'
---

majorization-minimization, non-convex optimization, latent variable models, expectation 