Markov networks for low-level vision

Bill Freeman and Egon Pasztor

We seek a learning-based algorithm that applies to various low-level vision problems. For each problem, we want to find the scene interpretation that best explains image data. For example, we may want to infer the projected velocities (scene) which best explain two consecutive image frames (image). >From synthetic data, we model the statistical relationship between local image and scene regions, and between neighboring scene regions. Given a new image, we then propagate Bayesian beliefs in the Markov network to infer the underlying scene. (In joint work with Y. Weiss, we justify using belief propagation in loopy networks.) This yields an efficient method to infer low-level scene interpretations, which we call VISTA, Vision by Image-Scene Training. We apply this to the "super-resolution" problem (estimating high frequency details from a low-resolution image), showing good results. For the motion estimation problem, we show resolution of the aperture problem and filling-in arising from application of the same probabilistic machinery.

pdf file