Forming perceptual groups and individuating objects in visual scenes is an essential step towards visual intelligence. This ability is thought to arise in the brain from computations implemented by bottom-up, horizontal, and top-down connections between neurons. However, the relative contributions of these connections to perceptual grouping are poorly understood. We address this question by systematically evaluating neural network architectures featuring combinations of these connections on two synthetic visual tasks, which stress low-level "Gestalt" vs. high-level object cues for perceptual grouping. We show that increasing the difficulty of either task strains learning for networks that rely solely on bottom-up processing. Horizontal connections resolve this limitation on tasks with Gestalt cues by supporting incremental spatial propagation of activities, whereas top-down connections rescue learning on tasks with high-level object cues by modifying coarse predictions about the position of the target object. Our findings dissociate the computational roles of bottom-up, horizontal and top-down connectivity, and demonstrate how a model featuring all of these interactions can more flexibly learn to form perceptual groups. The ability to form perceptual groups and segment scenes into a discrete set of object-based representations constitutes a fundamental component of visual intelligence. Decades of research in biological vision have suggested a coarse dichotomy between perceptual grouping tasks that can be solved by feedforward (or bottom-up) processes vs. those that require feedback (or recurrent) processes (Roelfsema, 2006; Roelfsema & Houtkamp, 2011; Wyatte et al., 2014) . Feedforward processes group scenes by encoding increasingly more complex feature conjunctions through a cascade of filtering, rectification and normalization operations. As shown in Fig. 1a , this visual strategy can be sufficient to detect and localize objects in scenes with little or no background clutter, or when an object "pops out" because of color, contrast, etc. (Nothdurft, 1991) . However, as illustrated in Fig. 1b , visual scenes are usually complex and contain objects interposed in background clutter. When this is the case, feedforward processes alone are often insufficient for perceptual grouping (Herzog & Clarke, 2014; Pelli et al., 2004; Freeman et al., 2012; Freeman & Pelli, 2010) , and it has been suggested that our visual system leverages feedback mechanisms to refine an initially coarse scene segmentation (Ullman, 1984; Roelfsema & Houtkamp, 2011; Treisman & Gelade, 1980; Lamme & Roelfsema, 2000) . Extant theory suggests that there are two distinct types of feedback strategies. One strategy involves grouping low-level visual features with their neighbors according to Gestalt laws like similarity, good continuation, etc. (Fig. 1b, top; Jolicoeur et al., 1986; 1991; Pringle & Egeth, 1988; Roelfsema et al., 1999; Houtkamp & Roelfsema, 2010; Houtkamp et al., 2003; Roelfsema et al., 2002) . Another strategy is object-based and mediated by high-level expectations about the shape and structure of perceptual objects. In this strategy, feedback refines a coarse initial feedforward analysis of a scene with high-level hypotheses about the objects it contains (Fig. 1b, bottom; Vecera & Farah, 1997; Vecera & O'Reilly, 1998; Vecera, 1993; Zemel et al., 2002) . Both of these feedback strategies are iterative and rely on recurrent computations. What are the neural circuits that implement Gestalt vs. object-based strategies for perceptual grouping? Visual neuroscience studies have suggested that these strategies emerge from specific types of neural interactions: (i) horizontal connections between neurons within an area, spanning spatial locations and potentially feature selectivities (Stettler et al., 2002; Gilbert & Wiesel, 1989; McManus et al., 2011; Bosking et al., 1997; Schmidt et al., 1997; Wannig et al., 2011) , and (ii) descending top-down connections from neurons in higher-to-lower areas (Ko & von der Heydt, 2018; Gilbert & Li, 2013; Tang et al., 2018; Lamme et al., 1998; Murray et al., 2004; . The anatomical and functional properties of these feedback connections have been well-documented (see Gilbert & Li 2013 for a review), but the relative contributions of horizontal vs. top-down connections for perceptual grouping remains an open question. Perceptual grouping is essential for reasoning about the visual world. Although it is known that bottom-up, horizontal and top-down interaction contribute to perceptual grouping, their relative contributions are not well understood. We directly tested a long-held theory related to the role of horizontal vs. top-down connections for perceptual grouping by screening neural network architectures on controlled synthetic visual tasks. Without specifying any role for feedback connections a priori, we found a dissociation between horizontal vs. top-down feedback connections which emerged from training network architectures for classification. Our study provides direct computational evidence for the distinct roles played by these cortical mechanisms. Our study also demonstrates a clear limitation of network models that rely solely on feedforward processing, including ResNets of arbitrary depths, which are strained by perceptual grouping tasks that involve cluttered visual stimuli. Deep ResNets performed better on the Pathfinder challenge, whereas the shallower ResNet-18 performed better on the cABC challenge. reference ::::::::::: feedforward models, and made decisions on Pathfinder and cABC images that were significantly more similar to those of human observers. Our study thus adds to a growing body of literature (George et al., 2017b; Nayebi et al., 2018b; Linsley et al., 2018b; ?; Kar et al., 2019) :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: (George et al., 2017b; Nayebi et al., 2018b; Linsley et al., 2018b; a; Kar et al., 2019) which suggests that recurrent circuits are necessary to explain complex visual recognition processes. We will release our code and datasets upon publication to encourage progress in modeling perceptual grouping in biological vision.
Horizontal and top-down feedback connections are responsible for complementary perceptual grouping strategies in biological and recurrent vision systems. Using neural networks as a computational model of the brain, examines the efficiency of different strategies for solving two visual challenges.