Principles of perceptual grouping: implications for image-guided surgery

Principles of perc eptual grouping : implications for image- guided surgery Birgitta Dresp-Lan gley ICube UMR 7357 CNRS , University o f Strasbourg (France) E-mail: birgitta.dresp @icube.unistra.fr Cite as: Dresp-Langley B. Principles of perceptual grouping: implications for image-guided surgery. Front Psychol. 2015; 6:1565. doi:10.3389/fpsyg.2015.01565 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4611091/ Abstract Gestalt theory has provided perceptual science with a conceptual framework which has inspired researchers ever since, taking the field of perceptual organization into the 21st century. This opinion article discuss es the importance of rules of perceptual organization for the testing and design of visual interface technology. It is argued that major Gestalt principles, such as the law of good continuation or the principle of Prägnanz (suggested translation: salience ), taken as examples here, are important to our understanding of visual image processing by a human observer. Perceptual integration of contrast information across collinear space, and the organization of objects in the 2D image plane into figure and ground are of a particular importance here. Visual interfaces for image- guided surgery illustrate the criticality of these two types of perceptual processes f or reliable decision making and action. It is concluded that Gestalt theory continues to generate powerful concepts and insights for perceptual science placed within the context of major technological challenges of today. Key words: Gestalt theory; Law of good continuation ; Principle of Prägnanz ; Collinear integration; Border ownership; Fig ure-ground; Image-guided surgery 3 The laws and principles which predict how perceptual qualities can be extracted from the most elementary visual si gnals were discovered by Gestalt psychologists (e.g. Metzger, 1930, and Wertheimer, 1923, translated and re-edited by Spillmann in 2009 and 2012, respectively). Their seminal work has inspired visual science ever since, and has led to exciting discoveries which have confirmed the Gestalt idea that the human brain would have an astonishing capacity for selecting and combining critical visual signals to generate output representations for decision making and action. This capacity of selection and integration enables the perception of form and space, and the correct estimation of relative positions, trajectories, and distances of objects represented in planar images. The Gestalt laws and principles relate to the ability of human observers to assess 1) which parts of an image belong together to form a unified visual object or shape, and 2) which parts should be nearer and which further away from the observer if the represented objects were seen in the real world. Two of these, the principle of Pr ägnanz (suggested translation: salience) and the law of good continuation are considered here. The Gestalt principle of Prägnanz governs specific perceptual processes that generate cues to shape and relative distance (figure-ground) in the visual field. These processes use local signals of contrast and orientation to fill in specific regions of an image and thereby enable the perception of surfaces. The associated perceptual sensations of local contrast enhancement make visual objects in the image appear to stand in front of other objects represented in the same plane. Such sensations are often deemed "illusory" because they have no phy sical origin, i.e. the re is no objective difference in local luminance that would explain the resulting percepts (e.g. Heinemann, 1955; Hamada, 1985; O'Shea, Blackburn, and Ono, 1994; DeWeert & Spillmann, 1995; Grossberg, 1997; Dresp & Fischer, 2001; Dresp, Durand, and Grossberg, 2002; Guibal & Dresp, 2004; Devinck, Spillmann and Werner, 2006; Pinna & Reeves, 2006; Dresp- Langley & Durup, 2009; Dresp-Langley & Reeves, 2013, 2014). An essential aspect of this process of figure-ground segregation is the perceptual a ssignment of border ownership (see the review by von der Heydt on this topic). The Gestalt theorist Rubin (1921) was among the first to point out that a figure has distinct perceptual qualities that make it stand out against the rest of the visual field, which there by acquires the perceptual quality of ground (or background). A figure occludes the ground and, therefore, owns the borders which separate it from the latter. Zhou, Friedman, and von der Heydt (2000) found neurons predominantly in V2 (but also V1) of the monkey that respond selectively to the location of borders in the visual field. Selective visua l 4 attention to the figure strengthens the neuronal responses to its borders (Qiu, Sugihar, and von der Heydt, 2007 ). The Gestalt psychologists also correctly presumed that, to recover a representation of a whole from parts, the brain must achieve the perceptual integration of visual information across collinear space (e.g. Wertheimer, 1913; Metzger, 1930). The visual integration of contrast information across collinear image space plays a crucial role in form vision under conditions of stimulus uncertainty a nd configurative ambiguity (e.g. Dresp, 1997; Grossberg, 1997). It is governed by the so -called law of good continuation , and reflected by interactive effects between co-axial stimuli in the visual field. Specific response activities of visual cortical neurons are triggered by these co-axial interactions (cf. the first observations by Nelson & Frost, 1978 and von der Heydt, Peterhans, and Baumgartner, 1984 in monkey visual cortex), revealing the functional properties of brain mechanisms designed to complete physically discontinuous contrast input across collinear visual space. Collinear spatial integration is crucial for the detection of alignment, virtual trajectories, and shape borders in a world where most objects are seen incompletely. It enables a human observer to assess the continuity of image fragments under conditions of diminished visibility and heightened stimulus ambig uity. Experimental data on collinear visual integration have shown that the perceptual recovery of global representations of collinear space involves many levels of visual processing, not a single one, from the visual detection of local image detail to the perception of global association fields (e.g. Dresp, 1993; Field, Hayes, and Hess, 1993; Polat & Sagi, 1993, 1994; Kapadia et al., 1995; Polat & Norcia, 1996 ; Wehrhahn & Dresp, 1998 ; Yu & Le vi, 1997, 2000; Chen, Kasamatsu, Polat, and Norcia, 2001; Chen & Tyler, 2001; Tzvetanov & Dresp, 2002; Dresp & Langley, 2005; Chen & Tyler, 2008; Huang, Chen, and Tyler, 2012; Spillmann, Dresp-Langley, and Tseng, 2015). The laws of perceptual organization formulated by Gestalt theory ha ve not lost any of their significance. They turn out to be as relevant as ever in the context of vi sual interface technology for image-guided surgery, for example. I mage-guided surgery aims to use images taken before and/or during the procedure to help the surgeon navigate. The goal is to augment the surgeon's capacity for decision making and action during the procedure (see Perrin et al., 2009, for review). In augmented reality, the guidance is provided directly on the surgeon's view of the patient by mixing real and virtual images (Figure 1). This includes the visual tracking of devices relative to the patient, registration and alignment of the preoperative model, and the suitable rendering and visualization of 5 the preoperative data. Visualization in this context means translating image data into a graphic representation that is understandable by the user (the surgeon), as it conveys important information for assessing structure and function, and for making (the right!) decisions during an intervention. The field has evolved dramatically in recent years, ye t, the most critical problem for image-guided surgery is still the one of task-centred user interface design. During a surgical intervention, the timing of the ge neration of image data is absolutely critical, and to facilitate navigation through large cavities with multiple potential obstacles, such as within the abdomen, complex displays have been designed to provide navigational aids. They combine surface renderings of anatomy (Figure 1, left) from preoperative imaging with intra-operative visualization techniques. A common strategy here is representing volumetric data as 2D surfaces with vary ing opacity. The efficiency of renderings for fac ilitating decisions of the human user can be e valuated in terms of the perceptual salience of critical surfaces (principle of Pr ägnanz ) that repre sent regions of interest to the surgeon. Moreover, intra-operative imaging often provides further diagnostic information and permits assessing risks as well as perspectives of repair. In this context, image- guided instrument tracking is a major challenge for current research and development in this field (West et al., 2004; Huang et al., 2007). A critical problem for the surgeon is detecting and keeping track of the relative positions of the surgical tools he/she is using during the intervention (Figure 1, right). Visual tracking of the tooltip trajectories is also a precious aid for evaluating skill evolution in trainee surgeons, the positional accuracy of the tooltips being critical during an intervention (e.g. Jiang et al., 2014) . The development and testing of new visual aids to facilitate the detection of alignment, relative position and trajectories (perceptual law of good continuation ) is urgently needed here. Ultimately, technology where the surgical tool itself will become itself a visual navigation aid in image-guided surgery is to be developed in the near future and psychophysical testing should have a major impact on these developments. References Chen, C.C., Kasamatsu, T., Polat, U., and Norcia, A. M. (2001). Contrast response characteristics of long-range lateral interactions in cat striate cortex. Neuroreport , 12, 655-661. Chen, C. C. , and Tyler , C. W. (2001). Lateral sensitivit y modulation explains the flanker effect in contrast discrimination. The Proceedings of the Royal Society (London) Series B , 268, 509-516. Chen, C.C., and Tyler, C.W. (2008). Excitatory and inhibitory interaction fields of 6 flankers revealed by contrast-masking functions. Journal of Vision , 8,10, 1 – 14. Craft, E., Schuetze, H., Niebur, E., and von der Hey dt, R. (2007). A neural model of figure-ground organization. Journal of Neurophysiology , 97, 4310-4326. Devinck, F., Spillmann, L., and Werner, J. S. (2006). Spatial profile of contours inducing long-range color assimilation. Visual Neuroscience , 23, 573 – 577. De Weert, C. M. M. and Spillmann, L. (1995) Assimilation: Asymme try between brightness and darkness. Vision Research , 35: 1413 – 1419. Dresp, B. (1993). Bright lines and edges facilitate the detection of small light targets. Spatial Vision, 7, 213-225. Dresp, B., & Bonnet, C. (1991). Psychophy sical evidence for low-level pro cessing of illusory contours. Vision Research, 10 , 1813 – 1817. Dresp, B. (1997). On ‘illusory’ contours and their functional significance. Current Psychology of Cognition, 16, 489 – 517. Dresp, B. and Fischer, S. (2001). Asymmetrical contrast effects induced by luminance and color configurations. Perception & Psychophysics , 63, 1262 – 1270. Dresp, B., Durand, S. and Grossberg, S. (2002). Depth perception from pairs of overlapping cues in pictorial displays. Spatial Vision , 15, 255 – 276. Dresp, B. and Langley, O.K. (2005). Long-range spatial integration across contrast signs: A probabilistic mechanism? Vision Research, 45, 275 – 284. Dresp-Langley, B., and Durup, J. (2009) A plastic temporal code for conscious state generation in the brain. Neural Plasticity , 1-15. Dresp- La n gley, B. and Reeves, A. (2012). Simultaneous contrast and apparent depth from true colors on grey: Chevreul revisited. Seeing & Perceiving , 25(6), 597 – 618. Dresp-Langley, B. and Reeves, A. (2014). Effects of saturation and contrast polarity on the figure-ground organization of color on gray. Frontiers in Psychology . doi: 10.3389/fpsyg.2014.01136 Field, D.J., Hayes, A., and Hess, R.F. (1993). Contour integration by the human visual system: Evidence for a local "association field". Vision Research, 33 , 173 – 193. Grossberg, S. (1997). C ortical d ynamics of 3- D figure-ground perception of 2-D pictur es. Psychological Review , 104, 618-658. Guibal, C. R. C. and Dresp, B. (2004). Interaction of color and geometric cues in depth perception: When does red mean near, Psychological Research , 10, 167 – 178. Hamada, J. (1985). Asymmetric lightness cancellation in Craik- O’Brien patterns of negative and positive contrast. Biological Cybernetics , 52, 117 – 122. Heinemann, E. G. (1955). Simultaneous brightness induction as a function of inducing and test-field luminance. Journal of Experimental Psychology , 50, 89 – 96. Huang, J., Triedman, J.K., Vasilyev, N.V., Suematsu, Y., Cleveland, R.O., and Dupont, P. E. (2007). Imaging artefacts of medical instruments in ultrasound-guided interventions. Journal of Ultrasound Medecine , 26,1303-22. Huang, P.C., Chen, C.C., and Tyler, C.W. (2012). Collinear facilitation over space and depth. Journal of Vision , 12 , 20, 1-9. Hubel , D.H. and Wiesel, T.N. (1959). Receptive fields of single neurons in the cat’s striate cortex. The Journal of Physiology, 148, 574-591. Hubel, D. H. and Wiesel, T. N. (1968).Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology, 195, 215-243. Jiang X., Zheng, B., and Atkins, M.S. (2014) Video Processing to Locate the Tooltip Position in Surgical Eye-Hand Coordination Tasks. Surgical Innovation . doi: 10.1177/553350614541859. Kapadia, M.K., Ito, M., Gilbert, C.D., and Westheimer, G. (1995). Improvement in visual 7 sensitivity by changes in local context: parallel studies in human observe rs and in V1 of alert monkeys. Neuron, 15, 843 – 856. Kapadia, M.K., Westheimer, G., and Gilbert, C.D. (2000). Spatial contribution of contextual interactions in primary visual cortex and in visual perception. Journal of Neurophysiology, 84, 2048-2062. Metzger, W. (1930) Gesetze des Sehens , English trans. L. Spillmann (2009) Laws of Seeing Cambridge, MA: MIT Press. O’Shea, R. P., Blackburn, S. G. and Ono, H. (1994). Contrast as a depth cue. Vision Research , 34, 1595 – 1604. Peterhans, E. and von der Heydt, R. (1991). Subjective contours-bridging the gap between psychophysics and physiology. Trends in Neurosciences, 14, 112 – 119. Polat, U. and Sagi, D. (1994). The architecture of perceptual spatial interaction. Vision Research , 34, 73 – 78. Polat, U. and Sagi, D. (1993). Lateral interactions between spatial channels: suppression and facilitation revealed by latera l masking experiment s. Vision Research, 33, 993- 999. Polat, U. and Norcia, A. M. (1996). Neurophysiological evidence for contrast dependent long-range facilitation and suppression in human visual cortex. Visi on Research, 36, 2099-2109. Pinna, B. and Reeves, A. (2006). Lighting, backlig hting, and the laws of figurality in the watercolor illusion. Spatial Vision , 19, 341 – 373. Perrin, D. P., Vasilyev, N. V., Novotny, P.,Stoll, J., Howe, R. D., Dupont, P. E., Salog, I. S., and del Nido, P. J. (2009). Image Guided Surgical Interventions. Current Problems in Surgery , 46,730-766. Qiu, F.T., Sugihar, T., and von der Heydt, R. (2007). Figure-ground mechanisms provide structure for selective attention. Nature Neuroscience , 10, 1492-1499. Rubin, E. (1921). Visuell Wahrgenommene Figuren: Studien in psychologischer Analyse . Kopenhagen: Gyldendalske. Spillmann, L., Dresp-Langley, B., andTseng, C.H. (2015) Be y ond the classic receptive field: The effect of contextual stimuli. Journal of Vision , 15, 7. Tzvetanov, T. and Dresp, B. (2002). Short- and long-range effects in line contrast detection. Vision Research, 42, 2493-2498. von der Heydt, R., Peterhans, E., and Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260 – 1262. von der Heydt, R. and Peterhans, E. (1989). Mechanisms of contour perception in monkey visual cortex: I. Lines of pattern discontinuit y. Journal of Neuroscience, 9, 1731 – 1748. Wertheimer, M. (1923) Perceived Motion and Figural Organization , English trans. L. Spillmann, M. Wertheimer, K. W. Watkins, S. Lehar and V. Sarris (2012) Cambridge, MA: MIT Press. West, J.B. and Maurer, C.R. Jr. (2004). Designing optically tracked instruments for image-guided surgery. IEEE Transactions on Medical Imaging , 23, 533-45. Yu, C. and Levi, D. M. (1997). Spatial facilitation predicte d with end-stopped spatial filters. Vision Research, 37, 3117 – 3128. Yu C. and Levi, D. M. (2000). Surround modulation in human vision unmasked by masking experiments. Nature Neuroscience, 3, 724 – 728. Zhang, N.R. and von der Heydt, R. (2010). Analysis of the context integration mechanisms underlying figure-ground organization in the visual cortex. Journal of Neuroscience, 30, 6482-6496. Zhou, H., Friedman, H. S., and von der Heydt, R. (2000). Coding of border ownership in 8 monkey visual cortex. Journal of Neuroscience, 20, 6594 – 6611. Figure caption Figure 1 In image-guided surgery, visual guidance is provided directly on the surgeon's view of the patient's anatomy by mixing real and virtual images. The efficiency of such renderings ( left ) for facilitating surgical decision and action is critically determined by the salience of the rendered surfaces (principle of Prägnanz ) that represent regions of interest to the surgeon. Visual tracking of the tooltip trajectorie s is important for evaluating skill evolution, the positional accuracy of the tooltips being critic al . Technology facilitating the positional accuracy of tool-tip movements ( right ) by generating visual data for relative position, alignment, and trajectory anticipation (perceptual law of good continuation ) is needed. Figure 1