Back to Home

Lightness and brightness processing in visual cortex
Cortical mechanisms of perceived angular size in 3D context
Perception of Surface Color and Albedo in 3D Complex Scenes
Surface Material Perception
Shape Perception from Shading Information in 3D
Binocular Stereo Vision

List of publications   

  •  Include mktree.js in your page
  •  Include mktree.css in your page
  •  Set CLASS="mktree" in your <UL>
  • Controlling The Tree
    Custom Classes
    Lightness and Brightness processing in visual cortex

    Luminance is a photometric quantity that represents the amount of light emitted by a surface. Brightness (that very much confused and misused term!), on the other hand, is the apparent level of light emitted by a surface and depends not only on the actual luminance but also on the context in which the surface is viewed. Many compelling visual illusions demonstrate the "failure" of visual system to accurately compare the raw luminance values, like the well known Craik-O'Brien-Cornsweet stimulus shown below.

    In this rectangular version of classical Craik-O'Brien-Cornsweet stimulus, the left flank appears to be darker than the right one, although they actually have the same luminance value. The brightnesses effect is caused by the central region composed of two opposite sign luminance gradients and a contrast border between them (try covering the central portion with a piece of paper to convince yourself that the flanks are actually equiluminant.)


    Craik-O'Brien-Cornsweet stimulus is a fairly simple stimulus, in more complex scenes perceptual grouping can also influence surface brightness. Amodal completion - when two spatially separated surfaces appear to be grouped together behind an occluder - is one of the mechanisms which can facilitate perceptual grouping, and indeed it influences the brightness as demonstrated in a stimulus generated by us, shown in figure below.Click here for a demo experiment (Java applet.)

    Cross fuse the left pair of images, or uncross fuse the right pair. You should see in 3D a rectangular surface behind two vertical bars. Although spatially sperated, the surface appears amodally completed. Just like in the Craik-O'Brien-Cornsweet stimulus, equiluminant flanks appear to have different brightnesses. Note, however, that the neuronal mechanisms underlying the two effects are likely to be different. (Click on the image for a larger version.)


    Using dynamic versions of these stimuli, and carefully chosen control stimuli, Fang, Murray, Kersten and I studied the fMRI activity in early retinotopical visual cortex in regions corresponding to the locations where actual luminance remained constant but brightness varied. We found that the activity in retinotopic cortex, including primary visual cortex (V1), is closely correlated with perceived luminance rather than the actual luminance. This result provides strong evidence that V1 plays a critical role in stimulus driven brightness processing (Boyaci, Fang, Murray, and Kersten, 2007).

    The exact nature of V1's involvement in this process is yet to be discovered. For example, we do not yet know whether the activity is a result of feedback mechanisms or lateral interactions within V1. FMRI experiments sensitive to temporal dynamics of the cortical activity and covering multiple cortical areas are good candidates to provide answers to these questions. Such experiments are currently being planned and conducted in our lab. Two primary purposes of this project are developing neuro-physiologically plausible computational models of brightness processing, and understanding the principles of perceptual grouping.

      back to Home     Back to Top


    Cortical mechanisms of perceived angular size in 3D context

    Perceived size of an object depends not only on its projected size on the retina, but also on its perceived distance to the observer and on the context in which it is viewed. One of the most fascinating example of this phenomenon is the moon illusion: the moon looks much larger when it is near horizon compared to when it is high in the sky despite the fact that the angle it subtends does not differ. (Check this fragment from the TV show "The Simpsons", where Homer mistakes a mini helicopter in their bedroom for a giant one in the distance - you may need xvid codec and/or mplayer to play this).

    Murray, Kersten and I studied the neural correlates of a similar size illusion in computer rendered scenes. We presented spherical objects with identical image sizes in a hallway either in "near" or "far" position. We found, consistent with previous literature, that the perceived image size of the "far" object was substantially larger than the "near" object, even though their actual image sizes were identical. Click here (large, 7MB) or here (smaller, 1.5MB) for a demo of the behavioral experiment (Java applet).


    Although the image sizes are the same, the front ball looks smaller than the back ball (click on the image for a larger version).


    Using functional magnetic resonance imaging (fMRI) we found that retinotopic mapping of visual space in the primary visual cortex (V1) reflects the perceived rather than the actual visual angle of the objects. The retinotopic mapping in early visual cortex was assumed to have a static relation with the visual angle subtended on the retina, however our findings demonstrate a previously unknown correspondence between perceived size and retinotopic mapping (Murray, Boyaci, and Kersten, 2006).

      back to Home     Back to Top


    Perception of Surface Color and Albedo in 3D Complex Scenes

    The amount and chromaticity of light reflected from a surface depends on the properties of the light sources illuminating it, as well as its intrinsic reflectance properties. They also depend on geometry, such as the orientation of the surface with respect to the light sources. Therefore, the light reflected from the same surface may differ drastically under different conditions. The only property that remains invariant across varying conditions is the reflectance of the surface, therefore one can argue that estimating the reflectance is ecologically more relevant for recognizing objects. But this is not such an easy task. Because given just one input, namely the intensity profile on the retina, the visual system has to somehow solve for more than one unknown, a task that seems to be particulary tough on us when the illumination is atypical (remember buying a gray pair of trousers from a "nicely" illuminated hip clothing store, only to see them dark green at home).

    Then, how well are we capable of taking into account the varying conditions in our surface reflectance estimates, particularly the 3D geometry? In one experiment (Boyaci, Maloney and Hersh, 2003), we investigated this. Observers judged the lightness (perceived achromatic surface reflectance) and the orientation of a test patch placed in the middle of a rendered 3D scene illuminated by neutral punctate and diffuse light sources. The scenes were rendered using computer-graphics methods and presented to the observer in a large, computer controlled Wheatstone stereoscope. In each trial, the test patch was presented in a different orientation. The accompanying figure shows a single scene from this experiment as a stereogram, with separate images for the left and right eye (uncrossed fusion). The results of this experiment indicate that observers in our experiments substantially compensate for changes in test patch orientation and that their settings were consistent with the geometry of the scene. Moreover, we have found that, to arrive at their estimates, observers use the three candidate cues, shading, cast shadows, and specular highlights present in the scenes (Boyaci, Doerschner and Maloney, 2006).


    A stereo pair from the experiment (un-crossed fusion). Click on the image for an animated-gif movie.


    We have also examined the perception of surface color when the punctate and diffuse lights were different chromaticity (Boyaci, Doerschner, and Maloney, 2004) and in scenes where large chromatic surfaces served as effective secondary light sources (Doerschner, Boyaci, and Maloney, 2004). The accompanying figure illustrates the latter. (Click on the image for an animated-gif movie).


    The picture that emerges from these experiments is that of a visual system that bases estimates of color and albedo on algorithms that incorporate physically-correct models of how light flows in natural environments. However, the exact nature of the process still remains largely unknown and requires further behavioral and neurophysiological experimentation.

      Back to Home     Back to Top


    Surface Material Perception

    The amazing simplicity of our ability to tell matte from shiny, wet from dry or slippery from non-slippery by just looking at things shouldn't fool one to think that these are computationally simple problems. As a matter of fact, we still don't have any idea how to model our perception of surface material properties. Moreover, we don't know what regions in the cortex play a role in the process, we don't even know where exactly to look at in the brain yet.

    The problem is not very different from the lightness or color perception: The light reflected, say, from a glossy surface depends on the reflectance properties of that surface as well as the nature and spatial distribution of light sources in the scene. Therefore the visual system faces the difficult problem of estimating two things given just a single image. To start with, K. Doerschner, L.T. Maloney and I investigated how the percept of glossiness varies with the illuminant. We asked observers to compare the glossiness of pairs of surfaces rendered under two different real-world illuminants (DIs) (Debevec, 1998). We used this data to estimate the transfer function that captures how perceived glossiness is remapped in changing from one DI to a second. The results suggest that the transfer function is linear and the transfer function from DI1 to DI2 and the transfer function from DI2 to DI3 could be used to predict the transfer function from DI1 to DI3. (Doerschner, Boyaci, and Maloney, 2004. See the poster). Other aspects of material perception are currently under investigation in our lab, as well as in many other labs globally.


       


    Obviously, the visual system combines prior knowledge and likely causes of an image when estimating surface properties. In fact you can cheat your visual system to mistake a matte ping-pong ball for a glossy one. Paint the ball with black matte spray paint and then put a tiny white dot on its surface. Suspend the ball in the air and shine a bright spot light on it in an otherwise dark room. Now step away 3-4 meters, look at the ball with the position of white dot roughly corresponding to where the highlight on an actual shiny ball would be, does the ball look matte or shiny?

      back to Home     Back to Top


    Shape Perception from Shading Information in 3D

    How do we estimate shapes from shading information in 3D? The shading on the surface of an object depends on the illumination and the surface reflectance properties as much as it depends on the shape of the object. So, how does the visual system solve for all those unknowns given a single image (or two in stereo vision) and estimate the shape of an object? Unlike in 2D, where shape from shading is "ill-posed", in 3D an additional stereoscopic depth cue may help disambiguate the problem and contribute to the percept of shape of a matte object. This depth cue emerges based on the disparity information, which derives from matching the luminance values of each image pair.

    Do the human observers use and combine monocular and stereo shading cues to estimate shapes? To find out, we presented computer rendered ellipsoids (like shown in figure below) to the observers and asked them to judge the surface orientation at certain predetermined positions in both binocular and monocular vision. The same procedure was repeated after a nonlinear transformation of the image luminance map (compare right and left images). Such a nonlinear transform leaves the stereo cue unchanged, but alters the monocular cue. The effect of this perturbation on observers' perception reveals the way the two cues are combined by the visual system. Findings of this study suggest that the observers make use of both cues when the stereo information is available. (Boyaci and Maloney, 2002).


       


      Back to Home     Back to Top


    Binocular Stereo Vision

    From the viewpoint of a computer scientist correspondence problem, the problem of finding out which two points on two retinas correspond to the same spatial location in the visual field, is a hard one. It is a hard one because there is still no algorithm that can perform as well as humans, or that can predict failures of humans. But this doesn't mean that it is such a hard problem for the visual system, it may not be a computationally more demanding process compared to shape from shading for example. Part of the difficulty that we face may lie in the choice of the coordinate system used. Davi Geiger and I worked on developing a novel, cellular automata type computer algorithm for solving the correspondence problem of stereo vision in the so called cyclopean coordinate system. The method replaces the formerly accepted constraints of ordering, uniqueness and smoothness with more natural conditions. Many phenomena in human stereo vision, such as subjective contours and half occlusions follow naturally. Moreover, the use of the cyclopean system introduces reduction in array sizes and increase in speed. Recent implementation of the algorithm could handle images which have illusory disparity steps such as the Kanizsa square shown in figure below (crossed fusion left pair, uncrossed fusion right pair).


         


      Back to Home     Back to Top


    Publications

    Fang, F., Boyaci, H., Kersten, D. (2009). Border ownership selectivity in human early visual cortex and its modulation by attention. Journal of Neuroscience, 29 (2), 460-465. doi:10.1523/JNEUROSCI.4628-08.2009 pdf

    Fang, F., Boyaci, H., Kersten, D., Murray, S.O. (2008). Attention-dependent representation of a size illusion in human V1. Current Biology, 18 (21), 1707-1712. doi:10.1016/j.cub.2008.09.025 pdf

    Doerschner, K., Boyaci, H., and Maloney, L.T. (2007). Testing limits on matte surface color perception in three-dimensional scenes with complex light fields. Vision Research, 47, 3409-3423.
    doi:10.1016/j.visres.2007.09.020 pdf

    Boyaci, H., Fang, F., Murray, S.O., Kersten, D. (2007). Responses to lightness variations in early human visual cortex. Current Biology, 17, 989-993. 10.1016/j.cub.2007.05.005 pdf

    Boyaci, H., Doerschner, K., Snyder, J., and Maloney, L.T. (2006). Surface Color Perception in Three-Dimensional Scenes. Visual Neuroscience, 23, 311-321. doi:10.1017/S0952523806233431 pdf

    Murray, S.O., Boyaci, H., and Kersten, D. (2006). The representation of perceived angular size in human primary visual cortex. Nature Neuroscience, 9, 429-434. doi:10.1038/nn1641 pdf

    Boyaci, H., Doerschner, K., and Maloney, L. T. (2006). Cues to an equivalent lighting model. Journal of Vision, 6, 106-118. doi:10.1167/6.2.2 pdf

    Maloney, L.T., Boyaci, H., Doerschner, K. (2005). Surface color perception as an inverse problem in biological vision. Computational Imaging III, edited by Charles A. Bouman, Eric L. Miller, Proceedings of SPIE-IS&T Electronic Imaging, SPIE Vol. 5674. doi:10.1117/12.598463 pdf

    Boyaci, H., Doerschner, K., and Maloney, L. T. (2004). Perceived surface color in binocularly-viewed scenes with two light sources differing in chromaticity. Journal of Vision, 4, 664-679. doi:10.1167/4.9.1 pdf

    Doerschner, K., Boyaci, H., and Maloney, L. T. (2004). Human observers compensate for secondary illumination originating in nearby chromatic surfaces. Journal of Vision, 4, 92-105. doi:10.1167/4.2.3 pdf

    Boyaci, H., Maloney, L. T., and Hersh, S. (2003). The effect of perceived surface orientation on perceived surface albedo in binocularly viewed scenes. Journal of Vision, 3, 541-553. doi:10.1167/3.8.2 pdf

      Back to Home     Back to Top


    Conference Abstracts

    Boyaci, H., Fang, F., Murray, S.O., Albanese, G., Kersten, D. (2008). Time course of cortical responses to illusory and real lightness changes [Abstract]. Journal of Vision, 8(6):906, 960a. The Annual Meeting of the Vision Science Society, Naples, Florida, May 2008. (slides)

    Fang, F., Boyaci, H., Kersten, D. (2008). Border ownership representation in human early visual cortex and its modulation by attention [Abstract]. Journal of Vision, 8(6):393, 393a. The Annual Meeting of the Vision Science Society, Naples, Florida, May 2008.

    Olman, C., Boyaci, H., Fang, F., Doerschner, K. (2008). V1 responses to different types of luminance histogram contrast [Abstract]. Journal of Vision, 8(6):345, 345a. The Annual Meeting of the Vision Science Society, Naples, Florida, May 2008.

    Boyaci, H., Akarun, L. (2008). Bilgisayar ve insanda gorme - Color vision in humans and computers. IEEE 16. Sinyal Isleme, Iletisim ve Uygulamalari Kurultayi, April 20-22, Didim Turkey.

    Aksoy, S., Boyaci, H., Gokcay, D. (2008). Nesne tanimada baglam ve anlambilimsel siniflandirmanin onemi: Bilgisayarla gorme ve insanda gorme alanlarindaki calismalar - The importance of context and semantic descriptions in object recognition: Studies in computer vision and human vision. IEEE 16. Sinyal Isleme, Iletisim ve Uygulamalari Kurultayi, April 20-22, Didim Turkey.

    Boyaci, H., Fang, F., Murray, S.O., Kersten, D. (2007). Amodal completion affects lightness perception. The Annual Meeting of the Vision Science Society, Sarasota, Florida, May 2007.(poster)
    Abstract: Journal of Vision, 7, 235a (2007).
    http://journalofvision.org/7/9/235/

    Murray, S.O., Fang, F., Boyaci, H., Kersten, D. (2007). Perceived eccentricity difference is reflected by shifts in the spatial profiles of human V1 activity. The Annual Meeting of the Vision Science Society, Sarasota, Florida, May 2007.
    Abstract: Journal of Vision, 7, 832a (2007).
    http://journalofvision.org/7/9/832/

    Boyaci, H., Fang, F., Kersten, D.J., and Murray, S.O. (2008). Human cortical responses to illusory and actual luminance variations. The Annual Meeting of the Vision Science Society, Sarasota, Florida, May 2006. (poster)
    Abstract: Journal of Vision, 6(6), 706a (2006),
    http://journalofvision.org/6/6/706/

    Maloney, L. T., Boyaci, H., Doerschner, K., and Snyder, J. L. (2006). Surface color perception in three-dimensional scenes,
    Color Vision Symposium, Tagung experimentell arbeitender Psychologen: TeaP2006, Mainz, Germany, March, 2006.

    Doerschner, K., Boyaci, H., Maloney, L. T., (2005). Spherical harmonic representation of illumination in complex, 3D scenes,
    European Conference on Visual Perception, A Coruna, Spain, August 2005.

    Murray, S.O., Boyaci, H., Kersten, D.J., (2005). The emergence of object size invariance in the human visual cortex. The Annual Meeting of the Vision Sciences Society, Sarasota, Florida, May 2005.
    Abstract: Journal of Vision, 5(8), 744a (2005),
    http://www.journalofvision.org/5/8/744/

    Boyaci, H, Doerschner, K., and Maloney, L. T., (2005). Observers discount effects of multiple chromatic light sources when estimating surface color. 10th Congress of the International Colour Association, Granada, Spain, 8-13 May 2005.

    Maloney, L. T., Doerschner, K., Boyaci, H., (2005). Surface color perception in three-dimensional scenes with non-uniform spatial and spectral distribution of illumination,
    Computational and Systems Neuroscience, Salt Lake City, Utah, March 2005.

    Doerschner, K., Boyaci, H., Maloney, L. T., (2005). Representing the spatial and chromatic distribution of the illuminant in scenes with multiple punctate chromatic light sources. The Annual Meeting of the Vision Sciences Society, Sarasota, Florida, May 2005.
    Abstract: Journal of Vision, 5(8), 785a (2005). http://www.journalofvision.org/5/8/785/

    Maloney, L. T., Boyaci, H., Doerschner, K., (2005). Representing Spatially and Chromatically Varying Illumination Using Spherical Harmonics in Human Vision. The Annual Meeting of the Vision Sciences Society, Sarasota, Florida, May 2005.
    Abstract: Journal of Vision, 5(8), 786a (2005). http://www.journalofvision.org/5/8/786/

    Maloney, L. T., Boyaci, H. and Doerschner, K. (2005). Surface color perception as an inverse problem in biological vision,
    Keynote address, IS&T/SPIE Conferenceon Electronic Imaging; Science and Technology, San Jose, California, January, 2005.

    Boyaci, H., and Maloney, L. T. (2004),
    The effect of an illumination direction cue based on cast shadows on lightness perception in three-dimensional scenes .
    Paper presented at the Annual Meeting of the Vision Sciences Society, Sarasota, Florida, May 2004. (slides)
    Abstract: Journal of Vision 4(8), 121a (2004).
    http://www.journalofvision.org/4/8/121/

    Doerschner, K., Boyaci, H., and Maloney, L. T. (2004),
    Estimating the glossiness transfer function induced by changing illumination and testing its transitivity.
    Paper presented at the Annual Meeting of the Vision Sciences Society, Sarasota, Florida, May 2004. (poster)
    Abstract: Journal of Vision 4(8), 322a (2004).
    http://www.journalofvision.org/4/8/322/

    Doerschner, K., Boyaci, H., and Maloney, L. T. (2003),
    Human observers compensate for secondary illumination originating in nearby chromatic surfaces in making achromatic settings.
    Paper presented at the Annual Meeting of the Vision Sciences Society, Sarasota, Florida, May 2003. Abstract: Journal of Vision, 3(9), 450a (2003),
    http://journalofvision.org/3/9/450/

    Boyaci, H., Doerschner, K., and Maloney, L. T. (2003),
    Achromatic settings in three-dimensional scenes with two light sources differing in chromaticity.
    Paper presented at the Annual Meeting of the Vision Sciences Society, Sarasota, Florida, May 2003. Abstract: Journal of Vision, 3(9), 451a (2003),
    http://journalofvision.org/3/9/451/

    Maloney, L. T., and Boyaci, H. (2002),
    Observers correct perceived albedo for perceived orientation when stereo disparity cues are available.
    European Conference on Visual Perception, Glasgow, Scotland, August 2002. Abstract: Perception Volume 31 Supplement, page 5-6 (2002).
    http://www.perceptionweb.com/perception/ecvp02/0392.html

    Boyaci, H. and Maloney, L. T. (2002),
    Binocular perception of shape from shading/contour is invariant under ordinal transformations of image intensities.
    Paper presented at the Annual Meeting of the Vision Sciences Society, Sarasota, Florida, May 2002. Abstract: Journal of Vision, 2(7), 309a (2002),
    http://journalofvision.org/2/7/309/

    Maloney, L. T., Boyaci, H., and Hersh, S. (2002),
    Human observers do not correct perceived lightness for perceived orientation.
    Paper presented at the Annual Meeting of the Vision Sciences Society, Sarasota, Florida, May 2002. Abstract: Journal of Vision, 2(7), 554a (2002),
    http://journalofvision.org/2/7/554/

    Boyaci,H., Maloney, L. T. (2001), Effect of complexity and ordinal transformations of luminance on binocular shape perception.
    Paper presented at the 24th European Conference on Visual Perception Kusadasi, Turkey 26-30 August 2001. Abstract: Perception Volume 30 Supplement, page 21 (2001).
    http://www.perceptionweb.com/ecvp01/0038.html

    H. Boyaci, Laurence T. Maloney (2001),
    Patterned failures of shape invariance under rotation.
    Paper presented at the Annual Meeting of the Association for Research in Vision and Ophthalmology, Fort Lauderdale, Florida April 29-May 4, 2001. Abstract: IOVS Volume 42, page S868 (2001).

      Back to Home     Back to Top