What you see is what you get

A post arising from a chance encounter with reference 1, with the chance encounter having been prompted by reference 7. The lead author, Jan Johan Koenderink, is a Dutch physicist and psychologist known for his research on visual perception, computer vision, and geometry. He is interested, inter alia, in the business of how we make tidy and coherent percepts out of the messy and complex stimuli arriving at the eyes.

I see linkage to the world of LWS-N introduced at reference 5, in that I think we are both supposing our percepts to be driven by some cunning but two dimensional arrangement of neurons, at some remove from the various stimuli arriving from the outside, in this case all the various stimuli arriving from all the muscular activity involved in moving our eyes and head around, moving the images around on the retinas in order to get a proper look at the three dimensional objects in the real world. How is this arrangement organised, or in the word of LWS-N, compiled?

Note also, that while we might think of a visual percept as being very much like still photograph, all that stimulation, all that muscular activity involves time and it takes a little time to compile that percept. Perhaps a few hundred milliseconds. I associate to the fact that if there is no movement in the outside world and no muscular activity, then after a few seconds, any image will fade, be it ever so interesting. A fact first demonstrated during the middle of the middle of the last century, with a Russian, Alfred Lukyanovich Yarbus, being one of the pioneers, and with a rather more recent description to be found at reference 7.

The experiment on which this paper reports, involving the participation of 75 or so students, is about the size and shape of the visual field. Not how it is out in the world, but how it is in our minds – where, at the current state of the art, in cannot be inspected and measured in a direct way, in a proper and replicable way with electrical and mechanical instruments. We have to rely on the unreliable reports of the students.

Most of Figure 2

We start with a translucent, white hemisphere about one metre across. We mount the hemisphere at a convenient height on the back of an opaque wall panel and cut a peep-hole in the panel at the centre of the hemisphere. What do we see if we illuminate the hemisphere from behind and put an eye – just one eye – to the peep-hole?

So far, the answer would be nothing much. Some people will see a surface and most people will see a sort of white mist (with a similar trick having been played at the Light Show at the Hayward Gallery, back in 2013. See reference 8). So we help them along by pasting a whole lot of circular, black polka dots, all the same size, all over the interior of the hemisphere, more or less at random. With the completed set-up being illustrated above – bearing in mind that what we have above is not what you would see from the peep-hole, rather what the authors call a ‘orthographic frontal view’. Think of the huge difference between what you see on a three dimensional globe and what you see on a two dimensional, Mercator projection of same. With the result here being that, after a few seconds of a ‘luminous, misty space, with black balls floating about in random configuration’, nearly everybody will now see a surface.

Furthermore, the authors claim that the fact that all the polka dots are circular is a strong cue for what they call ‘global local frontoparallelity’, which I think amounts to saying that what you think you are seeing is a whole lot of polka-dots on a plane surface a little way in front of you. No doubt there is plenty of experimental evidence to this effect.

There is support for the ‘illusion’ of frontoparallelity in the trick with the soapy mirror that Gombrich tells of at reference 3. Stand in front of a soapy mirror and trace the image of your face in it – to be surprised how small that image is – having thought that it would the more or less the same size as the face itself.

Whereas I had thought that the circles would have been a strong cue that the surface was everywhere orthogonal to the line of sight, that is to say a hemisphere, with the point of view at the centre. Otherwise some of the circles would have been flattened, something in the way of the skull in Holbein’s famous picture of the ambassadors in the National Gallery, noticed at reference 4.
Now none of the students were privy to what is behind the peep-hole. There were no patterns, no lines and the surface looked pretty much the same everywhere. So what sort of a cue was going to be given by the polka dots?

Part of Figure 5

So when they had had a good look around, without any of the fixation straight ahead business which is common in experiments of this sort, they were asked to draw the shape of what it was that they saw, in the form of a horizontal section. They were given a large sheet of paper containing a helpful dashed line, along the lines of those above, with the eye at the peep-hole at the bottom and the centre of the field at the top.

The snap above contains about a third of the total, one for each student, tidied up a bit. Now what was surprising to me was that only a small proportion of the students reported anything like the right answer, that is to say a hemisphere, which would appear here as half a grey, circular disc. A quarter of a grey, circular disc, rather flattened, was much more common, which might be described as being half way between the frontoparallelity which was cued and the hemisphere which was the case.

Noting here that the students had been primed to report what they actually saw, not what they thought they ought to see, given their knowledge of eyes and so forth. Not too much interaction between top-down and bottom-up; a tricky point, but one which the authors thought they had covered. With the upshot being that the students were not much good estimating either the angular dimension of their field of view or the curvature of what it was they were looking at. There was lots of variation between one student and another and there were a lot more students who got it very wrong than there were who got it more or less right.

The experimenters were not so surprised, having had the benefit of knowledge about what various thinkers past and present had made of this problem. It seems that some early modern painters knew all about it, not to mention nineteenth century savants. See again, for example, reference 3.

One might worry about the tricky point, but that would not disturb the robust finding that they students did not, in the main, see the hemisphere that was there. A failing that evolution would not much care about, polka dotted plexiglass domes not being that common in either jungle or savanna.

The paper then went on to talk about internal local signs and external local signs, with local sign being a term that had first been introduced in the middle of the nineteenth century by one Rudolf Hermann Lotze, and which had then gone out of use for many years. I do not yet claim to understand what these signs are about, but maybe the internal local signs are what puts a bit of intelligence into array of neurons which is generating the subjective visual field, something in the way of what is being attempted with LWS-N, while the external local signs are all the tricky, mostly small, movements which the eye makes in order to make sense of things in the world outside. It is from the way that the stimuli on the retina change with these movements that the brain gets knowledge about objects, their shapes and their positions. New knowledge which can then be merged with stored knowledge about the world and its ways. Maybe those movements are used to code this knowledge?

I think what the paper is telling us, inter alia, is that there is, often but not always, a deficiency of internal local signs, with the result that most of us make a bit of a mess of interpreting what we see through the peep hole. What your eyes see is very often not what your brain projects into consciousness.

But I shall persevere with local signs for a bit longer. I may well have got hold of the wrong end of the stick.

PS: perhaps we should try out one of those robots with eyes and see what it makes of the hemisphere? What does computer vision do with tricks of this sort?

References

Reference 1: Wide distribution of external local sign in the normal population - Jan J. Koenderink,  Andrea J. van Doorn, James T. Todd – 2009.

Reference 2: http://gestaltrevision.be/en/. With gestalt being, in this context, a word capturing the way that the brain imposes order on chaos, the way that the brain works hard to see something sensible out there. With Koenderink being on the staff.

Reference 3: Art and Illusion – Gombrich, E. – 1960. See for example, chapter VIII in Part Three, ‘Ambiguities of the Third Dimension’. There are peep-holes to be found there.

Reference 4: http://psmv2.blogspot.com/2014/06/jigsaw-8-series-3.html.

Reference 5: http://psmv3.blogspot.com/2018/01/an-introduction-to-lws-n.html.

Reference 6: The fading of stabilized images: Eye movements and information processing - Stanley Coren, Clare Porac – 1974.

Reference 7: The Mind Is Flat: The Illusion of Mental Depth and the Improvised Mind - Nick Chater – 2018.

Reference 8: https://www.southbankcentre.co.uk/blog/light-show. An exhibition which was noticed at reference 9 following.

Reference 9: http://psmv2.blogspot.com/2013/02/an-outing-in-three-parts.html.