Shopping lists

A look at the presently elusive purpose of human consciousness in the light of building shopping lists.

In what follows there will be reference to LWS-N, the local or layered workspace, neural version, material about which can be found at reference 4.

Let us suppose that we go shopping once a week, say Monday morning.

Let us suppose we have a groceries cupboard.

Let us suppose that all the groceries come in small, distinctive boxes. Maybe there are few enough different sorts of grocery that the boxes can be colour coded.

The idea is that, the regular weekly shop completed, there should be a suitable number of boxes of all the groceries, where this number reflects the average consumption during the weeks past and, perhaps, any plans we might having for the weeks coming. We might have a menu for the coming week, from which one derives the shopping requirement. Perhaps allowing a margin for error, loss or change.

So on Sunday evening we peer into the groceries cupboard. We take in the then current stock of all the different groceries and compute the list. This will take a little time with the eyes moving around the cupboard to take in the stocks of all those different groceries.

This task will be harder as our cupboard is muddled up. If we keep all the different groceries in nice tidy heaps, one heap for each sort of grocery, there is much less to compute. Beyond perhaps raising an interrupt in the case that we come across a second heap of corn flakes, when there should only be the one heap – a task which requires memory, short term or otherwise.

Figure 1

On Monday morning, we drive down to Sainsbury’s Kiln Lane and execute that list, snapped by Google above and with the all important trolley park taking pride of place (on which see reference 6). The memory of this list needs to be long term, at least relative to the short term just mentioned. The list needs to be retained for 24 hours or so, rather than the 24 seconds or so needed to scan the cupboard. Maybe best to write it down.

A robot could have a fair stab at all of this without bothering with consciousness, while we probably could not. So what are the relevant differences?

Breaking the task down

At simplest, we loop through all the groceries, make a shopping decision for each one, committing groceries to the list accordingly.

So how do we loop through the groceries? If we can rely on the cupboard being tidy, we can just scan the cupboard in some orderly way. Maybe the cupboard can be assumed to be a shelf and all we have to do is to scan the shelf from left to right, pausing at each heap in turn? A task which requires the ability to maintain a pointer (sometimes called a cursor) so that we keep our place on the shelf.

It may be necessary to call on memory to check for items which are not just low but missing. Not enough just to look for the small heaps. Unless we know, for example, that every square marked on the floor of the cupboard should be occupied by a heap. And then we would need to know what should occupy that square. Perhaps a label or a colour code in the square?

There is a certain amount of error checking going on. We have a good idea what the stock was and what the consumption has been. So some part of the brain is checking current stock against what it knows of the past. For which purpose, just gazing rather blankly at the heap, but holding that heap in mind, in attention, gives the brain the space it needs for these background tasks. Space which would be denied by carrying on a conversation at the same time (see below). We associate to global workspace theories of consciousness which are partly built around requirements of this sort.

And looking ahead to the work to be done at Kiln Lane, it might also be that the shop is organised in the same way, making things easier still. And if one had a good memory for images, in particular for the image of the grocery cupboard from the evening before, there might be no need for a list at all.

But suppose for the present that we have, or have already computed, the desired stock level for each item of grocery that should be in our grocery cupboard. So all we need to do is compare the desired stock with the actual stock, and where the one is greater than the other, write the name of that item and an amount to buy to the list.

With practise, one can probably do this more or less unconsciously. One needs to be awake, but one can write the list without paying either it or the cupboard much conscious attention; one could be listening to the radio or carrying on a conversation of sorts at the same time.

This might not work so well with a child, with children often having fairly short attention spans and minds that are apt to wander if there are distractions, as there often are. The child needs to be told to attend to the task in hand, that is to say to the cupboard and to the list. Maybe some of this is unconscious, having been drummed into the child by appropriate punishment.

But while we are learning, while we still need to pay attention, is the best way to proceed to empty the conscious mind of thought, for there to be no inner thought but plenty of attention? One just looks at each heap in turn and writes something down, if appropriate, on the list. Or would one allow a bit of conscious inner thought about the number of boxes in the heap in question before actually committing pen to paper?

We suspect that for most people, writing the list down is necessary, even though they may well remember chunks of it anyway, even without trying to commit it to memory. That said, some people find that writing the list down is a useful aid to memory formation, just as some find that reciting it out loud is a useful to aid to memory formation.

Which suggests that a  housewife who can neither read nor write might have difficulty. Lists may not have been important while we were in the jungle, but they would have become more important by the time that we had people living in towns and life had become more complicated – a time which would have predated universal literacy.

Would she have made a mental list by attaching groceries and amounts to the successive numbers, starting at one? And doing the shopping by ticking off the list in a strictly serial, starting at the beginning and working forwards fashion? But this is a topic for another day and we turn now to our literate housewife.

One scenario

Figure 2

We suppose we have an open, roughly eye-level cupboard along the lines illustrated above, with a number of colour coded pigeon holes, so arranged that we can tell at a glance how many boxes – perhaps shaped like the boxes that tubes of toothpaste come in and stacked up opening end forward – there are in each pigeon hole, perhaps along the lines shown for indigo, third from the right on the top row.

We suppose that the long term goal, after we have been shopping, is to have between nine and twelve boxes in each of the pigeon holes, which gives us a certain amount of latitude, a certain amount of choice. The short term goal is to inspect the cupboard and to prepare a written shopping list. This is the task with which we are concerned in what follows.

We do not worry here about most people’s difficulty with naming more than a handful of colours, related to the difficulties noticed at reference 1.

The provision of choice could make the task easier, in that the person concerned could just skip quickly over the pigeon holes with nine or more boxes already present, a conveniently whole number of pigeon hole rows. Or not, if they liked to exercise some choice. For now we suppose that our person is into speed rather than choice. So we also suppose that where topping up is needed, our person chooses to top right up, rather than to choose some level between nine and twelve. An alternative, possibly more economical, rule might be to always buy some multiple of three. A complication we do not go into, might be the need to constrain the total number of boxes to be bought on the list. Either, perhaps, because of shortage of funds or because of shortage of space in the shopping basket.

One strategy would be to first quickly check the whole cupboard to work out whether or not there was some shopping to be done, and then, on the assumption that there was, to work more carefully across the top row, from left to right, adding items to the shopping list as appropriate, then the middle row and, lastly, the bottom row.

Which would go something like opening scan, scan, write, scan, write, scan etc. Where second and subsequent scans would only stop at those pigeon holes which were low, below nine boxes.
Which does assume that one could keep one’s place between successive scans.

Noting here that a random scan, that is to say a scan in which the conscious experience would seem random, would remove the need to keep one’s place but would probably be slower and the result would be much less reliable. One would be much more likely to miss things. Where by ‘seem random’ I mean that the unconscious might well be up to something behind the scenes, that there might be some organising principle to the seemingly random movement of the eyes around the cupboard.

Both eyes and hands would be reasonably fully occupied during such a scan, just about enough action left in them to smoke at the same time, or perhaps to hold a conversation of sorts, but nothing too testing.

Now, a fairly bottom of the range robot could manage this task, with no need for anything remotely like consciousness. So what is consciousness bringing to the party?

High level scan

To which end, we turn to the business of scanning in more detail, sketched in Figure 2 below.

Figure 3

Figure 3 takes the form of the sort of diagram used by computer programmers and while we do not claim that this is how the average brain does things, the brain does have to do these things, in one way or another, if it is to build a shopping list. With full knowledge, one must be able to map from what it actually does to this diagram.

Let us suppose we have reached the middle of the diagram, the box getting the state of the current pigeon hole. That is to say to count the number of boxes in the current pigeon hole. We then need to recover or otherwise retrieve the shopping rule, the refresh condition, and apply it to that state, implying at least two items in working memory, with the result being a small integer. If that number is zero, we simply go to the next pigeon hole and carry on. Otherwise, we combine the number with the name of the pigeon hole, that is to say its colour, into an item for the shopping list and append it to that list, coloured yellow in the figure because we are leaving the world of the head and sending the complicated instructions out to the eyes, the body and the limbs needed to actually write the instruction down. Again implying at least two items in working memory, the name of a colour and a number of boxes to be bought.

There is also a sort of pause, with the subjective experience being mainly the suspended image of the current pigeon hole. A pause during which the unconscious scurries around checking that the state of that pigeon hole is something reasonably plausible, what might be expected given our prior knowledge and expectations. A frame of consciousness in the parlance of LWS-N. In computer speak, the dominant process suspends itself for a few hundred milliseconds, giving subordinate processes a chance to do their stuff, raising an interrupt with the dominant process, should that be necessary.

A further item in working memory is some kind of marker or label for the current pigeon hole, it not being enough to keep the eyes on the right place in the cupboard because the eyes are needed to help with writing the shopping list. There has to be something in working memory – perhaps amounting to row and column number – to tell them where to go back to, a something which might need to be there for some small number of seconds, say two or three.

Sometimes during such a scan, one pauses, drifts off somewhere else. Then pause again, realise one has drifted but is able to recover the place from which one drifted. Sometimes one loses one’s place and cannot so recover; presumably the relevant bits of working memory have been recycled.

Low level scan

Figure 4

Sometimes, particularly with very young people, or with people who are not good at counting, it is not enough just to gaze at a pigeon hole to be able to bring the count into consciousness.

In such cases a more conscious effort is needed. The task of counting the boxes in a pigeon hole needs to be broken down, perhaps into a low level scan of the heap of boxes. An orderly scan from left to right, from top to bottom. A scan which requires the ability to fix on a box, to increment a count and then to move the fix, the cursor, along a bit. Or perhaps to go back to the beginning of the row above.

All of which is apt to give us another diagram along the lines of Figure 3 above.

Alternatively, in this particular case, the brain might simply memorise the count corresponding to each shape of heap, only resorting to an actual count in extremis. Providing we have a tidy housekeeper, the number of shapes will be quite small – and perhaps the odd untidy heap will be the occasion for a bit of counting.

Usually, with a bit of practise, this counting of the boxes in a pigeon hole can slip into the unconscious, and one can rely on the right number popping up, perhaps just dipping down into the detail of the process from time to time, when a gear slips. Or something.

Another scenario

This might be like the first scenario, but making a mental rather than a written list. We believe this to be a more testing task – but do not go further into it here.

The function of consciousness in all this?

So on the one hand it is clear that an unconscious person could not make a shopping list and reasonably clear that most people could not make a shopping list while being continuously engaged on some other demanding task, like listening to a string quartet or watching a football match. One might hear the quartet but could not really listen to it while making a list. Ditto football match. It is also clear that a robot, without any pretensions to consciousness at all, might manage perfectly well.

So what is it that is important about consciousness here?

Maybe the answer lies in the robot being much better at working memory. A robot, being a computer, is awash with working memory. Working memory which we might encapsulate as one of the expressions introduced at reference 2, expressions which are quite like the XML which powers much of the Internet, in function if not appearance – with the introduction to which at reference 3 being a little more accessible than that offered by Wikipedia.

An expression which consists of an ordered list of phrases, with each phrase consisting of an optional label and a data item (which might be a simple token, a bit of text, but which might be an expression in its own right or something more complicated still). And working memory offering functions like:

• Get the first (unlabelled) data item
• Get the last data item
• Get the next data item (with the system remembering whether we were working from left to right (normal, default) or from right to left)
• Get the data item with such and such a label
• Delete such and such a data item.

Stuff which would provide a shopping list algorithm with the working data that it needed. Stuff which, in the case of a computer, ought to include provision for errors. For, for example, there not being a phrase with the nominated label.

So the hypothesis is that a human can only do this sort of thing at all well when he is conscious. Which is not quite the same as saying that consciousness is needed for decent working memory, just that working memory seems to work much better when the host is conscious.

We speculate that working memory is actually stored in one of the layers of LWS-N, layers which are not available unless LWS-N is up and running, suggesting that the host is conscious, or at least near conscious, also that working data is apt to be cleared in the passage from one frame of consciousness to the next, unless something takes a bit of care.

The catch with this is that it is a bit of a bolt-on. One is not usually conscious of the contents of working memory in the way that one is conscious of the boxes in the cupboard. A catch which is not necessarily fatal as without knowing, it seems quite plausible that lots of evolutionary engineering is very ad-hoc. Evolution just grabs and adapts whatever is available, without much regard for the niceties of design. So, for example, we believe that the jaws of vertebrates are derived from the gill arches of fishes, structures which had and still have a quite different purpose, breathing rather than eating, with just the need for an orifice in common.

We look now at another scenario.

Tall building

We are standing in the street, looking at the apartment tower in Vauxhall called Vauxhall Tower, aka St. George’s Wharf Tower, all forty something floors of it and we are trying to count them, perhaps for the purposes of the computation of business rates.

Figure 5

We leave aside needing to decide what to do about the untidy bits top and bottom, to concentrate on the main business.

We suppose that there are no distinguishing features to help us distinguish one floor from another and the clouds are no help because they move about. If one loses one place, one simply has to start over.

What we do is concentrate our attention on a floor of the tower and then, maintaining that attention, ease it up one floor. At the same time, some other bit of the brain increments the number. If you click to enlarge, you can try this with the snap above – without using any kind of a pointer to mark your position – which would be cheating.

Part of what seems to happen is that one’s attention is on a floor. Then one attends to both that floor and the next floor, the floor above it. Then quickly moves one’s eyes up to that next floor and let the first floor go. An important part of this is having bright, clear markers between the floors. Another part is keeping the head and eyes as still as possible, this being helped by a bit of local variation in the visual texture which helps detect and so inhibit movement. Although not enough variation to help with identification – not a distinguishing feature of the sort just mentioned.

Fixation point is a useful concept here, something that humans, along with some other animals, seems to be able to manage: to hold attention – more precisely the fovea – to a more or less fixed point in the visual field. In this case the ability to do this is supported by the presence of clear boundaries. One is holding attention inside a clearly delineated patch of visual field. One’s brain seems to know about and to be able to control crossing that boundary on the cardinal points – north, east, south and west, otherwise up, right, down and left.

Figure 6

So in the figure above, one holds one attention on the place marked with the red spot. This needs to wobble about a bit – microsaccades and ocular drift in the jargon – to maintain the visual stimulus, but manages to stay well inside the boundary of the chosen rectangle. One then allows attention to drift to the right until it jumps across that boundary and locks onto the place marked with the green spot. Spots which do not actually exist on the windows of the tower, virtual spots brought into being by the position of the fovea. A wheeze which is not going to work when the windows get too small to be differentiated in this way.

Sometimes one is not sure that one has got it right, perhaps one has eased up two floors or no floor. This is usually fatal to maintaining the necessary attention.

In any event, something for which one certainly needs to be conscious and to concentrate on. Watching the football match with part of one’s attention is not going to do.

Something also that one does not need to do in the case of scenario one, at Figure 2 above. In that scenario, one might well maintain attention on a pigeon hole, but that attention can be supplemented by knowing that the pigeon hole in question is top row, third from the right, and because the number of pigeon holes is small none of the numbers involved need be more than about five, a small enough number to be managed without having to count. Equally important, information which can be coded as text and carried from one frame of consciousness to another, unlike the position of the fovea, a point to which we return below.

And digressing, one might go so far as to speculate that the act of visual fixation is the foundation on which visual consciousness is built. In which connection, we shall be reading reference 5 with interest.

Figure 7

Returning to our computer, we suppose that it cannot take a suitable picture of the tower all in one go and that it needs to do it in a sequence of overlapping chunks, working up the tower.
First, we would train the computer to pick out the strong horizontals and verticals which mark out the grid of windows. We would get it to do a bit of checking that these horizontals and verticals were reasonably regular, reasonably evenly spaced, which would help deal with the inevitable noise in the image of a chunk. For a chunk, that would then give us something like Figure 7 above.

Figure 8

It then works up the tower, producing a sequence something like Figure 8 above, aligning the successive chunks by matching the images of individual windows. It would not particularly matter which particular window was matched, within reason, once again allowing for a bit of noise in the otherwise stable light conditions. As we have already suggested, there should be enough local variation in visual texture to support this identification of individual windows.

The key to all this is that the computer can get an image into its memory banks and overlap organising information onto that image. Working up each chunk, incrementing the count of floors at each horizontal bar is then straightforward. An essentially offline task; the computer only needs to be online when it is photographing its successive chunks, a succession which it can manage with camera angles.

This, we suggest, the human brain cannot manage at all and it has to work in real time, relying on the rather unstable ocular system to maintain its position, rather than just keeping a few numbers in working memory.

If the human had been given a series of numbered photographs (numbering making things a lot easier), he might do something comparable to what the computer did, ruling lines on photographs and matching windows across photographs. But this does not seem to be an algorithm which we are able to internalise – at least not without a good deal of practise.

If, as suggested above, working memory is part of LWS-N, we  might have a working memory label amounting to current position in one layer linked by a column object to that position in a layer object representing the current chunk of the tower. A linkage which is destroyed when the compiler builds the next frame. So part of the difficulty of the count is maintaining the frame for far longer than its average duration of a second or so. Or in visual processing terms, the difficulty of maintaining inhibition of normal saccades for more than a second or so.

We speculate that what consciousness is bringing to this party is maintaining concentration and attention. The conscious brain knows what it is supposed to be doing and can suppress interrupts and quite a lot of noise. It gives the labile, unstable ocular system space in which it can lock onto floors. Something else that suggests that consciousness is not just a display-only device, as suggested by LWS-N. There is at least something in the consciousness system (whether or not that happens to be in some particular place) which interacts with other brain systems.

Figure 9

Maybe states of consciousness, our frames of consciousness in LWS-N, require energy to get there – and then more energy to get away. LWS-N locks a good chunk of the brain onto the current object of attention, onto the current goal; it stops most of other stimuli floating around in the subconscious from reaching a critical level. Not like climbing to the top of a hill and the least knock and we are off down again like Jack and Jill of the nursery rhyme, more like the volcano of Figure 9 above: once you get past the lip of the crater, the critical point, you are caught in the crater. We associate first to a planet moving through a space containing the occasional star. Sometimes the planet goes into orbit around one, eventually getting knocked out of that orbit by something or other, eventually arriving in some other orbit. Second to the escapement mechanism of mechanical clocks; the delicate two way interaction between the bit of machinery giving time (the pendulum or the hair spring) and the bit of machinery giving power (the weights or the main spring).

All of which gives us another behavioural correlate of consciousness, rather than an explanation.

Odds and ends

Pre-school children will sometimes learn to recite their numbers, say up to twenty or so, before they learn to count things. A neat illustration of the fact that counting things – as opposed to just counting – is a reasonably complicated activity.

One could offer the Vauxhall task to a group of technical colleges as a robot challenge, making it into one of those TV contest shows.

Weekly variations to include counting the number of courses in a brick wall, the number of books in a uniform edition on a library shelf and the number of spikes on a length of barbed wire.

We understand that a sloping version – that is to say not simply horizontal or vertical – would be harder for humans if not robots, but it would be interesting to know whether horizontal versions of the game were harder than vertical versions.

It would also be interesting to know about the saccades which were going on during the Vauxhall task, with there presumably having been a large number of them, maybe up to a hundred at a guess, by the time the task was completed.

We note that while a computer would not have trouble with either the shopping list or the apartment tower, programming a computer to do such a thing is a long way from just telling a general purpose computer what do in natural language, in the way that you might ask your telephone about the nearest Indian restaurant – the sort of task it can cope with.

We have commented here and elsewhere on the brain not being very good at sequences. Which does not sit very well with it – or at least the cerebellum – doing very well at the sequences of commands to the muscles needed to move us around the world in good order.

Conclusions

All of which suggests that consciousness is bound up with attention, goals and working memory, without going so far as to claim a causal link in either direction. Stuff which is maybe included in some layer of LWS-N, but without necessarily being projected into the subjective experience.

With maintaining attention on one object or goal including keeping other objects and goals well in the background. Keeping somewhere in mind some notion of what the current goal is and knowing when one is straying off-message. That if my eyes stray to passers by on the street outside, some supervisory process chips in to stop that straying, pull them back to the matter in hand.

We also suggest that the concept of ‘next’ is important, perhaps part of working memory. A concept which says give me the thing – whatever sort of thing that might be – that comes after the current thing.

References

Reference 1: https://psmv3.blogspot.com/2018/08/old-sevens.html.

Reference 2: https://psmv3.blogspot.com/2017/01/expressions-and-their-orders.html.

Reference 3: https://www.w3schools.com/xml/.

Reference 4: https://psmv3.blogspot.com/2018/05/an-update-on-seeing-red-rectangles.html.

Reference 5: Temporal Encoding of Spatial Information during Active Visual Fixation - Xutao Kuang, Martina Poletti, Jonathan D. Victor, and Michele Rucci – 2012. A reference from Wikipedia.

Reference 6: https://psmv3.blogspot.com/2018/03/trolleys-126-to-130.html.