Layers and columns

[Figure I]

Figure I is a soft box model of the structure we are to describe in what follows. A powerful technique, very popular in the 1980’s, and, for all I know, still popular now – with google turning up reference 9 fast enough.

We use A, B, C to denote layers; K, L, M to denote patterns; R, S, T to denote sets; and, X, Y, Z to denote objects. Possibly with a numeric suffix; so, for example, ‘K2’. And words which have a special meaning are italicised when those special meanings are being defined.

At reference 1, we built what we will call layer objects on a grid using repeating patterns, patterns defined by the values of the points of a rectangle, moving in steps over the grid, and which had to repeat to count. At reference 4, we had an excursion into expressions, a relation called instantiation and the partial order resulting therefrom. At reference 6, we did lines. In this fourth post in the series, we move onto layers, onto column objects and to some of the things we can say about all these objects.

We suppose a plane substrate, or foundation, idealising a patch of cortex, a square grid with each point having a pair of real coordinates. A foundation which could be used to plot the position of the millions of neurons involved. The neurons are buzzing away in time, collectively generating an electrical signal of some sort. Roughly speaking, at any point on our grid we can analyse that signal into frequency bands, around 10 of them, an analysis which we hold good for an interval of time of the order of a second or so. In what follows we consider just one such interval of time, here and elsewhere called a frame, a frame of consciousness. A frame which has been built out of the raw data by a process of compilation.

To give a bit more context, this building, this process of compilation is updated from time to time, giving us a number of frames in a take. And then, at rather longer intervals, perhaps of several seconds or more, we start over with a new take, with a recompilation of the data, more or less from scratch. With takes, in turn, being grouped into scenes. See reference 2 for a slightly different perspective on frames.

Our objective is to describe an environment in which consciousness might come to pass. We are not so interested in exactly how the necessary data has been collected, rather in how it is put together and then processed in this frame of consciousness to give us the subjective experience.

In posts to come, we will look further at what we will be calling the frame scanning processes. In order to do this processing, we will allocate some of the higher (frequency) layers to process, while leaving the lower layers to data, with the current thought being that several pulses, several repetitive processing sweeps over the data will constitute one of our frames, a moment of consciousness, the subjective experience of consciousness. With a frame lasting at least some tenths of a second, perhaps more than a second. Noting in passing that this hypothesis that consciousness is located in quite a small part of the brain, albeit needing support from further afield, excludes the possibility of large, distributed entities like the internet being conscious, however clever and human-like in their behaviour they might be. Not sure where this leaves the octopus, with its rather distributed nervous system, but which might, from its behaviour, be considered a candidate for some sort of rudimentary consciousness, along with the fishes.

But for the moment we are just considering the data structure embodied by our frame, the data in which is supposed to pretty much fixed for the duration of a frame. Maybe not invariant, but not changing much.

So, our structure has layers, corresponding to our ten or so frequency bands, all built on our one foundation, with the number of points to the sides of our grid being in the order of a small number of thousands, roughly comparable to those of the pictures taken by my telephone. The value of the brain at a point at a layer is then the amplitude or power of the signal at that frequency, binned into some modest number of bins, following reference 1, say around 20 of them.

Layer objects

At reference 1, we introduced patterns and objects defined on a grid, a grid which amounts to one of the layers introduced here. What we now have is the possibility, the probability, of objects defined on the same pattern cropping up in more than one layer. This can happen because the way in which we have specified patterns and objects is independent of layer, can be applied indifferently to any layer.

At reference 6, we introduced the special cases of lines and loops. And now, in the case of a loop object, we talk of the closure of the object, which is obtained by adding the points of the interior to the points of the loops. We worry a little about possible confusion to come between an object as a whole and its boundary, its perimeter.

So, starting at one particular object, say object X on layer A defined on pattern K, we might find several other objects defined on that same pattern K, several peers of that object on that same layer A. In some rough way, we imagine the peer objects to be different instances, difference members of some set of other, perhaps the different members of a football team. Then several more K objects at some other layer B. And, perhaps, more again at layer C. We call the set of objects defined on pattern K, P(K). Or, hopefully without confusion, the set of object defined on the pattern which defined the object X, P(X). The power set on K, or on X.

We could label all the members of these sets, all these objects with a triple {Position, Pattern, Layer}, where position is the coordinates of some point inside the object (leaving aside the possibility of holes), pattern is the specification or the name of the pattern which defines the object and layer is the number of the layer. But we do not suppose that the brain does this, rather it goes some of the way with pointers, made in a brain out of axons and synapses, to which we will come to below.

But we do extend the notion of peer object by saying that two members of P(Y) which are distinct, which are not the same object, are in-laws.

Column objects

Next, we define column objects with patterns by analogy with the definition of what we are now calling layer objects with patterns at reference 1. These objects are motivated, in part, by the observation of small groups of neurons which respond to things like particular faces or places and to the cliques put forward by Tsien and others (and for which see reference 3).

Note that the firing of such cliques of neurons conveys no information in itself, we just have a bunch of neurons at some particular place in the brain firing away, any more than you knowing that my national insurance number is YC763982B tells you anything much about me, without access to the national insurance files. In the present case, what is needed is links to other neurons to make up the sort of information that we can apprehend.

To this end, we accommodate such column objects by saying that that our frequency power has a low value designated L and a high value designated H, with low being noise with little if any information content, with there being some leakage across layers and with there being some correlation. To the point that if we have a high value on any layer, we have it on all layers, we have a column across the layers rather than a shape within a layer. We then have a point on our grid which is flashing strongly and which could be, could correspond to one of these clusters or cliques, tied to some particular event, feature, face or object. Also sharing some of the properties the proper nouns of natural language and with layer objects sharing some of the properties of common nouns. And looking ahead, we have arousal whizzing up and down the column, shifting from one frequency to another, something which might be energy expensive. Harmonics good, inharmonics not so good.

Information about the point, information which gives some point to the point, can be accommodated by one or more of the layers carrying a layer object which includes our column object, in which we would need to be helped along by allowing the ‘H’ of our column as a wild card when it comes to pattern matching. With all these layer objects then occupying more or less the same small patch – overlapping and not like the spread-out objects envisaged at reference 3, with each object occupying its own bit of the plane. A column of layer objects.

Note that, viewed from above, these various layer objects will overlap without coinciding.
Projecting them onto our foundation, we have a set of points which we call the closure of our column object, with projection generally being defined in the next section.

Projection

[Figure II]

We have a projection from our objects onto our foundation, a projection which maps each point in the object onto a square on the foundation plane defined by that grid point, in the same way as a cell on an Excel worksheet is defined by row and column. A conversion of points in space to space.

So, in Figure II above, we have projected six points from some object onto squares on the foundation, taking a bit of liberty between the squares. Note that the squares are deemed to exclude the top edges and the left hand edges, so that they add up nicely. That is to say we have the half open squares defined as sets of the form: {(x,y) | 0 <= x <1, 0 <= y <1}.

[Figure III]

While in Figure III above, we have a layer object made up of nine occurrences of a pattern defined on a rectangle with six rows and five columns. Each pattern is made up of thirty points, and each pattern maps onto one of the pink rectangles in the example. We then take the union of our nine pink rectangles to make the projection of the object as a whole.

In this way, we have mapped our layer objects, defined in terms of points, onto rectangular polygons on our foundation plane – where by rectangular polygon we mean a polygon where all the edges are either vertical or horizontal lines.

We tweak the definition in the case of a loop, to project the closure, giving us something rather bigger, something without a hole in the middle.

[Figure IV]

While, in the case of a column object, we have the same point on each layer and we would just have that one point to project, onto one of the small pink squares of Figure II, drawn at a rather large scale than Figure III. However, what we actually do is project the closure of the column object, giving us something rather bigger, more a collection of big rectangles in the way of Figure IV which shows, in a rather rough and diagrammatic way, the union of three layer objects – pink, blue and green – something potentially more complicated than the projection of a single layer object. In this example, the column object itself would be somewhere in the middle, a place where all three colours overlap.

This projection enables us to apply the usual terms from set theory, from plane geometry to the objects in our world. This set is a subset of that set, or more correctly the projection of this object is a subset of the projection of that object. This set is disjoint from that set. The set S1 is the union of the disjoints sets S2 and S3.

Sometimes we are a little sloppy and omit the projection bit, but hopefully things will be clear enough.

Inclusion

We are now in a position to define inclusion, a relation between objects derived in a straightforward way from the set inclusion relation on their projections. An object X is said to include the object Y if the projection of Y is a proper subset of the projection of X. A definition which excludes X including itself, that is to say the inclusion relation is neither reflexive nor commutative – but it is transitive.

Note that we are not here distinguishing between inclusion inside the line which makes a loop and inclusion inside the closure of that loop. And it is entirely possible for an object to lie inside a line.

On the other hand, an object inside a loop can be defined on a pattern on the same layer as that defining the loop, which would not otherwise be possible, affording us some economy in the use of layers.

When the object X (defined on the pattern K) includes the object Y (defined on the pattern L), we say that the Y qualifies X or that Y describes X, tells us something about X. And what Y is, what is most relevant about Y to X, might be on the spot, included in turn within Y, while for information which is not so important, we might need to go elsewhere, perhaps to some other layer, to some other object defined (and identified) by that same pattern L for that information. Part of the point of this last device being that it allows us to escape from a hierarchical, nested tree structure to the more powerful network.

We say that an object X is terminal if it includes no other object.

But the main point is that we have used coincidence in space to link object X with object Y, a coincidence, a binding (a word much used in this sort of context) which is made possible by the device of layers.

Digressing slightly, we could even manage a full blown expression in the sense of reference 4, if we cared to spend three or more layers on them, something of the form ‘run(man(tall name=Peter) across=road(name=‘Manor Green Road’ with=pudding)’: expressions, with which, when supported by suitable conventions & customs, I believe that you can say more or less anything.

[Figure V]

A simpler expression than the running expression, with just two labelled phrases, is illustrated in Figure V above, with the large green rectangle carrying the expression as a whole; with the blue and pink rectangles carrying the phrase labels; and, with the brown and grey rectangles carrying the phrase bodies. But with a bigger page one could do better.

Pointers

During a frame of consciousness, waves of arousal will be washing around our structure.

Some of this movement will be up and down column objects, where arousal at one frequency will tend to spread out to other frequencies. Some will be around layer objects.

Some will be across peers, some will be across in-laws, some will be around layer objects with loop boundaries. Some will be between neighbouring layer objects.

Some of it will be further afield, using synaptic connections compiled into the structure of a frame.

These connections will join up pairs of objects in our structure. But, given that we think that some economy of connection is appropriate, needed to move us from noise to information, which of the many pairs in these sets will be connected?

Now, we can describe the nests of objects built with patterns & inclusion with expressions of the form: K1(L1 K2) or more complicated K1(L1(M1 M2) K2(K3 M2 M3) L2(M4 M5 L3(K4 L4(M6))) L5), in both of which, following our convention, all the strings like ‘K3’ stand for patterns. In the example given in Figure V, we might have K1(L1(M1) L2(M2) …).

If X is a layer object, then E(X) is its expression, an expression which will include all its subordinate objects. The terminal elements of these expressions – in the example just given, M1 and M2 – with also correspond to the terminal objects of the previous section. The expressions for the terminal objects themselves will be trivial. Otherwise, these expressions will usually capture information from a number of layers.

We also have a limited version of the expressions defined at reference 3, simplified to the extent that we do not label our phrases, and so we can partially order our objects using these expressions: X < Y if the expression describing Y is an instance of the expression describing X, in short, if Y is bigger than X.

We note in passing that an expression of the sort exemplified above does not capture all of what we have in our structure at that position, but it can capture quite a lot of it, rather in the way that the word ‘cat’, for someone who knows cats, can have much the same force as a picture of a cat or even a real cat. And what it does not capture it can point to.

We say that there will be two way pointers between X and Y if: X < Y; and, there is no Z such that X < Z < Y. Note that there might be more than one such Y, potentially giving us an upward pointing tree.

This definition of pointing does not call on layers, although it is likely that many pointers will cross from one layer to another. It might turn out that our processes might have preferences, at one time preferring to go from high frequency layers to low frequency layers, at another the other way around.

We make no claims here about in exactly what sense X and Y are about the same real world object or category; we are just defining some pointers, pointers which will be followed in the processing to be specified in a post to come. Pointers which bind, in this case, X and Y together. But see the note about peer objects above.

Threads

Sometimes there will be a natural clustering of the data in our structure into what we call threads.

Sometimes one is more conscious of one or more of the threads individually, sometimes one is more conscious of the frame as a whole. There is a sense of the threads being the foreground objects of consciousness and there might or might not be a thread for the background. Also, that there will sometimes be a self thread, which may, depending on the circumstances, give rise to anxiety or even fear. It is, in some way, exposed to attack by virtue of simply being there.

This clustering will be expressed in terms of column objects, with each thread being one or more column objects and the layer objects which interact with them. Any one column object can only appear in one thread and few layer objects will appear in more than one. The point of a thread is that it is reasonably self-contained.

Or as Tononi & Koch would have it at reference 7, both integrated and differentiated at the same time. Which might be achieved here by varying the balance between the processes scanning threads and those scanning the frame, the structure as a whole; between intra-thread scanning and inter-thread scanning. I believe something of the sort occurs with the partials of a church bell, but more of that in a post to come. But, in the meantime, for budding campanologists, there is reference 8.

With the expectation that, if one were to apply some statistical clustering tool, perhaps independent components analysis, to the structure as a whole, these threads are what one would come up with.

And with the further expectation that threads will persist, active or not and at least after a fashion, in time. To the extent of a thread about the dog running down the garden today being recognisably the same as that we had yesterday. With threads, along with objects generally, both building from current input and drawing down from permanent memory.

Sequences

We introduced lines and loops at reference 6, but deferring the matter of sequences, to which we now turn. With the allegation at reference 5 being that sequences are essential support for the activity planning which is particularly human.

The idea is that we define the line with a large enough pattern, on layer A, for the items in the sequence to be included in it.

[Figure VI]

In Figure VI above, the pink line holds our sequence, perhaps the alphabet, with the first six letters shown, the layer objects X1, X2, X3, X4, X5 and X6, defined on one or more layers other than A, being the members of the sequence. We will have rules about there not being any other objects in the same space, other than, perhaps, something marking the start. Maybe other small marker objects of the same sort – objects which function a bit like the small words and word endings doing grammatical things in natural languages. But we do not need to fuss about the shape or the position of these member objects, it is enough for them to be included by the right container.

The pink patterns function as containers, with the objects inside being the members of the sequence. Things are so arranged that we can arouse each member of the sequence in turn, with the position marked by a cursor, shown here on the fourth letter, at X4. The cursor moves on at some prompt from the outside and we thus arouse each member of the sequence in turn.

Odds and ends

We say that the structure is simple if for all patterns K, there is exactly one member of P(K) which is an instance of all other members of P(K). Or put another way, all the P(K) have maxima, an object which contains the most, all the information about the objects which have been defined on K. A property which appears to exclude the both possibility of holding contradictory information about an object, and that of using the detailed versions of objects for different members of the same category, say dogs. More work needed here.

We say that an object is grounded if it relates in some simple way to something out in the outside, the real world. An object – such as a pie – or an event – such as a duck crossing the road. Some of our objects are going to be grounded in this way.

We believe that by building our structure in this way, we are leveraging the visual image processing machinery that vertebrates have been working on for a long time. Leveraging that machinery to do something which is mostly confined to the human vertebrates.

Conclusions

We believe that, using the machinery which we have now introduced, we will be able to capture, to express a great deal of information about both the real worlds and the inner worlds, in a form which makes it accessible to neural processing, processing of a sort which might generate our grail, our consciousness. We shall turn to this processing in a future post.

But first, we will try out this machinery on more of the sort of data that we think that we might want to hold in our structure.

Words with special meanings

Column object, Frame, Structure, Foundation, Grounded, Inclusion, In-law, Layer, Layer object, Peer, Power set, Projection, Simple, Terminal, Thread.

References

Reference 1: http://psmv3.blogspot.co.uk/2016/12/from-grids-to-objects.html.

Reference 2: http://psmv3.blogspot.co.uk/2016/11/an-assembly-for-consciousness.html.

Reference 3: On initial  brain activity mapping of episodic and semantic memory code in the hippocampus - Joe Z. Tsien and others – 2013.

Reference 4: http://psmv3.blogspot.co.uk/2017/01/expressions-and-their-orders.html.

Reference 5: http://psmv3.blogspot.co.uk/2016/09/what-is-consciousness-for.html.

Reference 6: http://psmv3.blogspot.co.uk/2017/01/lines.html.

Reference 7: Consciousness: here, there and everywhere - Tononi & Koch – 2015.

Reference 8: http://www.hibberts.co.uk/.

Reference 9: http://www.sqa.org.uk/e-learning/SDM03CD/page_31.htm.

Reference 10: http://psmv3.blogspot.co.uk/search?q=sra.

Group search key: sra. For which you can click on reference 10 above.