Press the right key for the next slide (or swipe left)

also ...

Press the left key to go backwards (or swipe right)

Press n to toggle whether notes are shown (no equivalent if you don't have a keyboard)

Press m or double tap to see a menu of slides


Segmentation and the Principles of Object Perception

How do humans segment objects?


Recall that the way objects are ordinarily arranged in space, so that one occludes parts of another, prevents us from doing this in any simple way.
Infants from 4.5 months of age can use featural information to segment objects.

using featural information

In Amy Needham's 1998 study, infants were shown a display like this.
Featural information---the difference in textures of the objects---suggests that these are two separate objects.
But can infants use this information to detect that there are two objects?

Needham (1998)

Some infants were then shown the object being moved like this, so that it is clearly two separate objects.
Other infants where shown the object being moved like this.
If infants think there is one object, they should expect the second kind of movement.
But if infants think there are two objects---if, that is, they can use the featural information to segment objects---then they should expect the former kind of movement.
What were the results? ...
Needham showed that infants from 4.5 months of age can use featural information to segment objects.

Needham (1998, figure 4)

Interestingly, 4.5 month old infants were able to succeed even when the point of contact between the two objects was occluded, as in this diagram.

Needham (1998, figure 6)

These are the results for 4.5 month old infants.
One further thing: infants can also use shape information in segmenting objects, and shape information appears to trump featural information \citep{needham:1999_role}.

Needham (1998, figure 7)

Can we fully explain how infants segment objects just by appeal to features? To see why it couldn't be just features that we use to segment objects, consider some more cases ...

Could it all be features?

`infants perceive the boundaries of a partly hidden object by analyzing the movements of its surfaces: infants perceived a connected object when its ends moved in a common translation behind the occluder. Infants do not appear to perceive a connected object by analyzing the colors and forms of surfaces: they did not perceive a connected object when its visible parts were stationary, its color was homogeneous, its edges were aligned, and its shape was simple and regular' \citep{kellman:1983_perception}.
Here is an occluded object---a stick behind a box.
The movement is enough to convince 4-month-old infants that there is just one stick even though they never see its middle \citep{kellman:1983_perception}. We can discover this by measuring how different displayes cause them to dishabituate.

Spelke (1990, figure 2)

After being habituated to this this, 3-month-old infants were shown one of two displays.

Kellman & Spelke (1983, figure 3)

And here are the results (subjects were 3-month-old infants).

Kellman & Spelke (1983, figure 4)

The fact that infants can correctly segment partially occluded objects based on their movements already indicates that they can't be thinking about features only.
For more evidence, consider this display.
The two parts of the moving object are featurally different.
Despite this, infants expect to see a single connected object behind the block (\citealp{kellman:1983_perception}, Experiment 6; \citealp{Spelke:1990jn}).

Kellman & Spelke (1983, figure 13)

Here are the test stimuli (each groups is shown one or the other).

Kellman & Spelke (1983, figure 13)

And here are the results.
Subjects in this experiment were 4-month-old infants.
So we saw that infants can use featural information to segment objects, but the principle of cohension can trump featural indicators of difference.
So infants' abilities to segment objects are not based entirely on recognising features.

Kellman & Spelke (1983, figure 14)

If infants do not rely only on features to do this, then how do infants segment the objects in the displays we've just been seeing?

If not by features, then how? Principles!

\citet{Spelke:1990jn} suggests that infants rely on a set of principles to segment objects. But what are the principles?

Recall this diplay with on object moving behind a stationary block.
What kind of principle could be used to identify that the occluded thing is a single object?

Kellman & Spelke (1983, figure 13)

rigidity—‘objects are interpreted as moving rigidly if such an interpretation exists’

\citet{Spelke:1990jn} suggests the principle of rigidity. This principle says that ‘objects are interpreted as moving rigidly if such an interpretation exists’ The hypothesis that this principle describes in part how infants segment objects correctly predicts that they will treat the moving occluded stick as a single object.
But rigidity is not the only principle we need to explain how infants segment objects ...
Another principle which seems to be involved in segmenting objects is the principle of cohension. According to this principle, ‘two surface points lie on the same object only if the points are linked by a path of connected surface points’ \citep{Spelke:1990jn}.


‘two surface points lie on the same object only if the points are linked by a path of connected surface points’

(Spelke 1990)

Spelke (1990, figure 4)

For example, objects arranged as on your left were percevied by 3-month-olds as two objects, whereas infants treated the displays like that on your right as if they were one object.
(This was measured using a habituation paradigm \citep{kestenbaum:1987_perception}. Infants were habituated to the display. Then either one object's position changed, or both objects' positions changed but in such a way as to preserve the overall configuration of the two objects. Infants could show that they perceived the configuration as a single object by looking longer when just one object's position changed.)
Here's a second example.
In this case \citep[Experiment 2]{spelke:1989_reaching}, the researchers tested 5-month-old infants using a reaching paradigm.
The smaller of the two objects was always closer to the infants. Infants should reach more often for the smaller, nearer object when they represent the simuli as a single object than when they represent it as two objects.
Exploiting this fact, \citet[Experiment 2]{spelke:1989_reaching} showed that infants perceive the stimlui on your left as two objects but the stimlui on your right as a single object.
But how does the principle of cohension apply to this moving display? ‘When two surfaces are separated by a spatial gap (as in Figure 4a) or undergo relative motions that alter the adjacency relations among points at their border (as in Figure 4i), the surfaces lie on distinct objects.’
\citet{Spelke:1990jn} proposes that our ability to segment objects depends on four principles. We've already seen two of these in action (rigidy and continuity), and we will shortly see that a further principle is needed, too.

Principles of Object Perception

\textbf{Principles of Object Perception \citep{Spelke:1990jn}}
  • cohesion—‘two surface points lie on the same object only if the points are linked by a path of connected surface points’
  • boundedness—‘two surface points lie on distinct objects only if no path of connected surface points links them’
  • rigidity—‘objects are interpreted as moving rigidly if such an interpretation exists’
  • no action at a distance—‘separated objects are interpreted as moving independently of one another if such an interpretation exists’

(Spelke 1990)

What is the status of these principles?
Spelke’s position might be put like this:


\item We (as perceivers) start with a cross-modal representation of three-dimensional perceptual features which includes their locations and trajectories.

\item Our task is to get from these representations of features to representations of objects.

\item \emph{Descriptive component} We do this as if in accordance with certain principles (cohesion, boundedness, rigidity, and no action at a distance).

\item \emph{Explanatory component} We acquire representations of objects because we apply the principles to representations of features and draw appropriate inferences.


The key point for our purposes is the explanatory component.
The principles are not supposed to be merely heuristics for describing and predicting infants’ performance on preferential looking tasks.
Rather, these principles are supposed to explain why infants look longer at some things than at others.
This what motivates the hypothesis that infants know these principles and use them in reasoning about objects: unless this hypothesis is true, it’s hard to understand how the principles could have explanatory relevance.
Inspiration for Spelke’s view comes from Marr and Chomsky.
Marr showed that many visual processes can be modelled as inferences.
And Chomsky pioneered the idea that our knowledge of language depends on knowledge of principles of syntax.
What unites these three cases, Spelke on object segmentation, Marr on vision and Chomsky on syntax?
It’s that they are straightforwardly cognitivist in appeal to knowledge and inference.
Principles are known, and they are used via a process of inference.
(There’s a nice quote from Fodor on your handout underlining this point.)
‘Chomsky’s nativism is primarily a thesis about knowledge and belief; it aligns problems in the theory of language with those in the theory of knowledge. Indeed, as often as not, the vocabulary in which Chomsky frames linguistic issues is explicitly epistemological. Thus, the grammar of a language specifies what its speaker/hearers have to know qua speakers and hearers; and the goal of the child’s language acquisition process is to construct a theory of the language that correctly expresses this grammatical knowledge.’
\citep[p.\ 11]{Fodor:2000cj}

Marr & Chomsky

So what is the status of Spelke’s principles of object perception?
\subsection{The simple view}
The principles of object perception are things that we know, and we generate expectations from these principles by a process of inference.
The simple view is that the Spelke principles are just known in whatever sense anything is known.
The simple view isn’t exactly Spelke’s, but it’s a useful starting point for discussion.

the simple view

The Simple View is worth considering in its own right because it is so, well, simple. But our interest in it may be piqued by the fact that Spelke herself appears to have accepted the Simple View at one point in her thinking.

‘objects are conceived: Humans come to know about an object’s unity, boundaries, and persistence in ways like those by which we come to know about its material composition or its market value’

\citep[p.\ 198]{Spelke:1988xc}.

Spelke (1988, p. 198)

Now you might think that the case for these principles is not yet very strong. In that case, asking hard questions about their status would hardly be necessary. So let’s consider further evidence for these principles. We can do this by turning from segmentation (which was our first requirement on knowledge of objects) to representing objects as permanent.