\title {Joint Action \\ Lecture 09}
\maketitle
\def \ititle {Lecture 09}
\def \isubtitle {Joint Action}
\begin{center}
{\Large
\textbf{\ititle}: \isubtitle
}
\iemail %
\end{center}
distributive vs collective
‘The injections saved her life.’
Consider the statement, ‘The injections saved her life.’
This could be true in virtue of her receiving several injections on different occasions, each of
which saved her life.
In this case, the injections saving her life is just a matter of each injection individually
saving her life; this is the distributive interpretation.
But the statement is also true if she was given two injections on a single occasion where each
injection was necessary but not sufficient to save her life.
In this case the injections saving her life is not, or not just, a matter of each injection
individually saving her life; this is the collective interpretation.
The difference between distributive and collective interpretations is clearly substantial, for on
the distributive interpretation the statement can only be true if her life has been saved more
than once, whereas the truth of the collective interpretation requires only one life-threatening
situation.
\section{Collective Goals and Motor Representations}
\section{Collective Goals and Motor Representations}
Motor representations can ground collective goals in this sense:
in some cases, two or more actions involving multiple agents have a
collective goal in virtue of the actions being appropriately related to an
interagential structure of motor representations.
Motor representations concern not only bodily configurations and movements but also more distal outcomes such as the grasping of a mug or the pressing of a switch \citep{butterfill:2012_intention,hamilton:2008_action,cattaneo:2009_representation}.
Some motor processes are planning-like in that they involve deriving means by which the outcomes could be brought about and in that they involve coordinating subplans \citep{jeannerod_motor_2006,zhang:2007_planning}.
Motor processes concerning actions others will perform occur in observing others act \citep{Gangitano:2001ft}---and even in observing several others act jointly \citep{manera:2013_time}---and enables us to anticipate their actions \citep{ambrosini:2011_grasping,aglioti_action_2008}.
In joint action, motor processes concerning actions another will perform can occur \citep{kourtis:2012_predictive, meyer:2011_joint},
and can inform planning for one's own actions \citep{vesper:2012_jumping,novembre:2013_motor,loehr:2011_temporal}.
In some joint actions, the agents have a single representation of the whole action (not only separate representations of each agent's part) \citep{tsai:2011_groop_effect,loehr:2013_monitoring,Menoret:2013fk},
and may each make a plan for both their actions \citep{meyer:2013_higher-order,kourtis:2014_attention}.
An interagential structure of motor representation: \begin{enumerate} \item there is an outcome to which a joint action could be collectively directed and in each agent there is a motor representation of this outcome; \item these motor representations trigger planning-like processes in each agent which result in plan-like hierarchies of motor representations; \item the plan-like hierarchy in each agent involves motor representations concerning another's actions as well as her own; \item the plan-like hierarchies of motor representations in the agents nonaccidentally match. \end{enumerate}
\section{Motor Representation}
A \emph{goal} is an outcome to which an action is directed.
Motor representations represent goals such as the grasping of an egg or the pressing of a switch.
These are outcomes which might, on different occasions, involve very different bodily configurations
and joint displacements (see \citealp{rizzolatti_functional_2010} for a selective review).
Motor representations trigger processes which are planning-like insofar as they involve (a) computing
means from representations of ends; and (b) satisfying relational constraints on actions.
Recall how Ayesha takes a glass and holds it up while Beatrice pours prosecco;
and unfortunately the prosecco misses the glass, soaking Zachs’s trousers.
Ayesha might say, truthfully, ‘The collective goal of our actions was not to soak Zach's trousers in
sparkling wine but only to fill this glass.’
What could make Ayesha’s statement true?
motor representation
Let me mention some almost uncontroversial facts about motor representations and
their action-coordinating role.
\section{Kourtis et al (2014)}
WHY MOTOR REPRESENTATION? ONE ADVANTAGE OF THEORISING ABOUT MOTOR REPRESENTATIONS IS
THAT YOU CAN GENERATE TESTABLE PREDICTIONS. INTENTIONS ARE FICTIONS. YOU CANNOT GENERATE
TESTABLE PREDICTIONS FROM A THEORY ABOUT INTENTION. SO IF YOUR QUESTION IS ABOUT WHAT
ENABLES COOPERATION, YOU WILL PROBABLY NEED A STORY ABOUT PSYCHOLOGICAL MECHANISMS.
ONTOLOGY : YOU WOULDN’T DO IT WITHOUT THINKING ABOUT PHYSICS, WOULD YOU?
SOCIAL ONTOLOGY NEEDS PSYCHOLOGY LIKE ONTOLOGY NEEDS PHYSICS.
D'Ausilio et al (2009, figure 1)
‘Double TMS pulses were applied just prior to stimuli presentation to selectively prime the cortical activity specifically in the lip (LipM1) or tongue (TongueM1) area’
\citep[p.~381]{dausilio:2009_motor}
‘We hypothesized that focal stimulation would facilitate the perception of
the concordant phonemes ([d] and [t] with TMS to TongueM1), but that
there would be inhibition of perception of the discordant items
([b] and [p] in this case). Behavioral effects were measured via reaction
times (RTs) and error rates.’ \citep[p.~382]{dausilio:2009_motor}
Consider this case.
An agent fills a glass by holding it in one hand, holding the bottle in the other,
bringing the two together and pouring from the bottle into the glass.
[*demonstrate].
It’s a familiar idea that motor representations of outcomes resemble intentions in that they can
trigger processes which are like planning in some respects.
These processes are like planning in that they involve starting with representations of relatively
distal outcomes and gradually filling in details, resulting in a structure of motor representations
that can be hierarchically arranged by the means-end relation \citep{bekkering:2000_imitation,
grafton:2007_evidence}.
Processes triggered by motor representations of outcomes are also planning-like in that they
involve selecting means for actions to be performed now in ways that anticipate future actions
\citep{jeannerod_motor_2006,zhang:2007_planning,rosenbaum:2012_cognition}.
Now in this action of moving a mug, there is a need, even for the single agent, to coordinate the
exchange between her two hands.
(If her action is fluid,
she may proactively adjust her left hand in advance of the mug’s being lifted by her right hand
\citep[compare][]{diedrichsen:2003_anticipatory,hugon:1982_anticipatory, lum:1992_feedforward}.)
How could such tight coordination be achieved?
Part of the answer involves the fact that motor representations and processes concerning the
actions involving each hand are not entirely independent of each other.
Rather there is a plan-like structure of motor representation for the whole action and motor
representations concerning actions involving each hand are components of this larger plan-like structure.
It is in part because they are components of a larger plan-like structure that the movements of
one hand constrain and are constrained by the movements of the other hand.
We have just considered what is involved in performing an ordinary, individual action, where an agent
fills a glass from a bottle, taking one in each hand and moving them in a carefully coordinated way.
Compare this individual action with the same action performed by two agents as a joint action.
One agent takes the glass while the other takes the bottle.
The joint action is like the individual action in several respects.
First, the goal to which the joint action is directed is the same, namely to move the mug from here to there.
Second, there is a similar coordination problem---the agents’ two hands have to meet.
And, third, the evidence we have mentioned suggests that in joint action, motor representations and processes occur in each agent much like those that would occur if this agent were performing the whole action alone.
Why would this be helpful?
Suppose the agents' planning-like motor processes are similar enough that, in this context, they will reliably produce approximately the same plan-like structures of motor representations.
Then having a single planning-like motor process for the whole joint action in each agent means that
\begin{enumerate}
\item in each agent there is a plan-like structure of motor representations concerning each of the others’ actions,
\item each agent's plan-like structure concerning another's actions is approximately the same as any other agent's plan-like structure concerning those actions,
\item each agent's plan-like structure concerning her own actions is constrained by her plan-like structures concerning the other’s actions.
\end{enumerate}
So each agent’s plan-like structure of motor representations concerning her own actions is indirectly constrained by the other agents' plan-like structures concerning their own actions
by virtue of being directly constrained by her plan-like structures concerning their actions.
In this way it is possible to use ordinary planning-like motor processes to achieve coordination in joint action.
What enables the two or more agents' plan-like structures of motor representations to mesh is not that they represent each other's plans but that they processes motorically each other's actions and their own as parts of a single action.
So how does the joint action differ from the corresponding individual action?
There are at least two differences.
First, we now have two plan-like structures of motor representations because in each agent there is a planning-like motor process concerning the whole action.
These two structures of motor representations have to be identical or similar enough that the differences don’t matter for the coordination of the agents’ actions---let us abbreviate this by saying that they have to \emph{match}.
The need for matching planning-like structures is not specific to joint action;
it is also required where one agent observing another is able to predict her actions thanks to planning-like motor processes concerning the other’s actions (we mentioned evidence that this occurs above).
A second difference between the joint action and the individual action is this.
In joint action there are planning-like motor processes in each agent concerning some actions which she will not eventually perform.
There must therefore be something that prevents part but not all of the planning-like motor process leading all the way to action.
Exactly how this selective prevention works is an open question.
We expect bodily and environmental constraints are often relevant.
There may also be differences in how others’ actions are processed motorically \citep[compare][]{novembre:2012_distinguishing}.
\footnote{\citep[p.\ 2901]{novembre:2012_distinguishing}: 'in the context of a joint action—the motor control system is particularly sensitive to the identity of the agent (self or other) of a represented action and that (social) contextual information is one means for achieving this distinction'}
And inhibition could be involved too \citep[compare][]{sebanz:2006_twin_peaks}.
The Simple View
As you may recall, the Simple View says that
Two or more agents perform an intentional joint action
exactly when there is an act-type, φ, such that
each agent intends that
they, these agents, φ together
and their intentions are to their actions.
Let’s start by trying to get a pre-theoretical handle on the notion of joint action.
I’ve already given you some examples, but it’s even better to use contrast cases ...
Recall our question, What distinguishes joint actions from parallel but merely individual actions?
‘A first step is to say that what distinguishes you and me from you and the Stranger is that you and
I share an intention to walk together—we (you and I) intend to walk together—but you and the Stranger
do not. In modest sociality, joint activity is explained by such a shared intention; whereas no such
explanation is available for the combined activity of you and the Stranger. This does not, however,
get us very far; for we do not yet know what a shared intention is, and how it connects up with joint
action’ \citep[p.~152]{Bratman:2009lv}.
‘joint activity is explained by such a ; whereas no such explanation is available for the combined activity’ of those acting in parallel but merely individually.
Bratman, 2009 p. 152
So the idea is that shared intention is to joint action as intention is to ordinary, individual
action.
Of course, invoking shared intention doesn’t by itself amount to explaining anything. It simply
takes us in a circle. (Which is not necessarily bad.)
To characterise joint action, we have two tasks: (i) characterise shared intention; (ii) characterise
a relation between shared intentions and events such that the events are joint actions.
aggregate subject
a deeper understanding
But I argued that we need a deeper understanding.
We are wrong to think that there is a single binary distinction,
and we are wrong to think that we can identify a target for theorising by merely pointing to
intuitive contrast cases.
We need to do some work to turn intuitions into observations; these observations will then
provide the basis for a theory.
How to ground a theory of joint action?
Step 1: identify features ...
- collective goals
- coordination
- cooperation
- contralateral commitments
- experience
Step 2: ... which generate how questions.
A theory of joint action is then an attempt to answer the how questions.
Rakoczy et al (in progress)
This is just the beginning ...
\section{Four Challenges}
--- Rakoczy, Pacherie, Butterfill, Tomasello, Schilbach, Kita & Matthews (in progress)
\emph{The Diversity Challenge}
The fundamental challenge for an integrative framework of shared intentionality is to move beyond
the 'one size fits all' assumption that one set of simple necessary and sufficient conditions can
be given to characterize all forms of shared intentionality. Various dimensions along which
different forms of shared intentionality vary need to be taken into account. For instance,
cognitive representations and processes related to shared intentionality exist on various levels
of complexity, ranging from conscious intentions of the form “I intend that we spend the rest of
our lives together” to largely unconscious processes of alignment and coordination that occur
when partners sit side-by-side in rocking chairs or play drums together (Sebanz, Knoblich &
Butterfill, 2011). Different forms of shared intentionality vary dramatically with regard to both
their scale, ranging from two people carrying a table together to nation states of millions of
people fighting wars, and their scope, ranging from transient projects such as shaking hands to
long-lasting projects such as those implied by jointly adopting a child. From a psychological
point of view, such different forms of shared intentionality as, say, two young children rolling
a ball back and forth together and two adults writing a paper together, will vary dramatically
with regard to the participants’ representations of sharedness behind this activity. Finally,
varieties of shared intentionality differ to the degree to which they involve conventionality and
rule following: conventions and rules are necessary for marrying or playing chess, but not for
carrying a table together (e.g. Searle, 1969, 1995). Recognizing and charting diversity is
relatively easy. What is still missing – and what the Diversity Challenge demands – is a
systematic, comprehensive and differentiated conceptual apparatus which brings into view what is
common to all forms of shared intentionality, what sets them apart from merely individual
intentionality, and how various forms relate to each other. Meeting the Diversity Challenge
immediately raises three further, no less formidable challenges.
\emph{The Emergence Challenge}
The first challenge concerns emergence: What is involved in the transitions between different forms of shared intentionality? What are the ontogenetic motors, the neural underpinnings, and the phylogenetic mechanisms underlying these transitions?
\emph{The Interface Challenge}
A second challenge that follows from meeting the Diversity Challenge arises from the fact that different forms of shared intentionality will involve representations shared at various levels. These representations typically differ in their formats (from conceptual to motor), specificity of information, exact timing, and availability to consciousness. Thus, we need to understand how shared representations at different levels interface, and how the processes at play at each level interact to yield (or reliably give the appearance of) integrated systems of shared representations. Philosophy and cognitive science have made significant progress concerning such interface issues for individual intentionality (e.g. Pacherie, 2008, 2011; Butterfill & Sinigaglia, 2014). The challenge is now to do the same for shared intentionality.
\emph{The Integration Challenge}
So far the challenges have concerned shared intentionality narrowly. But shared intentionality is arguably the key to understanding a manifold of cognitive capacities. Here we focus on social cognition and communication. How are the different forms of shared intentionality variously integrated with abilities to track others’ mental states and actions, and to communicate with others?
Sharing a Smile
\section{Sharing a Smile}
\section{Sharing a Smile}
[THIRD POINT: smiling is a goal-directed action, the goal of which is to smile that smile]
My topic is sharing a smile. But first think about ordinary, individual actions like genuine smiles.
What distinguishes a genuine smile from a muscle spasm or the exhalation of wind?
I want to suggest that it's this: the smile is a goal-directed action where the goal is to simile that smile.
But why think of the smile as goal-directed? Because smiling the smile requires considerable motor coordination: it’s not a matter of simple muscle contractions but more like the production of a phonetic gesture where context affects what is needed to realise the smile.
Further, like grasping an object or articulating a particular phoneme, it is an action that can be realised by different bodily movements in different contexts.
This is why I put slides of two quite different but both genuine smiles.
[Objection:]
Now you might say that the smile can't be goal-directed because is isn't explicable by appeal to belief, desire and intention
This is because the genuine smile is spontaneous and not something that can be produced at will (although it could probably be inhibited, at least to some extent); after all, this is what distinguishes the genuine from the polite smile.
\footnote{
From web source: The Duchenne smile involves both voluntary and involuntary contraction from two muscles: the zygomatic major (raising the corners of the mouth) and the orbicularis oculi (raising the cheeks and producing crow's feet around the eyes). The zygomatic major can be voluntarily contracted but many people can't voluntarily contract the orbicularis oculi muscle.
}
So now we might be tempted by the view that a smile is merely caused by an emotion in the way that gasses can cause you to burp.
[Reply:]
Maybe there are smiles like this, but some genuine smiles are sustained.
And what sustains them is a process of controll
How could this be if such smiles are not consequences of beliefs, desires and intentions?
I think a reasonably natural view here is to think that part of what makes an event a smile, a goal-directed action and not just a muscle spasm caused by excess wind, is the way that motor control is involved. Specifically, the genuine smile will involve a motor representation of the outcome, the smile, and this motor representation will lead to movements by way of planning-like motor processes.
But you don't have to buy this to agree with me.
All you have to accept is that actions like some smilings can be goal-directed and controlled even in the absence of relevant beliefs, desires and intentions.
I think smiles fall into the category of actions like graspings, reachings and gesturings which are goal-directed but do not necessarily involve intention.
So far, then, I've suggested that smiling is a goal-directed action, the goal of which is to smile that smile.
Now imagine a situation where a single individual encounters and event (a clown’s falling) which causes amusement which causes her to smile
Note that the smile also modulates the emotion; if, for example, she supressed the smile, the quality of her amusement would change.
How could we gain insight into the fine-grained dynamics of others’ emotions?
How could we ever appreciate the unfolding of another’s grief, or the way their engagement leads to an explosion of ecstasy at the climax of a concert?
Part of the answer is obvious: by being there, with them.
[Not that this is the only possibility --- in some cases we might be told.]
But how exactly does being there, in the same situation help?
Merely being in the same situation is surely not enough.
It’s not enough that we each experience amusement, grief or ecstasy.
After all, individuals are different. Different individuals’ feelings don’t unfold in the same way just because they are in the same situation.
It’s just here that collective intentionality is relevant.
\textbf{What is involved in sharing a smile?}
Minimally, I think there have to be two kinds of connection between us for us to share a smile.
First, the way your smile unfolds is shaped by how mine unfolds and conversely.
I also suppose that our smiles can be minutely coordinated with each other.
But it’s not just that our smiles are interdependent in this way ...
It’s also that each of our smiles is shaping the way our amusement unfolds.
So the way your amusement unfolds is being controlled by, and controlling, the way mine unfolds.
In sharing a smile, we are emotionally locked together.
[*todo: remove motor stuff for this talk! Also: don't lose sight of idea that control is a way of knowing.]
[*todo: need slide with control arrows highlighted (my emotion controls yours).]
[*Structure:
(i) I know because my emotion controls yours;
(ii) But if my emotion controls yours, how can yours be amusement at the clown's falling? because control is partial, and reciprocal;
(iii) But the mere fact of control isn't enough for knowledge; rather, control must show up in experience somehow.
After all, for all I have said so far, we might, in sharing a smile, be unaware that our emotions are locked together.
(iv) There must be an experience that is distinctive of sharing a smile.
(iv) Note that I don’t want to say that someone who is sharing a smile needs to understand the situation in the way I’m describing it.
All I'm claiming is that the fact of reciprocal control somehow affects our awareness.
(v) It may affect in our awareness insofar as we are sensitive to contingencies between our own actions' and others' actions,
and between our actions and the causes of them.
(vi) So my position is this: the reciprocal control justifies each agent in making judgements about how the others' amusement is unfolding,
and this justification is at least indirectly available to the agents by virtue of their having experiences characteristic of sharing a smile.
]
Our being emotionally locked together means that
to a significant extent I am feeling what you are feeling,
that the way my amusement is unfolding matches they way your amusement is unfolding.
So if you know how your own amusement is unfolding and you know that we are emotionally locked together,
you can know much about how my amusement is unfolding.
So joint expressions of emotion like sharing a smile have the potential to enable us to know not just that others are amused but how their amusement is unfolding.
But the fact of reciprocal control (which means our emotions are locked together) together doesn’t all by itself mean that we can know how each other’s emotions are unfolding.
After all, for all I have said so far, we might, in sharing a smile, be unaware that our emotions are locked together.
Now you might think this sounds implausible because its hard to imagine sharing a smile without an experience that is distinctive of sharing a smile.
And it might be natural to describe this experience as an experience of sharing.
But even if that is correct, it’s necessary to say exactly why someone who is sharing a smile is in a position to know things about how the other’s emotion is unfolding.
I don’t want to say that interaction only helps if you know that your emotions are locked together.
That is, I don’t want to say that someone who is sharing a smile needs to understand the situation in the way I’m describing it.
But minimally the fact of reciprocal control must somehow feature in our awareness.
[*The idea in outline:
\begin{enumerate}
\item the ways our amusements unfold is locked together
\item this is in part because a single motor plan has two functions, production of your smile and prediction of my smile
\item the single motor process means that we might experience being locked together in some way (not that our emotions are locked together but that our actions are, in something like (but not exactly) we experience actions when seeing ourselves in a mirror or on CCTV (check Johannes’ discussion of this)).
\end{enumerate}
]
Here I want to offer a wild conjecture.
In joint expressions of emotion there is a single motor plan with two functions,
production and prediction.
The motor plan both produces your own smile and enables you to predict the way the other’s smile will unfold.
[*missing step about monitoring and experience. (The Haggard idea: motor planning can give rise to experiences concerning one's own actions \citep{Haggard:2005sc}.)]
Because your plan has this dual function, your experience of the other’s (my?) smile is special.
From your point of view, it is almost as if the other is smiling your smile.%
\footnote{
Joel caricatured this idea seeing me eating fruit: ‘it’s almost as if I’m eating that fruit.’
}
This means that sharing a smile has characteristic phenomenology.
This odd phenomenological effect means that in sharing a smile we can each think of the situation almost as if there were a single smile.
And almost as if there were a single state of amusement.
(In thinking of the situation like this it is important that we have a subject-neutral conception of the amusement and an agent-neutral conception of the smile.%
\footnote{
Tom Smith asked about this. I clarified that I wasn’t suggesting there was a state of amusement which is ours, nor that the subjects are thinking of the situation in this way.
That’s the point of the appeal to subject-neutral amusement.
It’s a partial model of the situation.
}
[*Here I think I’m shifting back from the perspective of the participants in sharing a smile to the perspective of the theorist.
Probably what I should say is, first, that a theorist can think of the situation in this way and use this to argue, second, that there is a simple, partial conception of the situation that doesn’t require understanding reciprocal control and interlocking emotions but is sufficient for each smiler to have knowledge of the way the other’s emotion unfolds.]
So my suggestion is that in sharing a smile you experience my smile almost as if it were yours (or: you experience me almost as if I were smiling your smile),
and so you might also experience our situation almost as if it involved a single state of amusement.
It's more like we each plan a single smile.
But---to reply to the objection---these plans have a dual function.
Your plan both produces your own smile and enables you to simulate---to experience---my smile.
And likewise for my plan.
The interdependence of our smilings means that we could each think of the situation as if it were one in which a single state of amusement were responsible for our actions.