Shared Representations as Coordination Tools for Interaction

Abstract

Why is interaction so simple? This article presents a theory of interaction based on the use of shared representations as “coordination tools” (e.g., roundabouts that facilitate coordination of drivers). By aligning their representations (intentionally or unintentionally), interacting agents help one another to solve interaction problems in that they remain predictable, and offer cues for action selection and goal monitoring. We illustrate how this strategy works in a joint task (building together a tower of bricks) and discuss its requirements from a computational viewpoint.

Appendix A: The Interactive Strategy from a Computational Viewpoint

From a computational perspective, we can model each agent as a generative (Bayesian) system in which hidden (i.e., not visible) cognitive variables, beliefs (B) and intentions (I), determine the selection of actions (A) and then motor primitives (MP); in turn, they determine the agent’s overt behavior, which becomes part of the world state (S). As the world state is only partially observable, we distinguish observables (O) from the full, unobservable state of the world S.⁹ Note that, different from what shown in Fig. 4, we assume that beliefs are not only relative to the task, but are more generally contextual knowledge that influences intention and action selection.

1.1 A.1 Action Planning and Execution

For the sake of simplicity, we can assume that each action A executed at time t achieves a certain action goal at time t + n, or in other terms it determines a future goal state S _t + n (in doing this, we skip the level of motor primitives MP that are involved in action performance). If we also assume that there is a way to map an agent’s intentions (I) into goal states S _t + n (e.g., the intention of realizing a tower of red bricks can mapped in a state of the world in which there are 5 stacked red bricks), then planning consists in the choice of an action, or a sequence of actions, conditioned to the agent’s belief and intentions, which (is expected to) realize the future goal state S _t + n (and typically, as a consequence, produce some reward, or even maximize reward), which can be done for instance with probabilistic planning methods (Bishop 2006).

In the passage from plans to action execution, however, the mapping from desired goal states to (sequences of) actions is typically ill-posed and difficult, and for this reason it has been proposed that the brain makes use of internal (inverse) models to solve it. In addition to that, during action execution internal (forward) models could be adopted as well to adapt the motor plan to the fast dynamics of the environment, in the cases in which feedback is too slow (Desmurget and Grafton 2000; Kawato 1999).

1.2 A.2 Prediction and State Estimation

As we have discussed, forward models serve (sensory and state) prediction during action execution. Formally, forward models permit to map state and action information into a (sensory or state) prediction at the next time step, or more than one time steps in the future (i.e., S, A → S _t + n ).

However, in dynamic environments, the effects of actions (S, A → S _t + n ) are not easy to determine, for many reasons: first, in general agents can only access the observable part of the environment (O) and not its “true” state (S); second, the environment changes over time (i.e., S _t is different from S _t + 1). Since actions can have different effects when executed in different contexts, and this can hinder the achievement of goals, a solution of this problem consists in estimating the true state of the environment (S) rather than acting based on O, and at the same time learning to predict the dynamics of the environment (S _t → S _t + 1). In generative Bayesian systems, the probabilistic inference P(S|O) can be done, for instance, via iterative methods such as Kalman filtering (Kalman 1960) (other, more sophisticated methods are necessary for non-linear cases).

1.3 A.3 Action Prediction, Understanding and Mindreading

In interactive scenarios, the situation is even more complex, since the behavior of other agents is an extra source of dynamics. Again, forward models can be used to predict the interactive dynamics and to adapt to them. However, remind that agents have only access to the observable part of the environment (O), and this makes prediction (and hence action planning and coordination) difficult. In analogy with state estimation, prediction accuracy can be ameliorated by estimating the “true” state (S _t ) behind its observable part (O). Now, the “true” state (S _t ) is determined by both environmental dynamics (S _t − 1 → S _t ) and actions (A) of performed by the other agent(s). Therefore, in addition to modeling the environmental transitions (S _t − 1 → S _t ), it is also advantageous to model and predict the actions performed by the other agent(s). (Note that agents have at their disposal also information of what parts of the current state are produced by their own past actions, therefore can “cancel out” the self-produced part from the stimuli and the explicandum Blakemore et al. 1998; Frith et al. 2000.)

More in general, by noting (intentionally or unintentionally) that actions (A) executed by other agents are selected on the basis of its hidden cognitive variables, its beliefs (B) and intentions (I), an even more complete solution consists in inferring these cognitive variables, too. From a computational viewpoint, then, mindreading can be conceptualized as the estimation of the hidden (cognitive) variables of the other agent, rather than simply the observation and prediction of its overt behavior.

Computational methods for mindreading under this formulation have formal similarities with state estimation, but are more complex, because the generative architecture that generates the observables is richer. What makes mindreading easier is the assumption that the observed agent has a similar generative architecture as one’s own; indeed, constraining the structure of a generative model makes it easier to infer the value of its variables.

Under this formulation, mental state inference can be performed at different levels (e.g., estimation of actions, intentions or beliefs), can use whatever prior information available, such as for instance knowledge of the preference of an agent for certain actions and not others (i.e., P(A)), and whatever source of information available. Current implementations of mindreading range from inverse planning, which compares an agent’s actions against a normative, rational principle (Baker et al. 2009), to motor simulation (Demiris and Khadhouri 2005; Wolpert et al. 2003), which compares an agent’s actions with the putative effects of one’s own (derived by re-enacting one’s own motor system ‘in simulation’); see also Cuijpers et al. (2006).

Note that, in this process, motor contagion or the alignment of motor primitives in performer and observer agents is highly advantageous, in that another’s MP can be considered as “observations” and used to facilitate inference of the underlying beliefs, intentions and action goals that could have generated them. A second element that facilitates estimation is a consideration of what environmental affordances are available. Indeed, availability of affordances and other contextual information (e.g. where the interaction takes place) can be considered as factors that raise the prior probability P(A) of actions that are associated to given affordances and contexts. If this information becomes available to an observer, it can help in the inferential process.

1.4 A.4 Advantages of Using Shared Representations and the Interactive Strategy

Within our model, shared representations (SR) can be considered as the aligned subset of beliefs, intentions and actions of two (or more) agents, or in other terms the equivalence of a subset I ^agent1 with a subset of I ^agent2, a subset of B ^agent1 with a subset of B ^agent2, and a subset of A ^agent1 with a subset of A ^agent2. It is worth reminding that it is not necessary (and not even true) that all the cognitive variables are shared. This also entails that, if agents have meta-beliefs on what is shared, they can be different. First, only cognitive variables pertaining to task achievement are generally shared. Second, as shown in Fig. 3, representations can become shared during the interaction; an indeed from a computational viewpoint the guarantee of successful interactions is that (only) representations that afford coordinated patterns of action execution and prediction become aligned.

At the beginning of the interaction only some beliefs and intentions are shared, mainly because of past experience, reliance on a common situated context, and the recognition of social situations and agreements. During the course of interaction, both agents can perform actions that have as their goal changing SR rather than achieving goals defined in terms of external states of affairs. These are communicative actions that typically change another’s beliefs and intentions.

One advantage of using shared representations during action planning and observation (including planning and observation of communicative actions) is that they are a novel source of information, and one that is available to both agents, and reliable, since both agents put their effort in maintaining it and signaling when it has to be updated. By using shared representations, an agent can more easily predict and understand the actions of the other agents even without estimating the “true” state of the world or its “true” cognitive variables (i.e., perform P(A|SR) rather than P(A|O) or P(A|I, O)). In turn, this can be done either by appealing to a principle of rationality (i.e., what would be the best action given SR), similar to the method in Baker et al. (2009) (with the difference that not the inference is much more constrained than asking what is the best action in general), or by using forward search and motor simulation, similar to Demiris and Khadhouri (2005), Wolpert et al. (2003), and also by using SR as priors that bias the search.

Note also that, as suggested in Section 4.5, in order to implement the aforementioned form of planning it is not necessary to maintain a model of the other agent, or a model for each of the other agents, but only individual representations for action tied to task achievement, which can be used for both action planning and prediction of actions of the other agents.¹⁰

Shared representations give advantages also in the planning of communicative actions. As we have highlighted, the standard way of expressing communicative intentions under this formulation consists in violating what would be predictable given SR. Here, again, the inference of what is surprising given SR is easy and does not require any complex inferential mechanism, since the necessary source of information (the SR) is already available to the planner agent. In turn, choosing when and what to communicate (or how much to share) is more complex; indeed, most systems inspired to the social-aware strategy implicitly assume that it is necessary to share a lot of information, such as for instance my own and your part of a common plan, my short-term and long-term intentions, etc.

However, as suggested in Section 4.5, communicative actions that change SR can be interpreted as a form of “teaching” for the other agents how to generate good predictions, or in other words communicating it what is the value of SR that is more accurate in predicting P(A|SR). This gives rise to a mutual form of supervised learning. For this form of learning to be efficacious, the “learning episode” (i.e., the message) should not be casual, but selected on purpose by the other agent to be on time and maximally informative.

In this context, being “on time” means that I should communicate when I know that your future inference P(A|SR) would be wrong, or in other terms I know that the common ground is not sufficient for you to do good predictions. This criterion is used to decide when it is necessary that I perform a communicative action, so as that my actions and intentions are not misunderstood. (In addition to that, occasionally I change the SR as part of my pragmatic actions, for instance adding a blue brick rather than a red one. In this case, however, the communicative implicature is automatic and no further planning is needed.)

To select “what” to communicate, instead, the criterion is informativeness, and consists in lowering uncertainty and entropy of the SR (which, in our example of Fig. 4, is equivalent to lowering uncertainty on B) and raising the probability that the next predictions of the other agent will be more accurate. This entails that not necessarily I have to share with you all that I believe or intend to do, but only create a good ground for you to predict my actions well, to detect violations of the SR, and ultimately to conduct a successful interaction. Any additional information should be better not shared, since sharing it has a cost in terms of the success of the interaction.