BCAM Scientific Seminar: Interpreting systems as agents

Under what circumstances can a system be interpreted to have beliefs and goals, and to act according to the beliefs in order to achieve the goals? How do such agency-related features relate to its physical/internal state? We try to answer this question using formal definitions of system, belief, goal, action, and interpretation.

For this, we recently proposed a notion of interpretation map, a function that maps the state of a Moore machine without outputs (the system) to a probability distribution representing its Bayesian beliefs about an external world. This function must obey a consistency condition ensuring that given an input the beliefs associated to the internal state update according to Bayes rule. In fact, this consistency is what turns the associated probability distributions into legitimate Bayesian beliefs. If we find a consistent interpretation map, we can use it to interpret the machine as a Bayesian reasoner.

This approach can be extended to interpretations not only in terms of beliefs but also in terms of goals and actions for Moore machines with outputs. To do this, we make use of the existing notion of partially observable Markov problems (POMDPs): we say that a system can be interpreted as a solution to a POMDP if it not only admits an interpretation map identifying its beliefs about the hidden state of a POMDP but also takes actions that are optimal according to those beliefs. The result is that we have a formal sufficient condition for when a Moore machine can be interpreted as a Bayesian agent. We believe that this is only a special case of agent interpretations and hope to extend both, the kinds of things we can interpret as agents (here only Moore machines) and the kinds of interpretations (here POMDP solutions).