Research topics

Interacting and solving general problems

Learning sensory representations through self-supervised prediction

Artificial General Intelligence (AGI) aims to create agents that can learn how to interact with their environment by themselves. In order to act in an environment, an agent needs to perceive its surroundings. Historically, researchers in Robotics and Reinforcement Learning have relied on either hand-designed sensory representations or detection and classification algorithms to detect and represent what is in the vicinity of the agent. However, these rely on biased information provided by the human experimenters.

We want to go beyond classical approaches and propose to learn these representations. This field, named State Representation Learning, benefits from the advances in Deep Learning (DL) and Deep Reinforcement Learning (Deep RL) and allows to leverage sensorimotor information to learn these representations in a self-supervised way. By building a predictive model of how an agent’s sensors will be influenced by its motor actions, we can build abstract representations and learn common sense knowledge, naïve physics, as well as representation of the agent’s own body and the space around it. The goal of this project is to endow an agent with the mechanisms that allow the learning of these representations. We have an end-to-end approach where an agent’s cognitive architecture is modeled using Deep Neural Networks, taking inspiration from different scientific fields, notably Neuroscience and Cognitive Neuroscience.


Featured publications:

Laflaquiere, A. and Garcia Ortiz, M. (2019). Unsupervised Emergence of Egocentric Spatial Structure from Sensorimotor Prediction. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox and R. Garnett (Eds.), Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 8-14 December, Vancouver, Canada. Advances in Neural Information Processing Systems, 32.

Garcia Ortiz, M. and Laflaquiere, A. (2018). Learning Representations of Spatial Displacement through Sensorimotor Prediction. In Proceedings of the Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob 2018), (pp. 7-12), 17 September, Tokyo, Japan. doi: 10.1109/DEVLRN.2018.8761034.

Fully connected associative learning networks

We are interested in investigating fundamental principles in associative learning and how these may be expressed computationally. We are particularly keen on those phenomena that so far seem to go beyond an associative analysis, and in the conceptual and formal modifications of the theory that would allow it to integrate them.

With this aim in mind we develop fully connected architectures as general computational models of associative learning such as the Double error Dynamic Asymptote (DDA) that explicitly incorporates into a Pavlovian structure interactions between so called neutral stimuli, physically present or associatively retrieved from memory. We envisage that Deep Learning architectures may be instrumental in building up higher order cognitive structures, given their capacity to learn representations as hierarchies of abstractions, and to do so in a serial manner.


Featured publications:

Kokkola, N., Mondragón, E. and Alonso, E. (2019). A Double Error Dynamic Asymptote Model of Associative Learning. Psychological Review, 126(4), pp. 506–549. doi: 10.1037/rev0000147.

Mondragón, E., Alonso, E. and Kokkola, K. (2017). Associative Learning Should Go Deep. Trends in Cognitive Sciences, 21(11), pp. 822–825. doi: 10.1016/j.tics.2017.06.001.

Incremental learning of sensorimotor contingencies

Artificial General Intelligence is concerned with implementing mathematical models for an agent to learn autonomously by interacting with its environment. The agent starts with very limited a priori knowledge about the world, how to perceive it, and how to act in it. However, through the control of its motor commands and the reception of sensory signals, it is able to modify its internal state (learn models) that is helpful to solve tasks.

In order to learn how to act in the world, the agent interacts with its environment by exploring it and interacting with the different elements composing this environment. A good candidate for learning these models is self-supervised prediction, therefore a technical challenge lies in the algorithms used for learning these models. One main scientific goal of this project is to study the learning of representations (State Representations Learning) in the context of incremental learning.

We are interested in the notion of disentanglement, which corresponds to the representations of world entities by their latent independent factors of variations. We recently proposed a mathematical definition of disentanglement based on Group Theory and proved that disentanglement requires for an agent to act in its environment, validating our sensorimotor prediction approach.

With Hugo Caselles-Dupré (PhD student) in collaboration with Softbank Robotics Europe and Ensta ParisTech.

Feature publication:

Caselles-Dupré H, Garcia Ortiz M, Filliat D. Symmetry-Based Disentangled Representation Learning requires Interaction with Environments.In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox and R. Garnett (Eds.), Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 8-14 December, Vancouver, Canada. Advances in Neural Information Processing Systems, 32.

Variational principles for Artificial General Intelligence

Whether animals behave optimally is an open question of great importance, both theoretically and in practice. Attempts to answer this question focus on two aspects of the optimization problem, the quantity to be optimized and the optimization process itself. In this paper, we assume the abstract concept of cost as the quantity to be minimized and propose a reinforcement learning algorithm, called Value-Gradient Learning (VGL), that is guaranteed to converge to optimality under certain conditions. The core of the proof is the mathematical equivalence of VGL and Pontryagin’s Minimum Principle, a well-known optimization technique in systems and control theory. Given the similarity between VGL’s formulation and regulatory models of behaviour, we argue that our algorithm may provide AI with a variational technique in pursue of Artificial General Intelligence.


Feature publication:

Alonso, E., Fairbank, M. and Mondragon, E. (2015). Back to Optimality: A Formal Framework to Express the Dynamics of Learning Optimal Behavior. Adaptive Behavior, 23(4), pp. 206–215. doi: 10.1177/1059712315589355.