Simulators of associative learning models
In collaboration with the Centre for Computational and Animal Learning Research we have developed computational models of associative learning and reinforcement learning with an emphasis on their implementation in (platform independent) software simulators as essential tools in the cycle of theory formation and refinement. This includes simulators of existing models such as the seminal Rescorla-Wagner's as well as extensions and new models that introduce novel representational formalisms and learning rules -for example, our Serial and Simultaneous Configural-Cue Compound Stimuli Representation for Temporal Difference Learning Model (SSCC TD) and the Double Error Dynamic Asymptote Model (DDA). We aim at maximising and exploiting the full predictive power of the error-correction learning paradigm, and to accommodate a wide range of behavioural and neural experimental findings.
The simulators are accessible via this page:
Single Shot Model (SSM) for COVID-19 detection
An out-of-distribution detection method for images that combines density and restoration-based approaches using Vector-Quantized Single Shot Model (SSM) segments lesions in chest CT scans and predicts the condition of the patient at the scan slice level (COVID-19, Common Pneumonia, Normal). SSM extends the state-of-the-art instance segmentation Mask R-CNN model to make global (image class) predictions. The Region of Interest (RoI) module consists of two parallel branches (instance segmentation and classification) that adapt to each problem. Segmentation branch outputs predictions at an instance level (separate lesions), classification branch outputs the distribution of ranked regional predictions, from which the model learns the class of the image. The final model with just 8.27M parameters has COVID-19 sensitivity of 93.16%, F1 score of 96.76% and an average segmentation precision of 42.45% (main MS COCO 2017 criterion). The classification part of the model was trained on a fraction of the data (3K images) and evaluated on the full test split (21K images). Weights are provided for evaluation and further modeling.
The Single Shot Model code is accessible via this page:
Simple-playgrounds
Simple-playgrounds is a library for quickly designing 2D environments where agents can move around and interact with objects.
The game engine includes simple physics, such as collision and friction.
Agents can act through continuous movements and discrete interactive actions. They perceive with realistic first-person view sensors, top-down view sensors, and semantic sensors.
Simple-playgrounds is easy to handle, allows very fast design of AI experiments and runs very quickly.
Simple-playgrounds is accessible via this page:
VQ-VAE Model for medical imaging
An out-of-distribution detection method for images that combines density and restoration-based approaches using Vector-Quantized Variational Auto-Encoders (VQ-VAEs). The VQ-VAE model learns to encode images in a categorical latent space. The prior distribution of latent codes is then modelled using an Auto-Regressive model. This approach enables the estimation of both sample and pixel-wise anomaly scores. This method was tested on medical imaging datasets, including brain MRI and abdominal CT scans.
The VQ-VAE model code is accessible via this page:
DMARL for Load Frequency Control
This is an application of Multi-Agent Reinforcement Learning (MARL) to approximate Load-Frequency Control (LFC), a classic power generation control problem, in a fully decentralized way. More precisely, we use Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to design and train a controller that keeps generation and demand balanced in a power electronics network in a cost-efficient way. The novelty of our approach is that the agents learn to operate in a close-to-optimal way without exchanging any type of information between them, while the state-of-the-art algorithms typically require central authorities to regulate the system.
Code is accessible via this page: