Research topics
Applied AI

Building a fair society

Quality control and motion correction for cardiac MR images

The effectiveness of a cardiac MR scan depends on the ability of the operator to correctly tune the acquisition parameters to the subject being scanned and on the potential occurrence of imaging artefacts such as cardiac and respiratory motion.

Together with colleagues at Imperial College, we developed a learning-based, fully-automated, AI-based quality control pipeline for cardiac MR short-axis image stacks. Our pipeline performs three important quality checks: 1) heart coverage estimation (to detect if the whole left ventricle, LV, has been imaged), 2) inter-slice motion detection (to estimate potential slice misalignments in the stack caused by respiratory motion and correct them), 3) image contrast estimation in the cardiac region (to estimate intensity differences between the LV cavity and myocardium). The pipeline was successfully validated on 3’000 cardiac MR scans of the UK Biobank dataset. It is currently being deployed on a larger UK Biobank cohort (~30’000) scans to detect potential factors associated with lower quality scans.


Featured publications:

Tarroni, G., Oktay, O., Bai, W., Schuh, A., Suzuki, H., Passerat-Palmbach, J. … Rueckert, D. (2019). Learning-based quality control for cardiac MR images. IEEE Transactions on Medical Imaging, 38(5), pp. 1127–1138. doi:10.1109/TMI.2018.2878509.

Tarroni, G., Oktay, O., Sinclair, M., Bai, W., Schuh, A., Suzuki, H. … Rueckert, D. (2018). A comprehensive approach for learning-based fully-automated inter-slice motion correction for short-axis cine cardiac MR image stacks.

Neural Networks for energy control

We are interested in the application of Recurrent Neural Networks to control connected converters in electric power systems that include smart grids, renewable energy resources and energy storage devices. In collaboration with the University of Alabama and the Missouri S&T, we have implemented a new Forward Accumulation Through Time (FATT) mechanism that is fed into the Levenberg-Marquardt algorithm to train the network, plus a novel input structure. This combination was compared against optimal controllers in close to real-life power converter switching environments. The excellent results indicate the feasibility of using our solution to approximate optimal control in practical applications


Featured publlication:

Fu, X., Li, S., Fairbank, M., Wunsch, D. and Alonso, E. (2015). Training Recurrent Neural Networks with the Levenberg–Marquardt Algorithm for Optimal Control of a Grid-Connected Converter. IEEE Transactions on Neural Networks and Learning Systems, 26(9), pp. 1900–1912. doi:10.1109/TNNLS.2014.2361267.

Semantic and instance segmentation

We developed a simple pipeline with a fully convolutional network (FCN) and a contour detector to predict coarse shapes of animals in videos taken at a beef cattle facility in Dublin county, Republic of Ireland. The model output masks of separate cows in every kth frame in a video, which were used to construct a dataset for training of deep convolutional networks (convnets).

We developed a Mask Splitter algorithm that predicts separate masks for overlapping objects (animals) in cattle facility videos with heavy partial occlusion. The algorithm is used as a module in an FCN that takes the score map of the image/frame and predicts separate score maps. The model was trained and evaluated on Pascal VOC 2012 and MS COCO 2017 datasets and benchmarked well against state-of-the-art instance segmentation algorithms like Mask R-CNN.


Interpretable supervised shape analysis from cardiac MR images

A wide range of cardiovascular diseases has been long associated with alterations in the shape of the LV myocardium (a phenomenon known as cardiac remodelling). However, current clinical methods to detect these diseases often rely on the visual assessment of cardiac images and on the estimation of global indices (e.g. cardiac volumes and ejection fraction). In recent years, deep learning approaches have achieved remarkable success in the classification of medical images, but typically lack interpretability in the feature extraction and decision processes, limiting their value in clinical diagnosis.

Together with colleagues at Imperial College, we exploited a 3D convolutional Variational Autoencoder (VAE) for the automated classification of 3D high-resolution LV cardiac shapes. Differently from classification networks, our approach allows the visualization and the quantification of the learned disease-specific remodelling patterns, which are of great clinical value.


Feature publication:

Biffi, C., Oktay, O., Tarroni, G., Bai, W., De Marvao, A., Doumou, G., Rajchl, M., Bedair, R., Prasad, S., Cook, S., O'Regan, D. and (2018). Learning interpretable anatomical features through deep generative models: Application to cardiac remodelling . In A.F. Frangi, J.A., Schnabel, C., Davatzikos, C. López Alberola and G. Fichtinger (Eds.), Proceedings of the 21st International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2018), pp. 464-471, 16-20 September, Granada, Spain. Lecture Notes in Computer Science, 11071. doi: 10.1007/978-3-030-00934-2_52.

Neural font style transfer

Generative and generative adversarial networks (GANs) have become a popular tool for neural style transfer. Usually one image (e.g. painting) is used as a style object and another image (e.g. a photograph) is a content object. We are working on sparse style transfer, using heavy metal logo as style and corporate logos as content. Loss functions (Gram matrix, mean squared error) are defined on deep layers and depending on the loss coefficient of that layer different features can be transferred. We are developing a new model that learns to predict these coefficients in order to balance the content and style features. Currently we employ only VGG16 to extract features, but we intend to explore more sophisticated architectures.


Deep learning for health and wellbeing

We propose an end-to-end deep learning architecture that automatically classifies a subject as ADHD or healthy control and demonstrates the importance of functional connectivity to increase classification accuracy and provide interpretable results. The proposed method, called DeepFMRI, is comprised of three sequential networks, namely, a feature extractor, a functional connectivity network, and a classification network. The model takes fMRI pre-processed time-series signals as input and outputs a diagnosis and is trained end-to-end using backpropagation. Experimental results on the publicly available ADHD-200 dataset demonstrate that this innovative method outperforms previous state-of-the-art. We are interested in applying the model to study other clinical conditions such as autism.


Featured publication:

Riaz, A., Asad, M., Alonso, E. and Slabaugh, G. (2019). DeepFMRI: End-to-end deep learning for functional connectivity and classification of ADHD using fMRI. To appear in Journal of Neuroscience Methods.