Please use this identifier to cite or link to this item: http://hdl.handle.net/1893/36632
Appears in Collections:Computing Science and Mathematics Journal Articles
Peer Review Status: Refereed
Title: Canonical cortical graph neural networks and its application for speech enhancement in audio-visual hearing aids
Author(s): Passos, Leandro A
Papa, João Paulo
Hussain, Amir
Adeel, Ahsan
Contact Email: ahsan.adeel1@stir.ac.uk
Keywords: Cortical circuits
Canonical correlation analysis
Multimodal learning
Graph neural network
Prior frames neighbourhood
Positional encoding
Issue Date: 28-Mar-2023
Date Deposited: 28-Feb-2025
Citation: Passos LA, Papa JP, Hussain A & Adeel A (2023) Canonical cortical graph neural networks and its application for speech enhancement in audio-visual hearing aids. <i>Neurocomputing</i>, 527, pp. 196-203. https://doi.org/10.1016/j.neucom.2022.11.081
Abstract: Despite the recent success of machine learning algorithms, most models face drawbacks when considering more complex tasks requiring interaction between different sources, such as multimodal input data and logical time sequences. On the other hand, the biological brain is highly sharpened in this sense, empowered to automatically manage and integrate such streams of information. In this context, this work draws inspiration from recent discoveries in brain cortical circuits to propose a more biologically plausible self-supervised machine learning approach. This combines multimodal information using intra-layer modulations together with Canonical Correlation Analysis, and a memory mechanism to keep track of temporal data, the overall approach termed Canonical Cortical Graph Neural networks. This is shown to outperform recent state-of-the-art models in terms of clean audio reconstruction and energy efficiency for a benchmark audio-visual speech dataset. The enhanced performance is demonstrated through a reduced and smother neuron firing rate distribution. suggesting that the proposed model is amenable for speech enhancement in future audio-visual hearing aid devices.
DOI Link: 10.1016/j.neucom.2022.11.081
Rights: This is an open access article distributed under the terms of the Creative Commons CC-BY license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. You are not required to obtain permission to reuse this article.
Licence URL(s): http://creativecommons.org/licenses/by/4.0/

Files in This Item:
File Description SizeFormat 
1-s2.0-S0925231222014758-main.pdfFulltext - Published Version1.53 MBAdobe PDFView/Open



This item is protected by original copyright



A file in this item is licensed under a Creative Commons License Creative Commons

Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

The metadata of the records in the Repository are available under the CC0 public domain dedication: No Rights Reserved https://creativecommons.org/publicdomain/zero/1.0/

If you believe that any material held in STORRE infringes copyright, please contact library@stir.ac.uk providing details and we will remove the Work from public display in STORRE and investigate your claim.