Because this speech network is widely distributed, a choice needs to be made on the cortical areas where to extract and decode signals for a speech BCI. Several brain areas are involved in speech processing, forming a wide cortical network, classically modeled by a ventral and a dorsal stream. Such perspective is indeed supported by an increasing number of studies reporting encouraging performances in decoding speech utterances, including phones, words or even full sentences, from brain activity. BCI approaches could thus also be applied to control a parametric speech synthesizer in real-time in order to restore communication by decoding neural activity from speech processing brain areas. However, speech remains our most natural and efficient way of communication. In the case of severe paralysis including aphasia (e.g., locked-in syndrome), ways of communicating can be provided by BCI approaches, mostly through a letter selection or typing process. The movements of different effectors, like a computer mouse or a robotic arm, were successfully controlled in several BCI studies, with increasing precision. In the past decades, brain-computer interfaces (BCIs) have been developed in order to restore capabilities of people with severe paralysis, such as tetraplegia or locked-in syndrome. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Ĭompeting interests: The authors have declared that no competing interests exist. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.ĭata Availability: The BY2014 articulatory-acoustic dataset is available for download at (doi: 10.5281/zenodo.154083).įunding: This work was supported by the Fondation pour la Recherche Médicale and by the French Research Agency (ANR) under the Brainspeak project ( under grant No DBS20140930785. Received: FebruAccepted: AugPublished: November 23, 2016Ĭopyright: © 2016 Bocquelet et al. PLoS Comput Biol 12(11):Įditor: Gabriel Mindlin, University of Buenos Aires, ARGENTINA In conclusion, these results open to future speech BCI applications using such articulatory-based speech synthesizer.Ĭitation: Bocquelet F, Hueber T, Girin L, Savariaux C, Yvert B (2016) Real-Time Control of an Articulatory-Based Speech Synthesizer for Brain Computer Interfaces. We found that real-time synthesis of vowels and consonants was possible with good intelligibility. A short calibration period was used to compensate for differences in sensor positions and articulatory differences between new speakers and the reference speaker. Then, to anticipate future BCI applications, we further assessed the real-time control of the synthesizer by both the reference speaker and new speakers, in a closed-loop paradigm using EMA data recorded in real time. In offline mode, highly intelligible speech could be obtained as assessed by perceptual evaluation performed by 12 listeners. This DNN is then used in both offline and online modes to map the position of sensors glued on different speech articulators into acoustic parameters that are further converted into an audio signal using a vocoder. ![]() The articulatory-to-acoustic mapping is performed using a deep neural network (DNN) trained on electromagnetic articulography (EMA) data recorded on a reference speaker synchronously with the produced speech signal. ![]() This synthesizer converts movements of the main speech articulators (tongue, jaw, velum, and lips) into intelligible speech. We present here an articulatory-based speech synthesizer that can be controlled in real-time for future BCI applications. To reach this goal, a prerequisite is to develop a speech synthesizer producing intelligible speech in real-time with a reasonable number of control parameters. Restoring natural speech in paralyzed and aphasic people could be achieved using a Brain-Computer Interface (BCI) controlling a speech synthesizer in real-time.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |