Work Package: WP6
Deliverable: This document reports on the progress of work on Deliverables 6.1, 6.2, 6.3.
This work itself is currently in its third year. All three deliverables have been met in part, and work on-going. In the early stages of planning we determined the need for pilot work, to establish basic methods and for training of the fellow. This entailed building a testbed for decoding neural responses to sound envelopes – an important component of the speech signal. This, and the secondment in the first year, delayed somewhat the planned deliverables. Things are however now on track as previously reported.
Deliverable 6.1, `Neural recordings of responses to the designed word set’. This deliverable has been met by the acquisition of a suitable dataset, with help from a former student in Dr Sumner’s group. The collection of the originally envisaged dataset was delayed due to constraints from the United Kingdom licensing system for animal experiments. However, the dataset acquired is proving adequate for testing the recognizer implemented in 6.2. Further experimental data will be collected should we find that this data is limited for our purposes with further testing.
Deliverables 6.2. ‘A single word speech recognizer working from neural responses’. A speech recognition system has been implemented, based on the HTK toolkit. This turned out to be a considerable endeavour. In this process we enlisted further help from an outside expert, a senior fellow on a different ITN (Dr Ning Ma, University of Sheffield). This model is able to successfully decode the response spoken words evoked by a physiological model of the cochlea, and is complete.
Deliverable 6.3. `Assessment of speech recognition using auditory cortical neurons’. We have tested 6.2 using a physiological model of the cochlea, and are currently evaluating its performance on a dataset of the response of neurons in the auditory midbrain to isolated elements of speech (consonants). This work is on-going, with the evaluation of different pre-processing algorithms to optimize performance.
The work (in particular 6.2) also covers milestone 20.
As all three deliverables are intrinsically linked, dissemination of the results are expected to be made through both publication routes and other methods to demonstrate the working setup once 6.3 is completed.
Contributors: Alban Levy, Chris Sumner, Steve Coombes, Ning Ma, Aristodemos Pnevmatikakis (UNOTT, Athens IT, University of Sheffield)