PhD at Risø Danish national laboratory
On speech recognition during anaesthesia
Alexandre Alapetite personal page 2003-11 to 2007-12


My PhD follows a 6-month traineeship about speech recognition in plane cockpits, which was the final part of my Postgraduate Diploma in Artificial intelligence, Pattern recognition, Robotics
This PhD is financed and integrated into the ADVISES European research training network (HPRN-CT-2002-00288), which is about Analysis Design and Validation of Interactive Safety-critical and Error-tolerant Systems.
The starting point of my thesis was to study how alternative inputs — such as voice recognition — could improve the traditional human-machine interfaces based on buttons, mousses or keyboards. The combination of two or more different inputs, called multi-modality, was also taken into account.

[PDF] Download thesis

After my defence on 4 December 2007, my work continued as a researcher at the Technical University of Denmark (DTU).



This PhD thesis in human-computer interfaces (HCI, informatics), studies the case of the anaesthesia record used during medical operations and the possibility to supplement it with speech recognition facilities.

Problems and limitations have been identified with the traditional paper-based anaesthesia record, but also with newer electronic versions, in particular ergonomic issues and the fact that anaesthesiologists tend to postpone the registration of the medications and other events during busy periods of anaesthesia, which in turn may lead to gaps and inaccuracies in the anaesthesia record.

The thesis first studies the role and the importance of the anaesthesia record as a work tool during operations. Related work procedures are also described in detail. Some small-scale surveys are conducted, which corroborate the observations mentioned above.

Supplementing the electronic anaesthesia record interface with speech input facilities is proposed as one possible solution to a part of the problem. A discussion paper made with a socio-ergonomist describes some of the short and long-term consequences if such an idea is to be deployed.

The thesis then investigates the possibilities and technical limitations of the most widely used speech recognition system in Danish for medical applications. Of particular interest is the deleterious effect of various background noises found in medical operation theatres. While loud noises in the operating room can have a predominant negative effect, recognition rates for common noises are found to be only slightly below performances obtained in an office environment. Other factors have a major impact as well, such as the words to be recognised, participants, the type of speech recognition system (natural or constrained language) and the type of microphone. Finally, a proposed redundant architecture succeeds in improving the reliability of the recognitions.

After that, a prototype of electronic anaesthesia record interface with speech input facilities is developed on the basis of the knowledge gained at the previous steps as well as interviews with some anaesthesiologists.

The next phase is based on full-scale anaesthesia simulations involving the prototype to compare it with the traditional touch-screen and keyboard interface. Inspired from the mathematical queuing theory, a special metric for characterizing differences in mental workload is developed to compare the two interfaces. Results show that the speech interface leads to much shorter registration delays and to a greater accuracy of the information compared to the traditional electronic interface.

The simulation-based experiments also permitted the testing of some speech input strategies chosen for the prototype (hands-free vocal interface activated by a keyword; combination of command and free text modes), which were successful, even with the ambient noise. Speaking to the system while working appeared feasible, although improvements in speech recognition technologies are still necessary.

The above experiments form the main results of the thesis. They are followed up by secondary investigations.

An opportunity is taken to study via questionnaires and other indicators the deployment, acceptance and success of a speech recognition system – sharing technological similarities with the above-mentioned prototype – used to produce patient records in a Danish hospital. Physician satisfaction with the use of the system is modest, yielding a posteriori an approximately even balance between those in favour of, and those against the introduction of speech recognition to transcribe medical record. One of the main reasons for users’ dissatisfaction is the new work procedure introduced simultaneously with the speech recognition technology, which requires physicians to spend more time on producing the records.

In order to get more objective data on the effect of introducing this speech recognition system on the quality of the medical records, a blinded comparison is done between former and new work procedures. The results show that records produced with the new work procedures involving possible use of speech recognition contain more errors than the ones produced with the former method where a secretary is in charge of the transcription. However, the difference between speech recognition and secretary based transcription is relatively small in terms of number of transcription errors, and does not apply to records that follow a fixed and recurrent pattern. These results may therefore not be construed as showing that speech recognition does not bring advantages when considering the gains, e.g. in total turnaround time.

The conclusion is that speech recognition is a very interesting modality that should be used when appropriate and only for tasks for which it is efficient when compared to other alternatives.


See newer publications

Alexandre Alapetite, Henning Boje Andersen, Morten Hertzum, 2008   [PDF]
“Acceptance of speech recognition by physicians: A survey of expectations, experiences, and social influence”
International Journal of Human-Computer Studies, August 2008 online, January 2009 67(1):36-49, doi:10.1016/j.ijhcs.2008.08.004
Long version (but older) in my thesis, pages 149-185.
Alexandre Alapetite, 2007   [PDF]
“On speech recognition during anaesthesia”
PhD thesis, Roskilde University 2007, ISSN:0109-9779 (RU/CS/RR #119), hdl:1800/3109
Alexandre Alapetite, 2007   [PDF]
“Speech recognition for the anaesthesia record during crisis scenarios”
International Journal of Medical Informatics, September 2007 online, July 2008 77(7):448-460, doi:10.1016/j.ijmedinf.2007.08.007
Long version in my thesis, pages 105-145.
Alexandre Alapetite, 2006   [PDF]
“Impact of noise and other factors on speech recognition in anaesthesia”
International Journal of Medical Informatics, December 2006 online, January 2008 77(1):68-77, doi:10.1016/j.ijmedinf.2006.11.007
Long version in my thesis, pages 59-99.
Ioannis Dokas & Alexandre Alapetite, 2006   [PDF]
“A view on the Web engineering nature of Web based expert systems”
Poster paper in the proceedings of ICSOFT’2006, the 1st International Conference on Software and Data Technologies, 11-14 September 2006, Setubal, Portugal, pages 280-283.
“A development process meta-model for Web based expert systems: the Web engineering point of view”
Long version in Risø-R-1570(EN), ISBN:87-550-3536-1, Risø National Laboratory, October 2006.
Henning Boje Andersen & Alexandre Alapetite, 2006   [PDF]
SEE – Sight Effectiveness Enhancement, Results of the Aeronautical Evaluation”
SEE (European project) deliverable D6.3, December 2005.
Risø-R-1573(EN), ISBN:87-550-3541-8, Risø National Laboratory, November 2006.
Alexandre Alapetite, 2005   [PDF]
Content accessibility of Web documents: Overview of concepts and needed standards
In COGAIN (European Network of Excellence) deliverable D6.1 “State of the art report of evaluation methodology”, pages 28-34, September 2005.
Long version in Risø-R-1576(EN), ISBN:87-550-3546-9, Risø National Laboratory, October 2006.
Alexandre Alapetite & Vincent Gauthereau, 2005   [PDF]
Introducing vocal modality into electronic anaesthesia record systems: possible effects on work practices in the operating room
In the proceedings of EACE’2005, annual conference of the European Association of Cognitive Ergonomics, 29 September - 1 October 2005, Chania, Crete, Greece; Section 2 on “Research and applications in the medical domain”, pages 189-196. University of Athens.
In the ACM International Conference Proceeding Series; Vol. 132, pages 197-204, ISBN:9-60254-656-5.
Updated version in my thesis, pages 37-55.
Alexandre Alapetite, 2005
“Voice recognition in multimodal systems: the case of anaesthesia patient journal”
Position paper in the proceedings of the first ADVISES Young Researchers Workshop, 7 January 2005, Liège University, Belgium
Risø-R-1516(EN), ISBN:87-550-3443-8, pages 5-9, Risø National Laboratory, April 2005.
Reworked version in my thesis, pages 23-34.

See older publications





Tutorial on human error in accidents and human behaviour during evacuations   24 August 2006

This tutorial was presented by Professor Chris Johnson (University of Glasgow, UK) at Risø National Laboratory. During the first session “Is Human Error a Major Cause of Incidents and Accidents?” was exposed a review of the causes and contributory factors for major transportation accidents in the USA. The second session, entitled “Can We Learn Anything by Simulating Human Behaviour during Evacuations from Public Buildings”, was an interactive presentation using a series of simulations to demonstrate models of human behaviour during emergency situations.

Course in Management in Science and Innovation   3-6 October, 28 November 2005

This PhD course was organised by Kristian Borch, from Risø, technology scenarios. Boundaries between university and industrial science, and between basic and applied research, are disappearing. Scientists have to produce basic scientific knowledge, to be innovative and transparent. Introducing the concept of innovation system on the course, science and innovation has been discussed on three levels: the national level, the organisational level, and the individual level. This course was equivalent to 5 ECTS.

Design Science in IT   13-14 October 2005

The course was organized by Richard Baskerville (Georgia State University, USA) and Jan Pries-Heje (IT University of Copenhagen, Denmark). A number of outstanding and influential papers on design science were presented and discussed; each of the participating students presented their research questions and discussed these in relation to design research. Furthermore, some cases and examples of design research were presented and critiqued. This course yields 2 ECTS.

Under the hood of advanced writing systems   30 May - 3 June 2005

The course, organized by John Paulin Hansen (IT University of Copenhagen, Denmark) in conjunction with the camp 2005 of the COGAIN European Network of Excellence, has given an introduction to alternative and mobile writing systems and the language technology applied in the systems. Methods for conducting user-performance evaluations of writing systems have been demonstrated. Participants were involved in specifying common design principles for writing systems, to guide the development of writing systems for gaze interaction. Teachers on the course were Scott MacKenzie (York University, Canada); Anders Sewerin Johansen (IT University of Copenhagen, Denmark); David J.C. MacKay, Chris Ball, Keith Vertanen, Phil Cowans (University of Cambridge, UK); Michael Kühn (University of Koblenz and Landau, Germany). This course yields 2 ECTS.

Tutorial on Petri-Nets and Cooperative Objects   25-27 Mai 2005

This tutorial was organised by Philippe Palanque and Bastiaan Schupp at the University of York, UK. Basic and high-level Petri-Nets, and Cooperative Objects have been introduced. The PetShop tool has been used for exercices. This tutorial was equivalent to 2 ECTS.

Gaze-Based Human-Computer Interaction   28-29 April 2005

This human-computer interaction tutorial, was organised by COGAIN at the Politecnico di Torino. Professor Kari-Jouko Räihä and Päivi Majaranta presented some theory about eye-tracking and applications for eye-typing, providing guidelines for eye gaze as an input device for HCI.

Organizational and technological changes and safety   4-7 January 2005

This workshop organised by Anne-Sophie Nyssen at the University of Liège, Belgium, was about human factors integration in the design and the management of safety critical systems; what has been learned and what has been made after major accidents. This workshop was equivalent to 2 ECTS.

Context-aware computing   30 November 2004

This academic conference, arranged by Crossroads Copenhagen in relation with the laboratory for Context-dependent Mobile Communication of the IT University of Copenhagen, was focusing on context-aware computing, including social and technical aspects, interaction and modelling.

An Introduction to Research in Information Systems (IS)   5 days in November 2004

This course was an overview doctoral level seminar arranged by Suprateek Sarker at the department of informatics of Copenhagen Business School. The course provided the participants with a broad coverage of the nature of the Information Systems discipline, and an exposure to the emerging norms/practices related to the use and adaptation of theories and methodologies (primarily qualitative) in this field. This course yields 5 ECTS.

Finite Metric Spaces and their Algorithmic Applications   30 August - 1 September 2004

This course, from Moses Charikar and Piotr Indyk at the IT University of Copenhagen, was a introduction of a number of tools and techniques based on finite metric spaces and distance preserving mappings, and their algorithmic applications (approximation algorithms, etc.). This course yields 1 ECTS.

IFIP World Computer Congress   23-26 August 2004

In Toulouse (France), I attended the HESSD Working Conference on Human Error, Safety and System Development. I attended also the topical day on Multimodal Interaction, about Design and Development of Multimodal User Interfaces and Domains of Multimodality. This congress was equivalent to 1 ECTS.

Task Modelling and Design of Multi-Device Interfaces   20 May 2004

This interactive tutorial, from Fabio Paternò (ISTI Institute of CNR in Pisa, Italy), was a presentation of task modelling for multi-device interfaces, migratory interfaces. It was followed by some exercises with TERESA software. This tutorial was equivalent to 1 ECTS.

Putting Humans into Control   8-12 March 2004

The topic of the course organized by John Paulin Hansen (IT University of Copenhagen, Denmark) ranged from fundamental issues associated with the nature of mind, matter, and what matters; from basic questions about the dynamics of control systems, to practical questions associated with the implications for designing socio-technical systems. Teachers on the course were Professor John Flach (Wright State University, Ohio, USA), Max Mulder and Rene Van Passen (Delft University of Technology, The Netherlands). This course yields 2 ECTS.

Human error analysis, barriers and the safety modelling language   6-7 January 2004

This tutorial was organised by ADVISES at the University of Paderborn, Germany. It introduced concepts and methods that can be used to analyse socio-technical systems for their vulnerability to failure, and provides a practical introduction to the Safety Modelling Language, and design and typology of barriers. This tutorial was equivalent to 2 ECTS.