My PhD follows a 6-month traineeship about speech recognition in plane cockpits,
which was the final part of my Postgraduate Diploma in Artificial intelligence, Pattern recognition, Robotics
This PhD is financed and integrated into the ADVISES European research training network (HPRN-CT-2002-00288),
which is about Analysis Design and Validation of Interactive Safety-critical and Error-tolerant Systems.
The starting point of my thesis was to study how alternative inputs — such as voice recognition —
could improve the traditional human-machine interfaces based on buttons, mousses or keyboards.
The combination of two or more different inputs, called multi-modality, was also taken into account.
After my defence on 4 December 2007, my work continued as a researcher at the Technical University of Denmark (DTU).
This PhD thesis in human-computer interfaces (HCI, informatics), studies the case of the anaesthesia record used during medical operations and the possibility to supplement it with speech recognition facilities.
Problems and limitations have been identified with the traditional paper-based anaesthesia record, but also with newer electronic versions, in particular ergonomic issues and the fact that anaesthesiologists tend to postpone the registration of the medications and other events during busy periods of anaesthesia, which in turn may lead to gaps and inaccuracies in the anaesthesia record.
The thesis first studies the role and the importance of the anaesthesia record as a work tool during operations. Related work procedures are also described in detail. Some small-scale surveys are conducted, which corroborate the observations mentioned above.
Supplementing the electronic anaesthesia record interface with speech input facilities is proposed as one possible solution to a part of the problem. A discussion paper made with a socio-ergonomist describes some of the short and long-term consequences if such an idea is to be deployed.
The thesis then investigates the possibilities and technical limitations of the most widely used speech recognition system in Danish for medical applications. Of particular interest is the deleterious effect of various background noises found in medical operation theatres. While loud noises in the operating room can have a predominant negative effect, recognition rates for common noises are found to be only slightly below performances obtained in an office environment. Other factors have a major impact as well, such as the words to be recognised, participants, the type of speech recognition system (natural or constrained language) and the type of microphone. Finally, a proposed redundant architecture succeeds in improving the reliability of the recognitions.
After that, a prototype of electronic anaesthesia record interface with speech input facilities is developed on the basis of the knowledge gained at the previous steps as well as interviews with some anaesthesiologists.
The next phase is based on full-scale anaesthesia simulations involving the prototype to compare it with the traditional touch-screen and keyboard interface. Inspired from the mathematical queuing theory, a special metric for characterizing differences in mental workload is developed to compare the two interfaces. Results show that the speech interface leads to much shorter registration delays and to a greater accuracy of the information compared to the traditional electronic interface.
The simulation-based experiments also permitted the testing of some speech input strategies chosen for the prototype (hands-free vocal interface activated by a keyword; combination of command and free text modes), which were successful, even with the ambient noise. Speaking to the system while working appeared feasible, although improvements in speech recognition technologies are still necessary.
The above experiments form the main results of the thesis. They are followed up by secondary investigations.
An opportunity is taken to study via questionnaires and other indicators the deployment, acceptance and success of a speech recognition system – sharing technological similarities with the above-mentioned prototype – used to produce patient records in a Danish hospital. Physician satisfaction with the use of the system is modest, yielding a posteriori an approximately even balance between those in favour of, and those against the introduction of speech recognition to transcribe medical record. One of the main reasons for users’ dissatisfaction is the new work procedure introduced simultaneously with the speech recognition technology, which requires physicians to spend more time on producing the records.
In order to get more objective data on the effect of introducing this speech recognition system on the quality of the medical records, a blinded comparison is done between former and new work procedures. The results show that records produced with the new work procedures involving possible use of speech recognition contain more errors than the ones produced with the former method where a secretary is in charge of the transcription. However, the difference between speech recognition and secretary based transcription is relatively small in terms of number of transcription errors, and does not apply to records that follow a fixed and recurrent pattern. These results may therefore not be construed as showing that speech recognition does not bring advantages when considering the gains, e.g. in total turnaround time.
The conclusion is that speech recognition is a very interesting modality that should be used when appropriate and only for tasks for which it is efficient when compared to other alternatives.
![[PDF]](../images/PDF.png){kind=small}
![[GeoURL]](../images/geourl.png){kind=small}
This tutorial was presented by Professor Chris Johnson (University of Glasgow, UK) at Risø National Laboratory. During the first session “Is Human Error a Major Cause of Incidents and Accidents?” was exposed a review of the causes and contributory factors for major transportation accidents in the USA. The second session, entitled “Can We Learn Anything by Simulating Human Behaviour during Evacuations from Public Buildings”, was an interactive presentation using a series of simulations to demonstrate models of human behaviour during emergency situations.
This PhD course was organised by Kristian Borch, from Risø, technology scenarios. Boundaries between university and industrial science, and between basic and applied research, are disappearing. Scientists have to produce basic scientific knowledge, to be innovative and transparent. Introducing the concept of innovation system on the course, science and innovation has been discussed on three levels: the national level, the organisational level, and the individual level. This course was equivalent to 5 ECTS.
The course was organized by Richard Baskerville (Georgia State University, USA) and Jan Pries-Heje (IT University of Copenhagen, Denmark). A number of outstanding and influential papers on design science were presented and discussed; each of the participating students presented their research questions and discussed these in relation to design research. Furthermore, some cases and examples of design research were presented and critiqued. This course yields 2 ECTS.
The course, organized by John Paulin Hansen (IT University of Copenhagen, Denmark) in conjunction with the camp 2005 of the COGAIN European Network of Excellence, has given an introduction to alternative and mobile writing systems and the language technology applied in the systems. Methods for conducting user-performance evaluations of writing systems have been demonstrated. Participants were involved in specifying common design principles for writing systems, to guide the development of writing systems for gaze interaction. Teachers on the course were Scott MacKenzie (York University, Canada); Anders Sewerin Johansen (IT University of Copenhagen, Denmark); David J.C. MacKay, Chris Ball, Keith Vertanen, Phil Cowans (University of Cambridge, UK); Michael Kühn (University of Koblenz and Landau, Germany). This course yields 2 ECTS.
This tutorial was organised by Philippe Palanque and Bastiaan Schupp at the University of York, UK. Basic and high-level Petri-Nets, and Cooperative Objects have been introduced. The PetShop tool has been used for exercices. This tutorial was equivalent to 2 ECTS.
This human-computer interaction tutorial, was organised by COGAIN at the Politecnico di Torino. Professor Kari-Jouko Räihä and Päivi Majaranta presented some theory about eye-tracking and applications for eye-typing, providing guidelines for eye gaze as an input device for HCI.
This workshop organised by Anne-Sophie Nyssen at the University of Liège, Belgium, was about human factors integration in the design and the management of safety critical systems; what has been learned and what has been made after major accidents. This workshop was equivalent to 2 ECTS.
This academic conference, arranged by Crossroads Copenhagen in relation with the laboratory for Context-dependent Mobile Communication of the IT University of Copenhagen, was focusing on context-aware computing, including social and technical aspects, interaction and modelling.
This course was an overview doctoral level seminar arranged by Suprateek Sarker at the department of informatics of Copenhagen Business School. The course provided the participants with a broad coverage of the nature of the Information Systems discipline, and an exposure to the emerging norms/practices related to the use and adaptation of theories and methodologies (primarily qualitative) in this field. This course yields 5 ECTS.
This course, from Moses Charikar and Piotr Indyk at the IT University of Copenhagen, was a introduction of a number of tools and techniques based on finite metric spaces and distance preserving mappings, and their algorithmic applications (approximation algorithms, etc.). This course yields 1 ECTS.
In Toulouse (France), I attended the HESSD Working Conference on Human Error, Safety and System Development. I attended also the topical day on Multimodal Interaction, about Design and Development of Multimodal User Interfaces and Domains of Multimodality. This congress was equivalent to 1 ECTS.
This interactive tutorial, from Fabio Paternò (ISTI Institute of CNR in Pisa, Italy), was a presentation of task modelling for multi-device interfaces, migratory interfaces. It was followed by some exercises with TERESA software. This tutorial was equivalent to 1 ECTS.
The topic of the course organized by John Paulin Hansen (IT University of Copenhagen, Denmark) ranged from fundamental issues associated with the nature of mind, matter, and what matters; from basic questions about the dynamics of control systems, to practical questions associated with the implications for designing socio-technical systems. Teachers on the course were Professor John Flach (Wright State University, Ohio, USA), Max Mulder and Rene Van Passen (Delft University of Technology, The Netherlands). This course yields 2 ECTS.
This tutorial was organised by ADVISES at the University of Paderborn, Germany. It introduced concepts and methods that can be used to analyse socio-technical systems for their vulnerability to failure, and provides a practical introduction to the Safety Modelling Language, and design and typology of barriers. This tutorial was equivalent to 2 ECTS.
![[PDF]](../images/pdf.gif){kind=small}
Download thesis