Back

Cogain
European Network of Excellence on Communication by Gaze Interaction
Alexandre Alapetite personal page 2003 - 2006

About

The Network of Excellence Cogain is a European project on communication by gaze interaction, mainly targeted to people with disabilities. It is part of the IST priority of the Cordis sixth framework programme. Reference [IST-2003-511598 (NoE)], September 2004 to August 2009.

I am participating in this project in parallel with my PhD thesis at Risø National Laboratory.

Personal work

June 2005: Accessibility of web documents: overview of concepts and needed standards

I have contributed to the deliverable D6.1 "State of the art report of evaluation methodology" (PDF 243KB); in particular, I have written the appendix A on Web accessibility.
The full version of this appendix has been published in October 2006: "Content accessibility of Web documents: Overview of concepts and needed standards" Risø-R-1576(EN) ISBN 87-550-3546-9.

July 2005: Dasher Live Wiki demo

Dasher is a revolutionary text-entry interface, using a navigation type metaphor. The Dasher team, the inference group of Cambridge University, participates in Cogain.

Thanks to the flexibility of Dasher, I have been able to build a prototype showing how Dasher can be used to enter formatted text using a wiki syntax converted on the fly in a RTF document that can be printed or used in most text processors (see the result in RTF).

Video of Dasher Live Wiki demo (Avi 3.15MB: XviD / MP3)

Hook to Dasher

This prototype is using the Microsoft Windows hook API to extract the text entered in Dasher by the user. This hook is written in Borland Delphi. Here is a modified sample in a mini Pascal program to illustrate the hook:

DasherHook.pas

program DasherHook;
uses Windows, Messages;

function HookDasher(): hwnd;
var dasher:hwnd;
begin
 dasher := FindWindow(PChar('Dasher'), nil);
 Result := FindWindowEx(dasher, 0, 'Edit', nil);
end;

function WmGetText(handle: hwnd): string;
var wmTextLength: Integer; text: string;
begin
 wmTextLength := SendMessage(handle, WM_GETTEXTLENGTH, 0, 0);
 Inc(wmTextLength);
 SetLength(text, wmTextLength);
 SendMessage(handle, WM_GETTEXT, wmTextLength, Longint(@text[1]));
 Result := text;
end;

var
 dasherEdit: hwnd;
 text: string;

begin
 dasherEdit := HookDasher();
 text := WmGetText(dasherEdit);
 Write(text);
end.

Microsoft Spy++ on Dasher to find the names of the handles

Wiki to RTF

The wiki text is then transferred to a C#.NET application thru a network socket. The wiki syntax is transformed into RTF using basic regular expressions. Here is a modified sample to illustrate this process:

LiveWiki.cs

using System;
using System.Text.RegularExpressions;
using System.Windows.Forms;

namespace CogainLiveWiki
{
 public class LiveWiki
 {
  ///<summary>List of regular expressions and their replacement</summary>
  private RegexReplace[] regexTable;
  private RichTextBox richTextBoxWiki;

  public LiveWiki()
  {
   this.regexTable = new RegexReplace[]
   {
    //\n
    new RegexReplace(@"$", @"\par", RegexOptions.Multiline),
    //=xxx=
    new RegexReplace(@"^=([^=\n]+)=[ ]*\\par$", @"{\fs44 ${1}\par}", RegexOptions.Multiline),
    //==xxx==
    new RegexReplace(@"^==([^=\n]+)==[ ]*\\par$", @"{\fs36 ${1}\par}", RegexOptions.Multiline),
    //===xxx===
    new RegexReplace(@"^===([^=\n]+)===[ ]*\\par$", @"{\fs28 ${1}\par}", RegexOptions.Multiline),
    //*xxx
    new RegexReplace(@"^\*(.+?\\par)$", @"\pard{\*\pn\pnlvlblt\pnf1{\pntxtb\'B7}}{\pntext\f1\'B7}${1}\pard", RegexOptions.Multiline),
    // '''xxx'''
    new RegexReplace(@"(?<=[ \n])'''(.+?)'''(?=[ ,.\n]|\\par)", @"{\b ${1}}"),
    // ''xxx''
    new RegexReplace(@"(?<=[ \n])''([^'].+?)''(?=[ ,.\n]|\\par)", @"{\i ${1}}")
   };
  }

  public void DisplayWiki(string wiki)
  {
   richTextBoxWiki.Rtf = Wiki2RTF(wiki);
  }

  private string Wiki2RTF(string wiki)
  {
   wiki = "\n" + regexTable[0].regexPattern.Replace(wiki, regexTable[0].stringReplacement);
   for (int i = 1; i < regexTable.Length; i++)
    wiki = regexTable[i].regexPattern.Replace(wiki, regexTable[i].stringReplacement);
   return @"{\rtf1\ansi\ansicpg1252\fs20" + wiki + "}\n";
  }
 }

 internal struct RegexReplace
 {
  internal Regex regexPattern;
  internal string stringReplacement;

  internal RegexReplace(string pattern, string replacement)
  {
   this.regexPattern = new Regex(pattern);
   this.stringReplacement = replacement;
  }

  internal RegexReplace(string pattern, string replacement, RegexOptions options)
  {
   this.regexPattern = new Regex(pattern, options);
   this.stringReplacement = replacement;
  }
 }
}

Conclusion

This demo illustrates the possibility to use Dasher to enter more than natural language, and a simple possibility to produce formatted text documents. At this step in the Cogain project, there is a need to establish a common protocol to plug alternative text entry systems such as Dasher to other programs.

January 2005: Multimodal Web browser

Follow-up of the Multi-modal Web browser project.

Firefox Web browser with custom speech recognition extension, GreaseMonkey user script, and gaze interaction using a custom theme:

cogain-link-numbers.user.js

// ==UserScript==
// @name	COGAIN link numbers
// @namespace	http://www.cogain.org
// @description	Add a number after each link
// @include	*
// ==/UserScript==
// Greasemonkey script http://greasemonkey.mozdev.org
// Made by Alexandre Alapetite [https://alexandre.alapetite.fr]
//  for COGAIN European Network

(function()
{
	var alllinks, thislink; 
	alllinks = document.evaluate('//a[@href]', document,null, XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE, null); 
	for (var i = 0; i < alllinks.snapshotLength; i++)
	{ 
		thislink = alllinks.snapshotItem(i);
		thislink.appendChild(document.createTextNode(' ('+ (i + 1) +')'));
	}
}
)()

Back