Linux input

Researchers had suggested it for decades: humans do not have a left hand with 101 fingers and a right hand with 1 to 3 fingers. Now that the computing industry is realising this, we need operating systems with a rich enough input model. Contributing to Linux-based operating systems is a good reality check for the lab's research on input management in interactive software. It can also help the aeronautical industry to better understand some issues associated to modern input systems.

The lab's involvement in Linux started with multitouch devices. Ultimately, we hope to be able to contribute to the evolution of input management so as to support a wider range of input configurations, and even multimodal input.

Public resources

  • our multitouch primer
  • our list of multitouch devices and their support status in Linux
  • our general Linux multitouch howtos
  • our Ubuntu-specific multitouch howtos
  • our Android-specific multitouch howtos
  • our tuiototouch bridge
  • our multitouch diagnostic application
  • out git repositories

    Related projects

    ShareIT, Istar

    Related publications

  • Stéphane Chatty, Alexandre Lemort, Stéphane Valès. Multiple input support in a model-based interaction framework . Proceedings of the second annual IEEE international workshop on horizontal interactive human-computer systems (Tabletop 2007), IEEE computer society, 2007

    Contact: chatty at enac.fr
    Updated Dec 29, 2010.

    • January 2011: multi touch typing

    How it works: Our experimental multitouch gesture recognition algorithm feeds simulated keyboard events to the Linux kernel. Linux does the rest.

    Contact: chatty at enac.fr, pengfei.zheng at eleve.enac.fr
    Updated Jan 19, 2011.

    January 2011: The tuiototouch bridge

    http://www.youtube.com/watch?v=J6k76vuw9Rk

    You need:

  • Ubuntu 10.10
  • a computer with a camera
  • a TUIO-compatible computer vision software (eg. CCV)
  • our tuiototouch bridge

    How it works: The tuiototouch bridge converts multitouch events emitted by the vision software on TUIO and injects it in the kernel, simulating a standard multitouch panel. Ubuntu Unity or your favorite multitouch-compatible software does the rest.

    Contact: chatty at enac.fr, charvet at lii-enac.fr
    Updated Jan 19, 2011.

    • December 2009: Support for multitouch in Xorg

    This video made by Benjamin Tissoires, Stéphane Chatty and Gilles Tabart from the Interactive Computing Lab at ENAC demonstrates how Stephane's kernel drivers and Benjamin's patched X.org evdev driver allow to use existing applications with multitouch displays.

    You need:

  • Linux kernel 2.6.31 or more recent;
  • a computer with a Linux-compatible multitouch device. Here is a list of available multitouch devices and their status;
  • the corresponding driver. Some are already in the Linux kernel sources, others are in the lab's git repositories or here: Stantum MTP, DiamondTouch;
  • you will also need to import bits from a recent hid-core.c and hid-ids.h into the drivers/hid directory of your kernel sources. Here is some more information on how to proceed with the kernel;
  • X.org version X11R7.5, available for instance in Fedora 12;
  • Benjamin's X.org evdev driver and multitouch management client (obsolete since september 2010). You can find more up to date information here on how to proceed.

    How it works: Actually, most of the work is done by MPX. When loading the customized evdev driver, the multitouch device is recognized as a touchscreen.
    By changing the property "Evdev MultiTouch", the user can control the number of contact points that are recognized. Thus the device only transfers the contact points to the newly created subdevices.
    This property can easily be modified with "multitouchctl x", where 'x' has the following meaning:

  • if x = 0, touchscreen emulation,
  • if x > 0 it is the number of contact points recognized.
    The daemon "multitouchd" monitors/controls the property and creates the master devices. It hides or shows the cursors depending on their valuator "TrackingID" and also sends the press/release events at the beginning/end of movements.
    We split the creation of subdevices and cursors in order to be able to implement a gesture recognition in user-space (TODO).
    To understand what is actually happening inside your server, the command "xinput" is very useful!

    Context: This work was done as part of project ShareIT, a research project in which we collaborate with our good friends the multitouch hardware makers at Stantum, the multitouch software and interaction experts at IntuiLab and the aircraft cockpit designers at Thales Avionics to explore the use of multitouch user interfaces in cockpits. But no, there is no plan to use Linux in the cockpit, this is just for the lab's research :-)

    Multitouch? Here is more information about multitouch interaction and available devices, multitouch on Linux, etc.

    Contact: chatty at enac.fr, tissoire at lii-enac.fr
    Updated Dec 29, 2010.

    • June 2009: Linux drivers for multitouch screens

    This demo made by Mohamed-Ikbel Boulabiar, Stéphane Chatty and Sébastien Hamdani from the Interactive Computing Lab at ENAC shows how one can use the multi-touch capabilities that Henrik Rydberg added to the Linux input system.

    You need:

  • Linux kernel 2.6.31 or more recent;
  • a computer with a Linux-compatible multitouch device. Here is a list of available multitouch devices and their support status;
  • the corresponding driver. Most are already in the Linux kernel sources, others are available from the list of devices above. Here is information about how how to install the drivers;
  • our demo code.

    How to use the code:
    The demo code reads input directly from the device file (/dev/input/eventX). The X server is not involved in input handling at this stage. The demos perform very simple gesture recognition then send DBus messages to Compiz to produce effects. You need to activate Compiz with the DBus plugin to get the demo working, and for window rotate/scale you will also need the freewins plugin.

    The roadmap:
    These demos were produced in Spring 2009. The multitouch events are in the kernel since version 2.6.30, as well as the Broadcom 5974 driver. Our multitouch code in the N-Trig driver is in the kernel since version 2.6.31. Our Stantum HID, Quanta, MosArt and 3M drivers are in the kernel since 2.6.34 and in Ubuntu since 10.04. Our DiamondTouch drivers will be submitted as soon as CircleTwelve and/or Mitsubish Electric give us the green light for redistributing their firmware under GPL. The lab is currently contributing to the multitouch management in X.org, and involved in the thinking on how to reorganise input management in both Xorg and the kernel to better deal with multitouch, including gesture recognition.

    Context:
    This work was done as part of project ShareIT, a research project in which we collaborate with our good friends the multitouch hardware makers at Stantum, the multitouch software and interaction experts at IntuiLab and the aircraft cockpit designers at Thales Avionics to explore the use of multitouch user interfaces in cockpits. But no, there is no plan to use Linux in the cockpit, this is just for the lab's research :-)

    Multitouch?
    Here is more information about multitouch interaction and available devices, multitouch on Linux, etc.

    Contact: chatty at enac.fr
    Updated Feb 6, 2010.