This project took place between July, 2015 and January, 2016. I worked on this project as an interaction designer / frontend developer for my client, Meditech Solution, a startup based in Bangkok, Thailand, building a communication aid system for people with stroke and amyotrophic lateral sclerosis (ALS).
I worked alongside the Meditech research team, web developer, Lek, and the product manager, Jack. I was responsible for making eye tracking works for their communication aid platform, Senze.
Meditech Solution was building the next iteration of Senze, a communication aid software for people with stroke and Amyotrophic Lateral Sclerosis (ALS). Senze’s last iteration consists of a touch screen with buttons for users to communicate their needs such as food, medication, and the need for caretaker. It also incorporated an obsolete blink detector that could only detect blinking but not eye movement. The software had to switch the focus between each element to wait for the user to click. The users have to wait until the desired element is focused and blink their eyes. This takes a long time and put a limit on the number of elements that can be on the screen. Meditech wanted to integrate modern eye tracking technology to allows users to move the cursor freely on the screen. That’s when my task came in. For Meditech, I need to design and build a desktop app that integrates an eye tracking device with the existing system. By the time I joined, Meditech had already got the design goals for the phase:
As the concept of eye tracking-enabled user interface was quite new, I proposed to go straight to functional prototyping. Wireframes and mockups are not going to be very useful as it is not testable for our users. The amount of time testing with real users is also very limited due to their consent and exhaustion (Using eye tracking devices can cause quite a fatigue).
Clicking is the most basic way users interact with graphical user interfaces. But, enabling clicking with an eye tracker can be tricky. Plugging an eye tracker in an getting on-screen gaze coordinates was just a small baby step. We also had to dealt with jittering, configuration sensitivity, and user’s fatigue. After some experiments, I came up with a first functional prototype. It was a Chrome Extension that only test two things: blink clicking and delay clicking.
Triggering the click by blinking your eye once. Although how intuitive it sounds, blink clicking was proven to be impractical. From our internal testing, no matter how trained the user is, blinking not only causes your eyes to roll and miss the desired click target and also causes the users to momentarily lost their visibility.
Delay clicking refers to a click mechanism in which the user has to steadily focus on an element for a period of time. This paradigm is borrowed over from gesture-based interfaces where clicking is also a challenge. Many Kinect games use delay clicking as the main clicking mechanism instead of push clicking to avoid ambiguity.
From our initial testing, delay clicking increases the success rate of a click and left us a lot of room for tuning such as delay time and cool down time.
I also implemented scrolling which is essential to consume modern web content. I introduced scrolling with hover buttons and scroll zones. From the sidebar, users can hover the cursor on scroll up or scroll down button to initiate scrolling. Users can also hover the cursor on ‘scrolling zones’ which are designated areas on the top and the bottom of the page.
In the second prototype, I focused on testing how could users interact with the web. I switched from a Chrome Extension to a Chrome App and experimented on making the eye cursor work with different websites.
Turns out the eye cursor alone is difficult to use with most websites because most of the web is created for hands. Elements are only big enough to be clickable but still too small to handle the jittering of the eye cursor. For that, I introduced Web Snapping mechanism where the cursor will be snapped to the button at a close enough distance. To detect a clickable element in the web page, I implemented a page traversal mechanism that could check if the cursor lies in elements worth snapping (buttons, textboxes, checkboxes and etc.)
Also, I found that several mobile sites are already more gaze-friendly than its desktop counterpart. So we chose to link to the mobile version of each website in our quick bookmarks menu later on.
Typing is another challenging task to accomplish with an eye tracker. It is not enough to just provide an onscreen keyboard with auto-snapping. I developed an auto-suggestion where a word is suggested according to user’s partial input. The system also allows adding of custom words from Senze’s administration backend.
Meditech research team tested the products at Hospitals and user’s home. We’ve collected quite a lot of valuable feedbacks.
The feedback was very valuable for the latter phase of the projects
We addressed insights from our testing. The improvement in the final iteration in this phase includes:
The Chrome App I designed and developed was released as a part of Senze 4.0. It helped Meditech to raise another round of funding from Invent ventures and won a pitch competition at Startup Thailand Conference in April 2016. As of late 2016, ten public hospitals in Thailand has adopted Senze. Also, because of Senze, one of our users was able to continue writing her novel despite the developing ALS condition.
A video produced by Meditech Solution demonstrating the latest version of SenzE
Working with eye tracking devices, I can feel the fatigue created by the device. This drove the design decisions down to the coding of each element.
Designers seems to fear skipping the wireframing and mockup phase to avoid design fixation. But, I learned that it might be productive to bring the functional prototype early on when the interaction concepts is hard to communicate in other medium. Also, functional prototypes are getting a lot easier to build with the technology we have today.
We are quite a long way from making the web accessible to everyone. In this project, I have got a chance to dig into ARIA (Accessible Rich Internet Applications) tag specifications and some web accessibility standards. The experience was very enlightening and provoked me to see the web in a different way.