How far would you go to help your parent if they had a stroke and couldn’t communicate with anyone? And what if you were a computer genius who knew his way around electronics and games?
Chad Ruble has hacked Kinect software to create an interface that helps his disabled mother, who can’t use a keyboard after suffering from a stroke 12 years ago, communicate again. He did it by digging into Microsoft's Kinect motion sensor peripheral to create a visual dashboard complete with happy, sad, angry, tired emoticons to convey her wishes.
The stroke caused irrevocable damage to the language functions in her brain and made it difficult to express both her thoughts and emotion.
So Ruble decided to design a keyboard with visual icons — “representing emotions including happiness, sadness, and feeling tired,” the story revealed. Not satisfied with that, Ruble went on to include a series of bars that his mother could use to track how intensely she felt, such as ‘very tired’ or ‘very happy’.
When a message is sent, it is translated into text, with most of the information being transmitted through the subject line: "Lindy feels very happy,” the report explained. “The system builds on a previous effort, dubbed Iconicate, which used the prototyping platform Arduino to provide a physical interface.”
Ruble then hacked his $109.99 Kinect using the SimpleOpenNI library “to process the input from the Kinect through gesture recognition code to track the position of his mom’s hand.”
To complete the hack, Ruble used a “sample processing sketch” provided by Daniel Shiffman to generate and send a complete message via e-mail by ‘clicking’ the green button. Once complete, by waving her hand the message can be reset by crossing the screen’s red “X” mark.
Edited by
Brooke Neuman
