Stanford team's brain-controlled prosthesis nearly as good as one-finger typing
Years of work have yielded a technique that continuously corrects brain readings to give people with spinal cord injuries a more precise way to tap out commands by using a thought-controlled cursor. A pilot clinical trial for human use is underway.
This video includes two clips. In the first, flashing targets on a virtual keypad are hit by monkeys (not shown) using their hands. The second clip also shows targets being hit. But this time, the motion is directed by an experimental device that taps into the monkey's brain. This device discerns their intention to hit the target and translates this thought into an electronic command that controls a virtual cursor. In the first clip the monkeys hit 10 targets in 9.9 second with their hands. It takes 11.4 seconds to hit 10 targets using the thought-control device.
When we type or perform other precise tasks, our brains and muscles usually work together effortlessly.
But when a neurological disease or spinal cord injury severs the connection between the brain and limbs, once-easy motions become difficult or impossible.
In recent years researchers have sought to give people suffering from injury or disease some restored motor function by developing thought-controlled prostheses.
Such devices tap into the relevant regions of the brain, bypass damaged connections and deliver thought commands to devices such as virtual keypads.
But brains are complex. Actions and thoughts are orchestrated by millions of neurons – biological switches that fire faster or slower in dynamic patterns.
Brain-controlled prostheses currently work with access to a sample of only a few hundred neurons, but need to estimate motor commands that involve millions of neurons. So tiny errors in the sample – neurons that fire too fast or too slow – reduce the precision and speed of thought-controlled keypads.
Now an interdisciplinary team led by Stanford electrical engineer Krishna Shenoy has developed a technique to make brain-controlled prostheses more precise. In essence the prostheses analyze the neuron sample and make dozens of corrective adjustments to the estimate of the brain's electrical pattern – all in the blink of an eye.
Shenoy's team tested a brain-controlled cursor meant to operate a virtual keyboard. The system is intended for people with paralysis and amyotrophic lateral sclerosis (ALS), also called Lou Gehrig's disease. ALS degrades one's ability to move. The thought-controlled keypad would allow a person with paralysis or ALS to run an electronic wheelchair and use a computer or tablet.
"Brain-controlled prostheses will lead to a substantial improvement in quality of life," Shenoy said. "The speed and accuracy demonstrated in this prosthesis results from years of basic neuroscience research and from combining these scientific discoveries with the principled design of mathematical control algorithms."
Brain dynamics
The new corrective technique is based on a recently discovered understanding of how monkeys naturally perform arm movements. The researchers studied animals that were normal in every way. The monkeys used their arms, hands and fingers to reach for targets presented on a video screen. What the researchers sought to learn through hundreds of experiments was what the electrical patterns from the 100- to 200-neuron sample looked like during a normal reach. In short, they came to understand the "brain dynamics" underlying reaching arm movements.
"These brain dynamics are analogous to rules that characterize the interactions of the millions of neurons that control motions," said Jonathan Kao, a doctoral student in electrical engineering and first author of the Nature Communications paper on the research. "They enable us to use a tiny sample more precisely."
In their current experiments Shenoy's team members distilled their understanding of brain dynamics into an algorithm that could analyze the measured electrical signals that their prosthetic device obtained from the sampled neurons. The algorithm tweaked these measured signals so that the sample's dynamics were more like the baseline brain dynamics. The goal was to make the thought-controlled prosthetic more precise.
To test this algorithm the Stanford researchers trained two monkeys to choose targets on a simplified keypad. The keypad consisted of several rows and columns of blank circles. When a light flashed on a given circle the monkeys were trained to reach for that circle with their arms.
To set a performance baseline the researchers measured how many targets the monkeys could tap with their fingers in 30 seconds. The monkeys averaged 29 correct finger taps in 30 seconds.
The real experiment only scored virtual taps that came from the monkeys' brain-controlled cursor. Although the monkey may still have moved his fingers, the researchers only counted a hit when the brain-controlled cursor, corrected by the algorithm, sent the virtual cursor to the target.
The prosthetic scored 26 thought-taps in 30 seconds, about 90 percent as quickly as a monkey's finger. (See video of hand- versus thought-controlled cursor taps.)
Thought-controlled keypads are not unique to Shenoy's lab. Other brain-controlled prosthetics use different techniques to solve the problem of sampling error. Of several alternative techniques tested by the Stanford team, the closest resulted in 23 targets in 30 seconds.
Next steps
The goal of all this research is to get thought-controlled prosthetics to people with ALS. Today these people may use an eye-tracking system to direct cursors or a "head mouse" that tracks the movement of the head. Both are fatiguing to use. Neither provides the natural and intuitive control of readings taken directly from the brain.
The U.S. Food and Drug Administration recently gave Shenoy's team the green light to conduct a pilot clinical trial of their thought-controlled cursor on people with spinal cord injuries.
"This is a fundamentally new approach that can be further refined and optimized to give brain-controlled prostheses greater performance, and therefore greater clinical viability," Shenoy said.
Paul Nuyujukian, a postdoctoral researcher and MD/PhD in neurosurgery and electrical engineering, also contributed to the research, as did Stephen Ryu, a neurosurgeon with the Palo Alto Medical Foundation and consulting professor of electrical engineering. Columbia University assistant professors Mark Churchland, in the Neuroscience Department, and John Cunningham, in the Statistics Department, completed the team roster.
Funding for the experiments came from a Director's Pioneer Award from the National Institutes of Health, a T-RO1 Award from the National Institutes of Health and two programs from the Defense Advanced Research Projects Agency: REPAIR (Reorganization and Plasticity to Accelerate Injury Recovery) and Neuro-FAST (Neuro Function, Activity, Structure, and Technology).
Media Contact
Tom Abate, Associate Director of Communications for Stanford Engineering, (650) 736-2245, tabate@stanford.edu
Dan Stober, Stanford News Service: (650) 721-6965, dstober@stanford.edu
