A Step Closer to Mind-Controlled Prosthetics

As so many good stories do, it all started in a bar.

Nearly 10 years ago, Michael Boninger, M.D., a doctor of rehabilitative medicine at the University of Pittsburgh Medical Center (UPMC), met two other members of the Pitt faculty, Douglas Weber, a bioengineer, and Andrew Schwartz, a neurobiologist. The three made plans to meet up at the local watering hole. Schwartz’s lab had developed new ways to use electrode arrays to extract movement-related information from the brains of monkeys. After a beer or two one evening, Schwartz told Weber and Boninger that he was sure if he could get a computer chip into the brain of a person with a spinal cord injury, he could not only restore their lost abilities to move, but he could also teach them to play the piano.

Boninger, who had an undergraduate degree in mechanical engineering from Ohio State said he laughed at first. “I had just started piano lessons for the first time. I was struggling with it. For the record, I’m still struggling with it,” he says. “But, in talking to Andy and Doug that night, it seemed like something within the realm of possibility. I told Andy, ‘I’m sure we can make that happen,’ and from there we formed this amazing collaboration.” 

Together, the three have developed a state-of-the-art brain-machine interface (BMI) that’s composed of a series of computer chips and electrodes implanted directly into the human brain. It allows individuals with tetraplegia, a type of spinal cord injury that results in a lack of movement and feeling to all four limbs, to operate a robotic arm—and do so by thought alone.

Such mind-controlled devices may seem like something straight out of a science fiction story, or, at the very least, a story that never leaves the bar where it was first concocted. But the UPMC “bionic” arm is actually one of several brain-controlled prosthetics being developed across the globe. Over the past five years, these prosthetics have enabled paralyzed individuals to grasp a water bottle, pour a cup of coffee, and play the video game Guitar Hero on a computer, all using their thoughts alone.

But that’s a long way from engineering safe, practical, brain-controlled assistive devices that can handle the mechanics of everyday living, such as walking on stairs or buttoning a shirt. Can scientists make BMIs both useful and usable outside the confines of the research laboratory?

The answer is still a ways off, but so far the UPMC team and several others have made real progress toward practical brain-controlled prosthetics, which are also called neuroprosthetics. Some new BMIs, like the UPMC model, involve invasive surgical implants that conduct and relay the brain’s natural signals to robotic limbs. Others rely on electroencephalography (EEG) caps, systems that can read and relay neural signaling patterns from outside the skull, to operate some form of prosthetic. Some power limb-like prosthetics like the one being tested at UPMC, while others command specialized exoskeletons that surround and mobilize paralyzed limbs.

The ultimate goal, Boninger said, is to develop new assistive technologies that will help individuals with tetraplegia—as well as those who may experience amputations, stroke, neurodegenerative disease, or other medical issues—maximize their independence.

“The technologies that are becoming available are incredible,” said Jose Contreras-Vidal, an engineer and neuroscientist at the University of Houston who has been developing an EEG-controlled robotic exoskeleton called Rex. “But at the end of the day, if the patient doesn’t want to use it, if we can’t create something that the patient can use in an independent way, it doesn’t matter.”

Decoding the Brain

Nathan Copeland admits he was a bit anxious to meet President Obama. A participant in the UPMC BMI research program, Copeland, 30, has been wheelchair-bound for nearly a decade after a car accident. As he and the UPMC team prepared for a big meet-and-greet with the former president at the White House Frontiers Conference last October, the researchers were fiddling almost constantly with the BMI set-up Copeland would use. Copeland says he kept telling himself to calm down, that President Obama was “just a guy.”

“He’s an incredible guy, sure, but still just a guy,” he says. “But it was such a cool thing to talk to him. He wasn’t just there for some photo op. He was there because he was really interested in the science.”

For a decade leading up to the White House Frontiers Conference, Boninger and his interdisciplinary team, including his Pitt drinking companions, moved step by step to build Copeland’s robotic arm. First they worked off Schwartz’s animal-model studies and tested whether they could implant recording arrays into the brains of epileptic patients who were scheduled for craniotomies as part of their treatment. They learned how to successfully record neural signals from the human brain using a specially designed biocompatible neural array. They then learned how to translate those signals into commands for the robotic arm, then, finally, developed a way to provide some sensory feedback to the person using the BMI to control the robotic arm.

Copeland’s current system is anything but simple. He first had an array of chips and electrodes surgically implanted into his brain. Those components are linked to a decoder—a device that reads and relays the brain signals—that protrudes from Copeland’s skull. Currently, that decoder is linked by thick wires to several computers as well as to the robotic arm. What’s more, while Copeland’s ability to move the arm is fairly effortless, there’s a lot of sophisticated and expensive computing firepower at work behind the curtain.

The researchers’ work is still far from done. Many people give up on assistive devices because they are too cumbersome, so minimizing the size, weight, and ungainliness of the equipment is still a priority for the team, Jennifer Collinger, a biomedical engineer on the UPMC team, said. What’s more, “[Nathan] is literally plugged into the system, which can limit his mobility,” she added.

For that reason, making the system wireless is critical, Boninger said: “The point is to develop a system that could one day help Nathan and others like him do a better job with everyday tasks like brushing his teeth, combing his hair, or holding and drinking from a cup of coffee unassisted. We’re not there yet.”

But technology moves fast. Many BMI system components have already significantly decreased in size, Contreras-Vidal said. The amplifiers that help strengthen the magnitude of neural signals used to take up half a room. Now, they can fit in the palm of a hand. They’ve also improved in terms accuracy and signal-to-noise ratio.

Contreras-Vidal, like Boninger, believes that, as long as researchers keep users’ needs front and center, they will be able to create BMIs that can truly benefit them outside the laboratory. That’s why both the UPMC and University of Houston teams collaborate with computer, electronic, and mechanical engineers to find smaller, more powerful system components—as well as biocompatible implants and light, durable materials for the prosthetics or exoskeletons themselves.

Prosthetics That Sense

To create usable systems, BMI designers also need to incorporate sensory feedback. After all, our limbs don’t work in a vacuum. Our hands, arms, and legs pick up important haptic information from the environment—information that’s processed in the brain or spinal cord to generate signals that tell the limbs the appropriate amount of speed, grip, and force to use. This allows our limbs to move effortlessly through space.

The UPMC BMI is set-up so that Copeland can receive sensory feedback directly to his brain when someone touches the robotic arm. While he describes those feelings as “weird,” they could one day help him better calculate the force with which to grasp and pull items.

That feedback already often improves his performance on such tasks. But Lewis Wheaton, Director of the Cognitive Motor Control Lab at the Georgia Institute of Technology, says that, despite the fact that BMIs can now enable an individual with a spinal cord injury to feel something, we still don’t know about how to optimally “sensorize” a hand.

“There are a lot of things we don’t necessarily understand even within a biological hand very well yet,” Wheaton said. “Where do you need feedback, exactly, to open a jar?  To grasp a fragile object?  To use appropriate force? These are open questions that will impact the mechanics—and where and what sort of sensors you put on the prosthetic,” he explained.

“We’ve had three decades of basic science that’s shown us how to translate neural activity into control signals for the robot,” Collinger added. “We’re at the point where we can move the arm in space fairly well and do simple grabs and postures.”

But more sophisticated levels of control are needed, she added, and that means figuring out how to actually use sensory feedback to do more dexterous manipulation tasks. “It’s something that is going to take some time.”

A Less Invasive Approach

Though many paralyzed individuals would welcome more freedom and mobility, they don’t necessarily want to have brain surgery to get there. That’s why Contreras-Vidal’s team has focused on a non-invasive approaching using EEG recording, involving a specialized hat that looks like a cross between an aviator’s helmet and your grandmother’s favorite shower cap.

Like the UPMC team, Contreras-Vidal’s team has had to overcome a number of engineering hurdles, including developing the right algorithms to decode the EEG signals. “The algorithms to read the signals are improving quite dramatically,” he said.

Like the implanted array, the EEG cap can track and translate the brain signals required to move a prosthetic—in this case, a robotic exoskeleton for the legs called Rex. But the EEG cap is not always as effective or accurate as the UPMC team’s arrays, Contreras-Vidal said. That said, it may still offer a less risky option, especially for individuals who may have medical issues beyond their tetraplegia or paralysis, he added.

Still, some people will balk at the idea of wearing a large, white wired cap out in the world, Contreras-Vidal admitted. This means there are design challenges to be addressed before such a cap could find widespread use.

“We’d like to come up with different looks for the cap that are based on people’s preferences,” Contreras-Vidal said. Alternatively, they could hide the recording electrodes just below the skin with a minor incision. “That way you won’t see them on top of the head and you can still have an easy wireless network of electrodes,” he said.

There are other engineering problems that need to be solved before any BMI system, internal or external, can see primetime, Contreras-Vidal said.

A real working system must be smart enough to self-correct from error states like a self-driving car, Contreras-Vidal said. It should also be available off the shelf, and be able to easily and effortlessly calibrate itself to the individual brains that must run it. For this reason, engineers in the field must agree about system inputs and outputs to create a plug-and-play system with different options. In that way, individuals who need different kinds of assistance can create a neuroprosthetic that will best work for them.

Toward Everyday Use

Most importantly, Contreras-Vidal said, BMI systems must be more than cool. They must be useful, and they must not be so burdensome or unbecoming that people don’t want to wear them.

Wheaton agrees. “While people have tried to integrate sensors and different elements of robotics into prosthetic design, you find that a lot of individuals, particularly those that use upper-limb prostheses, tend to go for simpler, hook-like devices,” he said. They’re easier to learn how to use, and often lighter and better balanced. “And you can’t ignore that they are a lot less expensive than the fancier options,” he added.

For that reason, engineers need to focus on how the device might actually be used to promote rehabilitation and activities of daily living, Wheaton said. They should consider whether the device will integrate into everyday behavior; whether it’s comfortable, appropriate or cumbersome; whether it’s natural feeling and easy to use; and whether it can handle weight or water. “Once you get out of the laboratory, all these questions start to bind together, and you need to find a way to answer them,” Wheaton said.

That’s why having a multidisciplinary research team is so important, Boninger said. It’s also why he, Schwartz, and Weber still meet at the local bar from time to time, but now they include a host of other researchers from a variety of science and engineering disciplines. With so many different perspectives, he is hopeful that he will one day be able to offer patients a system that maximizes their independence.

“This is going to happen by taking small steps,” Boninger said. “We may not get there in ten years. But we will get there eventually.” 

Kayt Sukel is an independent technical writer based in Houston, TX.