Historical note and nomenclature

Electrical stimulation of the nervous system has been used for millennia as a treatment modality for pain and other disorders (notably by the application of electric fishes and other organisms to the human body). In 1965, neuroscientist Jose Delgado, disguised as a matador, dared to enter the ring with a bull that had an electrode implanted in its caudate nucleus (57); when the electrode was stimulated remotely by the “neuroscientist matador,” the bull immediately aborted its charge and turned away. Less dramatic but more clinically useful experiments were carried out by others (27).

Major breakthroughs since a publication in 1973 entitled “Toward direct brain-computer communication” (85) include the first brain-computer interface (BCI) spelling device for severely paralyzed individuals (10), the first clinical evaluations of prototypes coming out of the BrainGate and similar projects (34; 21), and, very recently, the first BCI shown to be applicable with a completely locked-in syndrome (LIS) patient (19).

In addition to the “noninvasive” versus “invasive” categorization noted above, neuroprostheses can be categorized by their role. Some neuroprostheses serve as artificial sensory inputs, eg, cochlear or retinal prostheses. Others serve as modulators to improve or correct brain motor function, eg, deep-brain stimulation. Still others involve both monitoring of sensory input and modulation of motor output, eg, the NeuroPace® device for refractory epilepsy (79). Commonly referred to as “closed-loop stimulation,” a signal recording electrode (or electrodes) placed either in the brain (depth electrode) or on the surface of the brain (cortical surface electrode) communicates with a stimulating electrode. By monitoring brain electrical activity continuously to detect when an overt seizure is pending, another region of the brain can be stimulated to abort or at least attenuate the seizure.

A term commonly employed (as in the preceding paragraph) is “neuroprosthesis,” which is quite literally “substitute for the nervous system.” This term has the advantage of not implying a computer or a machine in the technique for correcting nervous system malfunction. Indeed, it is certainly possible in the future that stem cells or gene therapy techniques--or even proteins administered orally or by transdermal patches--may serve a “neuroprosthetic” role. Such techniques would obviate the need for a computer or machine interface at the level of the individual patient although computers and machines would be involved in the creation of such biological neuroprosthetic techniques.

Clinical approach

Technical aspects. The primary goal of most neuroprosthetics is either to restore brain function to its normal state or to substitute for lost function. This may entail a sensory prosthesis, such as a cochlear or retinal implant (ie, a device that substitutes for the defective part of the sensory organ and stimulates either the cochlear nerve or the retinal ganglion cells directly). Such sensory prostheses are “biomimetic,” in that they convert the environmental sensory input into a signal to the intact remaining sensory nervous system that mimics the latter. Even the sense of touch has been restored using a prosthetic arm plus a brain stimulation technique (80; 29).

Brain-computer interface (BCI). Unlike sensory neuroprostheses, a BCI will only “sense” (ie, monitor) brain activity, but with a view to specifically allow people to control some device or software. BCI systems have initially exploited “neurofeedback” paradigms (ie, operant conditioning with brain signals, where subjects learn to modulate their brain activity through feedback so as to optimize the control of a device) (31; 10). However, modern BCIs have fully transitioned to the artificial intelligence era and respect a pattern recognition and machine learning architecture. This means that all state-of-the-art systems comprise some form of the following processing modules: signal processing (eg, spatial and spectral filtering), feature extraction and selection, and classification or regression.

Clinical BCI applications rely on this prototypical design, as well as on two additional design choices: the type of neural interface or brain imaging method employed for capturing brain activity and the “BCI paradigm,” ie, the brain phenomenon that is exploited to enable interaction. The latter usually regards either evoked brain responses triggered by external stimuli (P300, Steady-State Visually Evoked Potentials (SSVEP), etc.) or some endogenous, self-paced and spontaneous modulation of brain activity (eg, task-specific sensorimotor rhythms regulation elicited by imagined or attempted movements).

A BCI may substitute for a defective motor output system, for example, bypassing the spinal cord in a patient who has suffered a spinal cord injury to stimulate the muscles of the arm directly (functional electrical stimulation). An alternative to functional electrical stimulation in a quadriplegic patient is to place an electrode in the motor or premotor cortex (usually an electrode array with 100 or more individual electrodes) to record the electrical activity of that part of the brain in order to guide a substitute motor output for the paralyzed arm or leg (13; 02). The original experiments involved patients learning to operate basic computer functions (allowing internet surfing, etc.); more recent BCIs allow patients to control a prosthetic arm with multiple degrees of freedom (see clinical vignette #1).

An attractive means of monitoring a patient’s brain electrical activity is by noninvasive scalp electrodes, such as those used for electroencephalography (EEG). The advantage of noninvasiveness may be offset by the poor resolution, which precludes recording electrical activity from small regions of the brain. However, for some applications the information from scalp electrodes is sufficient to guide the desired output. The P300 has already found successful clinical applications (75; 90). Other EEG-based interfaces provide end-users with more ecological, self-paced interaction (10; 62; 64; 46). Greater noninvasive precision can be obtained using magnetoencephalography (MEG) or functional magnetic resonance imaging (fMRI), but the need for the patient to be placed in a multimillion-dollar machine makes these techniques impractical for most brain-machine interface applications (12). Functional near-infrared spectroscopy (fNIRS) is a less expensive and more portable type of BCI (also based on the metabolic activity of the brain), which has been shown to provide a viable communication solution for locked-in syndrome (LIS) patients (20), as well as for enhanced gait rehabilitation when fNIRS is combined with deep neural networks (33).

An essential component of most BCIs is a sophisticated computer algorithm that converts one signal (or stream of information) into another signal in real time in order to bypass the nervous system function or component that is impaired or lost due to the brain disorder involved. In a sensory neuroprosthesis, the algorithm converts the sensory input (sound energy in the case of cochlear implants; light energy in the case of retinal prostheses) into electrical signals that are then carried via the usual channels to the appropriate cortical areas for processing--the “biomimetic” function noted above. However, to date, the biomimetic conversion or encoding is not perfect, and the brain must go through a learning phase (possibly many hours of training) before the prosthesis works as an effective functional substitute (see the clinical vignettes).

In a neuroprosthesis for a quadriplegic patient, either scalp electrodes, epidural or subdural surface contact electrode arrays, or (for the most precision) needle electrode arrays implanted in the cortex (the 100-electrode Utah array being one of the most common) are employed to collect brain electrical activity data (54). The algorithm converts that brain electrical activity into a signal to guide motor output--usually to a robotic arm or leg but potentially to the patient’s own extremity (in the case of spinal cord injury rather than amputation). The potential to use brain cortical electrical activity to guide a prosthetic or robotic arm was described decades ago (31). Advances in the ability to monitor and analyze cortical electrical activity have demonstrated the complexity of the relationship between that brain electrical activity and the motor activity required for fine control of a prosthetic arm. It has been shown that such neuroprostheses can result in the reorganization of brain electrical activity, eg, electrical activity between the cortex and subcortical structures such as the striatum, which is a demonstration of neuroplasticity in the mature brain (44; 89).

Brain-machine interface (BMI). Although the present deep-brain stimulation systems using a stimulating electrode greater than 1 millimeter in diameter have proven remarkably effective for movement disorders such as Parkinson disease, dystonia, and essential tremor, it has not been nearly as successful in treating disorders such as depression, obsessive-compulsive disorder, and many types of refractory epilepsy. The usefulness of more miniaturized, multielectrode microarrays (such as the Utah array noted above) for recording focal brain electrical activity with increased precision has suggested that much more attention needs to be placed on the actual tissue--electrode interaction in the brain, which is the initial link in BCI as well as the basis of the BMI. A number of groups have been working on improving the ability of electrodes to record and stimulate brain electrical activity down to the cellular level with minimal tissue trauma. Some groups are developing micron-level electrodes that can be inserted via the intracranial vasculature (down to the capillary level) to reduce the invasiveness of inserting electrodes through the brain parenchyma (58).

Because the regions of interest in optimizing brain tissue recording and stimulating involve such anatomical locations as the proximal axon and the synapse, the techniques must assume similar dimensions, ie, submicron or nanolevel. Fortunately, using nanotechniques (involving materials such as carbon nanotubes or nanofibers with conducting polymer coatings), electrode arrays with greatly reduced impedance and greatly enhanced capacitance can be realized (53; 41; 22; 16). The enhancements in charge transfer and signal-to-noise ratio of nanoarrays (in comparison with standard platinum electrodes) as well as the improvements in spatial and temporal resolution can reach several orders of magnitude. Such improvements in the brain tissue-electrode interface not only enhance the spatial and temporal resolution of the recording and stimulating device but also reduce remarkably the electrical current needed to drive the device. This may allow the device to be driven entirely by current generated from pressure changes in the brain (eg, cardiovascular pulsations) thanks to nanomaterials that capitalize on piezoelectric effects (93). This would obviate the need for the bulky subcutaneous battery required with the present deep-brain stimulation systems and eliminate the need for battery replacement operations in the future. Nanolevel techniques have now allowed the development of probes that can record activity from hundreds of sites simultaneously (37; 60). A needle 10 mm long and 20 x 70 μm cross-section has been fabricated with 960 12 x 12 μm recording sites, 384 of which recording sites may be used at one time. This technical feat has permitted gathering simultaneous electrical data in vivo (mouse and rat) from large neuronal populations for up to 8 weeks.

An alternative technique is a “living electrode” that utilizes living cortical neurons in a hydrogel microcolumn (about 300 μm diameter) implanted perpendicularly into the cortex to monitor/modulate brain electrical activity (01). Such a device, implanted in the cortex with one end of the column on the cortical surface and the other end at the desired depth (eg, layer IV), allows an LED array to optogenetically stimulate and a photodiode array to record—either array being on the cortical surface end of the column. Stereotactic insertion of such microcolumns into the rat cortex have been followed for up to one month in vivo; axonal growth and synaptic interaction with the host cortex have been documented (01).

Many, if not most, brain disorders are primarily disorders of brain chemistry (neurotransmitters) rather than disorders of brain electrical activity. The emphasis on brain electrical activity in brain-machine interface research has been an example of “measuring what we have the tools to measure, not necessarily measuring what is most important.” Even Parkinson disease, the most dramatic example of brain stimulation to treat a nervous system disorder, is the result of a loss of dopamine in the striatum (and not primarily a disorder of brain electrical activity). Indeed, it has been shown in a swine model that stimulation of the subthalamic nucleus (with parameters matching those used in subthalamic nucleus deep-brain stimulation in humans) produces an elevation of dopamine levels in the striatum (77). The same group has also monitored dopamine release in the caudate nucleus with subthalamic nucleus stimulation in a patient suffering from advanced Parkinson disease who was undergoing deep-brain stimulation surgery (39). An article has reviewed the role of neurotransmitter monitoring for closed-loop deep brain stimulation (67).

Fortunately, nanotechniques are also improving the chemical brain-machine interface--in addition to improving the electrical brain-machine interface. Fast-scan cyclic voltammetry (FSCV) and related techniques allow real-time in vivo measurement of neurotransmitter levels, including neurotransmitters (eg, dopamine and serotonin) that are involved in common mood disorders, such as severe depression and obsessive-compulsive disorder. Although FSCV using a standard microelectrode is unable to distinguish accurately the individual levels of dopamine and serotonin in mixtures simulating the brain in vivo situation, a carbon nanofiber electrode array has been shown capable of measuring individual levels of dopamine and serotonin in such a mixture (70). Another group has used microelectrode arrays to investigate electrical activity and glutamate levels in columnar microcircuits in the nonhuman primate brain in vivo (56). One group has developed a wireless system for in vivo neurotransmitter monitoring in humans (39). The same group has also monitored thalamic adenosine levels with thalamic electrical stimulation during surgery to implant deep-brain stimulation electrodes in eight patients suffering from essential tremor. A further step toward real-time monitoring of both electrical and chemical activity as well as electrical stimulation has been taken with a hybrid 1.2 mm diameter lead for deep brain stimulation that incorporates both fast-scan cyclic voltammetry for dopamine neurotransmitter recording and electrical recording (06).

One long-term goal for the BMI is actual “parts replacement,” ie, cellular and subcellular level devices to substitute for defective axons, synapses, neurons, glia, etc. One example is an artificial afferent nerve (43). Pressure sensors convert pressure information (from skin deformation) into action potentials, similar to a sensory nerve. The action potentials then stimulate the efferent (motor) nerve to result in muscle contraction and movement (in this case, in a cockroach).

Another example is a biomimetic synapse that replicates the plasticity of brain synapses (42). An organic neuromorphic device—a conducting polymer-coated gold on silicon microfluidic post-synaptic receptor—has been incorporated with presynaptic dopamine-secreting PC-12 cells to mimic a biological synapse. Such a device offers the possibility of going beyond precision monitoring and modulating brain electrical and chemical activity to the ability of creating an artificial neuromorphic synapse that can undergo the long-term conditioning and recovery of synaptic weight that is seen in the typical brain synapse where neurotransmitter oxidation and recycling (in this case, dopamine) takes place.

With a brain-machine interface that includes real-time continuous monitoring of both electrical and chemical activity in specific regions of the brain, the understanding of brain disorders ranging from depression to obsessive-compulsive disorder to epilepsy will improve dramatically. Subsequently, equally dramatic improvements in using brain-machine interfaces for treatment of such disorders will also be realized.

Clinical applications. The most common clinical neuroprosthetic applications to date have been sensory neuroprostheses, with well over 100,000 cochlear implants worldwide. The number of retinal implants for those with macular degeneration and other, less common forms of retinal disease (eg, retinitis pigmentosa) is much lower but will increase rapidly as the number of contact micro/nano electrodes in the implanted arrays increases--allowing vision enhancement beyond basic independent navigation and reading large print. The number of DBS implants over the past 20 years is also well over 100,000 worldwide, the majority used in patients with Parkinson disease and other movement disorders.

Clinical applications of BCI include brain-actuated spellers (10; 62; 84) and patients suffering from locked-in syndrome or late stage amyotrophic lateral sclerosis. Other applications include assistive mobility (46) and orthotic devices (65).

An increasingly important brain-machine interface application is stroke rehabilitation (68; 03; 66; 15; 30; 09; 17; 40), marking a shift in brain-machine interface research focus from motor substitution to motor recovery. Laboratory brain-machine interface research in rats has demonstrated that closed-loop brain-machine interface is superior to open-loop in fostering functional recovery after a focal head injury (32). It is reasonable to expect such closed-loop brain-machine interface systems to enhance functional recovery in humans with similar focal brain injuries (due to either trauma or stroke), as also demonstrated for spinal cord injury patients (24; 74). The role of brain-machine interface techniques for traumatic brain injury rehabilitation has been reviewed (83).

More complex brain-machine interface devices include those that allow a quadriplegic spinal cord injury patient, a limb amputee, or even a patient with the locked-in syndrome to interact with the environment (see clinical vignette #2). By virtue of recording brain electrical activity with scalp electrodes or arrays implanted intracranially (epidural, subdural, or cortical), this interaction may involve a computer for Internet surfing or driving a motorized wheelchair; it may be used, eg, to control a prosthetic limb.

A major breakthrough has been the ability of a complete locked-in syndrome patient to learn to control an invasive interface combining the benefits of invasive signals with the learning capacity of the brain, mostly exploited so far in noninvasive and primitive BCI designs (see vignette #5) (19).

Future directions. Future directions in brain-machine interface can be divided into techniques and applications.

Regarding future techniques, improvements in the physiological neural interface (using nanotechniques to enhance monitoring and modulating brain electrical and chemical activity) have been considered in the previous section. Organic and soft materials can more closely approximate the mechanical properties of brain tissue than metallic electrodes. Such biomimetic materials also can more accurately interact with brain electrochemical physiology. The topic of organic neuromorphic devices has been discussed (82).

The miniaturization of the biosensors may be taken a step further by eliminating the electrode altogether and replacing it with a large number of micron-sized “neural dust motes” distributed throughout the regions of interest in the brain (76; 52). These “neural dust motes” communicate ultrasonically with a subdural transceiver, which in turn communicates transcranially with the external transceiver. Because it was shown nearly two decades ago that it is possible to record brain electrical activity with an electrode inside a capillary as effectively as with an electrode in the brain parenchyma (47), for brain electrical activity monitoring such “neural dust motes” could remain within the capillary. Interventional neuroradiologists are becoming so adept at catheterizing small brain blood vessels that the possibility of implanting “neural dust motes” in the brain parenchyma for neurochemical monitoring by puncturing through the capillary wall is also feasible, similar to the implantation of endovascular stent electrodes noted above. The era of the truly minimally-invasive brain-machine interface or neuroprostheses for neuromodulation is close at hand.

Sophisticated computational analysis techniques will provide much more efficient brain-machine interfaces for applications such as deep-brain stimulation. One example comes from the modification of DBS in a primate model of Parkinson disease (81; 86). The therapeutic effects of DBS for movement disorders such as Parkinson disease usually are lost within minutes of discontinuing the stimulation. However, using algorithms that coax the abnormally synchronized brain electrical firing patterns in Parkinson disease back to more normal patterns--algorithms that also require much less energy (electrical current) than standard high-frequency DBS--it is possible to have sustained aftereffects of DBS, ie, the benefit on the abnormal Parkinsonian movements can last for up to 4 weeks following cessation of stimulation. When combined with the efficiencies noted in the discussion of technical aspects above, such computational analysis techniques promise to promote DBS from the ranks of a crude brain bludgeon to a subtle brain symphonic conductor (48). Furthermore, closed-loop DBS could reduce or eliminate adverse effects of conventional DBS and increase the battery life (59). Several research efforts are devoted to a better understanding of the still elusive mechanisms underlying the success of DBS (05).

The computational analysis of brain signals, made more complex as the number of electrodes monitoring brain electrochemical activity increases, will be markedly enhanced by techniques such a multi-threaded parallel processing (28). The lengthy learning curve for patients to master, for example, fine motor control of a prosthetic limb will be reduced by multi-tier distributed computing and linked data techniques, although coadaptation and learning and coping with nonstationarity (23) in brain-machine interface is likely to remain crucial for successful neuroprosthetic control. A pilot study used EEG-based brain-machine interface plus cloud servers, in-home fog servers, and smartphones (94). The “big data” generated by many patients will allow the development of personalized algorithms that should enhance the speed and efficacy with which a patient learns how to operate the neuroprosthesis (14; 73). However, the extent to which brain-machine interface can be treated as a purely decoding problem and the potential importance of the role of learning and of the coadaptation between user and machine need to be carefully investigated (64; 26; Perdikis and Millan 2020; 78).

Novel brain-machine interface techniques--what might be termed the software rather than the hardware of the actual electrodes themselves--are likely to be a major advance. One such technique has the subject judge whether the actions of the brain-machine interface or neuroprosthesis are correct or not (35). Thus, the subject “teaches” the BCI rather than the subject learning to adapt his/her thought processes to achieve the desired action. Another technique, neural dynamical modeling, may allow accurate predictions of future neural population activity based on single trials (38). This would shorten the often-lengthy learning curve for BCIs in real-world situations (14).

Future applications range from the obvious to the exotic. The obvious include refinements in the electrodes of the BCI/BMI to allow more sophisticated control of paralyzed limbs (mimicking intact hand and leg function) as well as, in the amputee, more realistic control of artificial or robotic limbs. Similar improvements will occur in sensory prostheses, notably cochlear and retinal implants. It is conceivable that such sensory prostheses may achieve a resolution exceeding that of normal intact humans--subject, of course, to the degree to which sensation is limited by the central brain processing rather than the peripheral sensory-neural interface (cochlea or retina).

Although the focus of clinical neuroprostheses to date has been primarily the replacement of lost nervous system function, the use of neuroprostheses to enhance brain function is an increasingly important application of the brain-machine interface. One group that has been quite active in this area is the United States’ Defense Advanced Research Projects Agency (DARPA). Although DARPA has funded brain-machine interface research for over 40 years, recent efforts have included using brain-machine interface techniques to enhance brain function beyond normal capabilities (rather than replace lost nervous system function) (50). These have generally taken the form of a noninvasive brain-machine interfaces (such as a scalp EEG) to improve human function beyond baseline. One such closed-loop BCI system, the Adaptive Peak Performance Trainer (APPT), was able to more than double the learning rate for novices undergoing rifle marksmanship training (50):

Another DARPA project uses the P300 of the EEG to enhance an individual’s ability to detect a target when analyzing images. In testing with over 40 professional imagery analysts, such a brain-machine interface system “resulted in up to a 10-fold increase in analysis throughput (area of imagery analyzed per unit time) with no loss of target detection sensitivity, as compared to the analysts’ performance using their standard imagery analysis approaches” (50). Although DARPA’s goals are understandably influenced by the needs of the military, the potential applications of such performance-enhancing noninvasive brain-machine interfaces to fields ranging from education to job productivity are endless.

Given the devastating conditions for which brain-machine interfaces are often used, an overlooked aspect regarding ultimate success is the patient’s acceptance of a specific BCI/BMI (device and technique, hardware and software, invasive vs. noninvasive, etc.). Fortunately, efforts at understanding patient acceptance are being made (72). In addition, practical problems in the clinical application of the BCI/BMI are increasingly taken into account. Rapid advances in dry electrode technology for EEG-based prostheses are expected to make the use of brain-machine interface less obtrusive.

A final, possibly disquieting, application of brain-machine interfaces has been demonstrated by a group at the University of Washington, with the goal of enhancing interpersonal communication—brain-to-brain interface (BBI). Given the limitations of verbal communication (eg, the inability of a virtuoso violinist to “tell” a novice how to master the violin), a direct brain-to-brain interface has been proposed (71). The demonstration involved the collaboration of two people to perform a simple computer game, the goal of the game being to press a button to fire a cannon that would shoot down a rocket fired to destroy a besieged city--but not shoot down a supply airplane headed for the city. The “sender” could view the game and imagine pressing the “destroy” button when a rocket (but not an airplane) appeared; the “receiver” had one finger placed close to the button. EEG recording from the “sender” was transmitted via the internet to a transcranial magnetic stimulation device placed over the motor cortex of the “receiver” (who was located in a separate building). The EEG activity of the “sender” was able to correctly command the “receiver” (via transcranial magnetic stimulation of the motor cortex) to press the “destroy” button when a rocket appeared over 80% of the time for one pair of participants. For another pair, the success rate was 37.5%, and for a third it was 25%. Analysis of the EEG during rocket versus airplane trials showed that the “sender” in pair one had much greater separation of the EEG log power between the rocket and airplane trials than the “sender” in pair 2, and the separation of the EEG log power for the “sender” in pair three was minimal--demonstrating that the “weak link” was the ability of the “sender” to differentiate between the rocket and airplane trials (in terms of EEG activity) and not the communication between the EEG activity, the transcranial magnetic stimulation device, and the response of the “receiver” (ie, the finger movement, resulting from the transcranial magnetic stimulation, by the “receiver” to push the “destroy” button). Interestingly, in another iteration of the technique dubbed “BrainNet,” 2 “senders” playing a Tetris-like game instruct the “receiver” through transcranial magnetic stimulation of the occipital (visual) cortex of the “receiver” (36). Additional rounds of information can be sent from “senders” to the “receiver” to improve performance. The authors conclude: “Our results point the way to future brain-to-brain interfaces that enable cooperative problem solving by humans using a “social network” of connected brains.”

This final application, the brain-brain interface, emphasizes that along with the technological advances in the BCIs/BMIs that are occurring at virtual “warp speed,” we will need to keep ethical standards evolving with similar diligence to maintain the BCI/BMI as a benefit rather than a curse for humanity.

Clinical vignette

Vignette 1. Control of devices like robotic arms, which have a high degree of freedom, benefits from craniotomy and implantation of intracranial electrodes. Previous electrodes have been implanted within the substance of the brain; the foreign body response in the parenchyma causes concern over the long-term viability of this type of brain-machine interface in clinical use. Clinical efforts have focused on developing nonpenetrating subdural electrode montages for motor control.

A University of Pittsburgh on-going study enrolled a 30-year-old right-handed male with tetraplegia. He had suffered a complete C4 level spinal cord injury 7 years prior to implantation (87). Electrocorticography (ECoG) was recorded using a custom 2 x 4 cm subdural array (composed of silicone 1 mm thick) with 32 platinum disc electrodes: 28 recording electrodes facing the cortex, four ground electrodes facing the dura. The subdural array was placed over the left hand and arm areas. To localize the left hand and arm cortical region, fMRI was performed preoperatively while the patient watched videos of hand and arm movement and was asked to attempt the same movement.

A small craniotomy was made over the left sensorimotor cortex, and intraoperative neuronavigation was used to place the grid over the previously localized area. The connecting wires were passed subcutaneously to the left infraclavicular area. Trials were conducted daily for 28 days until the grid was explanted, following FDA regulations for implantable devices.

Within 7 days, he was able to achieve 87% success in a 2D cursor control environment. When transitioning to 3D control, he started at 10% success in the 2nd week and rapidly improved to 80% control by the end of the 4th week. The ECoG device uses high-gamma band (80 to 100Hz) electrical activity rather than spiking activity of neurons (that at present can only be recorded using penetrating electrodes) (87). EcoG-based interfaces have shown their versatility in both spelling (84) and control of lower or upper limb exoskeletons (08). Penetrating electrodes achieve approximately 90% success after several weeks of training (34). Videos demonstrating these trials are available at: http://journals.plos.org. In another study, a tetraplegic individual has been shown to successfully manipulate a 7 degree-of-freedom prosthetic limb for reaching and grasping in 3 dimensions after 13 weeks of brain-machine interface training (21). Researchers have shown how intracortically recorded signals could be linked in real-time to muscle activation to restore movement in paralyzed humans (13; 02).

Vignette 2. Patients with complete locked-in syndrome are able to communicate one to three characters a minute with scalp EEG techniques (55; 90). A 58-year-old woman who had been anarthric and tetraplegic for 14 years after bilateral pontine infarction had a 96-channel intracortical microelectrode array surgically implanted into the hand/arm area of her motor cortex 5 years previously (07). Clinically, her extraocular movements and head rotation were intact; however, she had complete tetraplegia and absent hypoglossal and vagal nerve function resulting in inaudible speech. With this penetrating brain-machine interface, she was able to control a cursor on a screen to point and click on any image of a keyboard.

QWERTY keyboards were originally designed to minimize typewriters from jamming by forcing the typist to alternate hands for common word combinations - they are not designed for brain-machine interface control, which requires a user to move a single cursor back and forth across an image. A new radial keyboard was created for this patient that reset the cursor to the center after each letter was chosen, minimizing the distance she needed to travel between letter choices. As the letters are chosen, a “most likely” word appears at the side of the screen. This allows the patient to choose a common word by typing only a few letters; for example, the word “quick” appears after selecting “q”, “u”, “i”.

The patient described above was able to achieve 90% accuracy with both types of keyboards. Using the QWERTY keyboard, she was able to communicate at 5 to 6 characters per minute; with the radial keyboard she was typed at 10.4 correct characters per minute--a doubling of speed with only four (or less) letters presented and a 10x improvement over scalp EEG recording. This represents a significant increase in communication for patients with these conditions (07).

Electrocorticogram has also restored communication capabilities for complete LIS patients (84; 61), a major user category of brain-machine interfaces for communication and control. In addition, this technique seems promising regarding the possibility of directly decoding words during imagined speech (49; 04; 51).

Vignette 3. Conventional brain-machine interfaces like those presented in the previous clinical vignettes decode either physiological variables (kinematics, muscular activation) or a number of discrete mental states. The need for substantial subject training and the limitations of the current neural interfaces and algorithmic designs pose severe restrictions on the dexterity and the number of degrees-of-freedom achieved by state-of-the-art brain-machine interfaces for motor substitution. Iturrate and colleagues demonstrated an alternative paradigm able to overcome these limitations by means of decoding cognitive brain signals associated with monitoring processes relevant for achieving goals (35).

In this approach the neuroprosthesis exploits robotic and other artificial intelligence systems for skillfully executing a number of actions, which the subject evaluates as erroneous or correct. The brain-machine interface then exploits the brain correlates of this assessment to learn suitable motor behaviors. More specifically, the EEG correlates exploited reflect the difference of the event-related potential (ERP) for erroneous and correct device actions, a characteristic waveform with prominent fronto-central positive and negative peaks at around 300 and 500 s, respectively. The brain-machine interface implements a reinforcement learning framework employing the detected error-related potential (ErrP) as the reward signal. The user is thus able to “teach” a device to achieve a goal-directed movement. The low-level details of these movements are hardcoded into the device itself, implementing a biomimetic shared-control framework, because it is well-known that low-level execution in animals is delegated to subcortical, spinal cord, and musculoskeletal structures whereas cortical areas provide the abstraction for the desired goal.

The results by Iturrate and colleagues show that 12 subjects were able to teach a 1D cursor, a simulated robotic arm, and most importantly, a real robotic arm operating on a 2D plane to reach targets rapidly, on average after only four repetitions of the target-reaching task (35). More than 12 targets could be reached on average within the fixed time of a single “run” of the experiment, an outcome significantly above that achieved with a random policy. New targets could be learned without the need of retraining a new ErrP EEG decoder, which averaged a sufficient 70% to 75% detection accuracy (thus, teaching is possible despite a nonperfect reward signal). Overall, this alternative and scalable control paradigm is promising for bringing brain-machine interfaces a step closer to dexterous manipulation of neuroprosthetics.

Vignette 4. Brain-machine interfaces have been so far mostly employed in the contexts of communication and motor substitution, where brain signals are translated into commands to a brain-actuated assistive device or software. However, the most recent and promising research direction in the (especially noninvasive) brain-machine interface field regards motor recovery. In this novel setting, the main application pursued so far is upper limb rehabilitation after stroke.

The basic premise of brain-machine interface-based stroke rehabilitation is the idea that the brain plasticity effects that are known to accompany a brain-machine interface user's motor learning efforts during prolonged closed-loop use of a brain-machine interface will translate into functional improvements. A number of phase 1 randomized controlled trials seem to confirm this principle. Pichiorri and colleagues have shown that a group of 14 subacute stroke patients receiving brain-computer interface-based therapy with a virtual-reality feedback exhibited a significantly higher probability of achieving a clinically relevant increase in the Fugl-Meyer Assessment of Motor Recovery After Stroke (FMMA) compared to another group of patients performing motor imagery without feedback (66). FMA improvements correlated with the changes at rest in ipsilesional intrahemispheric connectivity.

This type of intervention is often combined with robotic therapy, where contrary to classical robotic treatment, the movements of the orthotic device are triggered by the brain-machine interface rather than being passive. Ramos-Murguialday and associates have demonstrated the efficacy of this type of intervention (over a sham group) in 28 chronic stroke survivors where the movements of the orthosis occurred randomly (68; 69). Positive changes in Fugl-Meyer Assessment of Motor Recovery After Stroke (FMMA) score were larger in the treatment group and correlated with changes in the fMRI laterality index. Similar findings, albeit with smaller overall effects, were shown by Ang and colleagues, where BCI-triggered robotic therapy was compared to robotic therapy alone and conventional physiotherapy (03).

Another trial of 27 chronic patients by the Millan group at the Swiss Federal Institute of Technology Lausanne (EPFL) showed that clinically relevant FMMA improvements are achieved when a motor attempt BCI is coupled with functional electrical stimulation (FES) for wrist extension movements of the affected hand (09). In this case, the enriched afferences induced by the muscle contractions on top of the proprioceptive feedback are thought to further assist towards functional improvements by reestablishing the intention-action-perception loop. This was the first work to quantitatively evaluate the importance of contingency between the efferent command and the afferent input for promoting functional recovery. A trial suggests that the effect of BCI-based stroke rehabilitation therapies might be greater in chronic rather than acute and subacute populations (30).

Following these positive proof-of-concept studies, larger clinical trials are currently underway to firmly establish the efficacy of brain-machine interface-based stroke interventions. This concept is also likely to extend to other patient groups, eg, spinal cord injury patients.

Vignette 5. Patients with severe brain trauma or late stage neurodegenerative disease like amyotrophic lateral sclerosis (ALS) may completely lose muscle control of their entire body. The situation where a human individual maintains intact or adequate self-awareness and cognition (ie, thus, also adequate remaining brain functionality), but no longer possesses motor functionality (including speaking, blinking or face muscle twitching), has been termed “locked-in syndrome” (LIS). These people are deprived of any means of communication and are essentially “locked” into their own bodies. Locked-in syndrome patients have always been considered the ultimate BCI user group because a BCI theoretically only requires cognitive capacity and spared brain activity to allow one to communicate and interact with their external environment, whereas all other AT will necessitate at least some minimal residual muscular and/or PNS activity.

Despite this theoretical expectation and although several works have reported BCI control by people in incomplete locked-in syndrome (75), there had been no known case of a complete locked-in syndrome patient taking control of a brain-actuated device. Certain relevant claims made were later contested (20). All other BCI application works reported in the literature regard people with residual functionalities who may have access to other communication and control channels and do not fully depend on a BCI. Consequently, it has been so far debatable whether neural-based communication remains possible in complete locked-in syndrome, with some works suggesting that it may in fact be unattainable due to extinction of goal directed thinking of humans staying long in locked-in syndrome.

However, in March 2022, Chaudhary and colleagues reported the first case of BCI control in complete locked-in syndrome, marking a historical breakthrough for BCI (19). Specifically, the researchers implanted two 64 microelectrode arrays in the supplementary and primary motor cortex of a complete locked-in syndrome individual with amyotrophic lateral sclerosis. Although the hardware employed has been similar to other state-of-the-art BCI approaches, what is particularly interesting is that the methodology followed took a step back and revisited the “neurofeedback”-inspired origins of BCI relying on operant conditioning. In particular, the patient learned to modulate neural firing rates based on auditory feedback over the course of 10 to 20 training sessions, and he ultimately used this strategy to successfully select letters and communicate with friends and relatives for more than a year after the BCI-based spelling skills were first acquired. This seminal case study provides for the first time evidence that brain-based volitional communication is possible even in complete locked-in syndrome, a milestone that may be thought of as the “holy grail” of BCI research. However, it is very important for the additional reason that it has verified that future BCI must pay equal attention to accommodating and exploiting the learning capacities of the human brain (even in disease, and even when high quality and low-SNR invasive recordings are at hand) along with the machine learning challenges of the BCI decoder, which have mostly attracted attention in the last 20 years (Perdikis and Millan 2020).