A team of researchers at the University of California, San Francisco has created a bilingual brain implant that utilizes artificial intelligence to help a stroke survivor effectively communicate in both Spanish and English languages.
Over the course of several years, around twelve scientists from the university’s Center for Neural Engineering and Prostheses have collaborated to develop a decoding system capable of translating the man’s brain signals into coherent sentences in both languages, subsequently displaying them on a screen.
An article published on May 20 in Nature Biomedical Engineering details the research conducted on a man known as Pancho. Following a stroke in the early 2000s, he was left severely paralyzed at the age of 20, unable to speak clearly but capable of making inarticulate sounds. Pancho, a native Spanish speaker who acquired English as an adult, received a neural implant in February 2019 under the guidance of Dr. Edward Chang.
Operating on an AI technique called a neural network, researchers trained the implant to interpret words based on Pancho’s brain activity patterns when he attempted to vocalize them. This method facilitated the brain implant, also known as a brain-computer interface device, to process data in a manner reminiscent of the human brain.
Through advancements made by 2021, Pancho’s ability to communicate improved significantly, albeit only in English.
The team from Dr. Chang’s research group expressed the importance of bilingual communication and the need to develop systems allowing bilingual individuals to communicate effectively in both languages, each contributing uniquely to their personality and worldview.
Building on their 2021 findings, the researchers were able to develop a bilingual decoding system for Pancho’s brain implant, enabling communication in both Spanish and English.
Enabling language switching based on preference
After discovering the presence of simultaneous cortical activity for both languages in Pancho’s brain long after his paralysis, the scientists realized they could utilize this to train a bilingual brain implant without the necessity for separate language-specific training.
The research group led by Dr. Chang sought to demonstrate the feasibility of transfer learning across languages, where data collected in one language could accelerate the training process for the second language due to their reliance on the brain activity associated with the intended vocal-tract movements, regardless of language.
In a subsequent effort in 2022, they used the artificial neural network to train Pancho’s implant on the distinct neural patterns generated by his bilingual speech.
Pancho successfully utilized the bilingual decoding system embedded in his brain implant to switch between languages during conversations based on his preferences, showcasing the potential for a “bilingual speech neuroprosthesis” to restore natural communication among bilingual individuals with paralysis, as outlined in the May 20 article.
For additional content from NBC Latino,subscribe to the weekly newsletter.
