While mind-reading may seem like a distant reality, the foundations of mind reading are grounded in scientific research. The lab of Kenneth Norman, Huo Professor in Computational and Theoretical Neuroscience and Chair of the Department of Psychology, has developed and employed novel applications to detect and measure brain activity. These applications, in conjunction with artificial intelligence (AI), are then used to interpret thoughts. Norman emphasized the potential of his work for advances in brain-computer interfaces, diagnosing mental illness and neurological conditions, and education.
Researchers begin by creating a series of classification and decoding schemes, which are then operated on by AI models to decode thoughts. Developing such schemes resembles circumnavigating the mind by using brain activity as a compass. AI is then used to create a mathematical representation of meaning, what Norman refers to as a “labeled Atlas.”
The decoding process behaves like a function, in which brain activity in the form of numbers is fed into the machine learning algorithm. The model then decodes the information and makes a prediction about the meaning of thoughts, contained in the form of numbers called meaning vectors. If the AI successfully decodes the thought, the input and output numbers will match.
To verify the accuracy of these neural networks, the assigned meaning vectors are then compared with human feedback on the similarity of the two concepts the vectors are supposed to represent.
For example, as Norman explains, if a neural network has assigned similar meaning vectors to say, a dog and a flower, then it becomes clear that something within the model has gone awry.
The integration of artificial intelligence into this work has rendered the interdisciplinary connections between neuroscience, computer science, math, and engineering more robust.
Norman attributed the foundations of his research to cognitive science. Cognitive science involves analyzing the brain during a task. Beyond behavior, the field delves into how areas of the brain behave or coordinate with other brain regions to perform tasks.
With regard to diagnosing mental illness, Norman explained the applications of his work as they have been demonstrated within a proof-of-concept study in collaboration with the University of Pennsylvania. The study built upon Norman’s research to capture and analyze the trajectories of a person’s thoughts while engaged by something negative, and then convey these thoughts back to the patients in the form of neurofeedback.
This technology can also help with therapy by making the person aware of the exact moment that their attention becomes fixated on something negative and help them course-correct, targeting a hallmark of depression. The pilot study enabled the collaborating scientists, with the addition of Yale University, to secure adequate funding for a large-scale study that is currently ongoing.
Additionally, Norman mentioned a new initiative called the Princeton Language and Intelligence (PLI), headed by Sanjeev Arora of the computer science department. The new initiative aims to provide local AI models to Princeton researchers, and it would help support Norman's work.
Norman explained that three principal methods are routinely used to carry out this groundbreaking work: functional magnetic resonance imagery (fMRI), electroencephalography (EEG), and magnetoencephalography (MEG).
Norman’s lab tuned their fMRI machines to detect oxygen level fluctuations within the brain. “If a particular part of the brain is active, then it uses up more oxygen from the blood,” Norman said.
However, blood oxygen level fluctuations are not exclusively correlated with cognitive activity, hence the need for other techniques.
Norman explained that fMRI is capable of localizing brain activity. However, he emphasized that a limitation of this technique is temporal smearing, or a blurry timestamp of the activity, owing to the rapidity of blood flow to the area.
“The blood flow response that brings in the oxygen to feed the neurons happens on the order of seconds,” Norman explained.
EEG measures the electrical activity in the brain and is often used to study epilepsy and localize the region(s) of the brain where the seizures are occurring.
Norman explained that while EEG is advantageous for capturing the rapid “zapping” of brain activity, it is less efficient than fMRI at pinpointing the region from where the activity originated, a phenomenon referred to as spatial smearing.
MEG, the third method, is highly innovative and addresses the limitations of the others. Norman explained that MEG measures “subtle magnetic field fluctuations,” avoiding the spatial smearing that occurs with EEG while maintaining the localization capabilities of fMRI. PNI is set to receive its own MEG machine in early 2024.
Behind this groundbreaking research is a complex process; however, Norman states the process might be simpler than imagined. All you have to do, he suggests, is “convert everything you care about into, you know, numbers in, numbers out.”
Norman also shared limitations of his work.
Norman acknowledges that the brains of two individuals will vary widely from each other because they are, structurally, “as unique as your face” and functionally distinct from one another due to varied life experiences.
Because this research involves utilizing machine learning, how these machines are taught becomes essential to their reliability. Norman credits his current and former colleagues, professor Peter Ramadge in Electrical and Computer Engineering and Dartmouth professor James Haxby, respectively, with “really smart ways of … aligning two people’s brain data so you can combine those people’s data without blurring it out.” This method involves comparing the brain activity of two people as they watch the same movie.
Norman attributed key developments in his work to conversations with colleagues which frequently sparked new ideas or addressed limitations. He gives ample credit to the consistent and efficient collaboration that he has experienced in his more than two decades of time at Princeton as the driving force behind “a lot of really wonderful discoveries.”
Norman also articulated the potential controversy surrounding his work.
He acknowledged that “it’s a powerful tool that is advancing quickly, and there are limits on it now, but it’s an open question what the limits are gonna be in five or 10 years.”
Norman echoed the level of complexity involved in this question, but nevertheless stated that his lab does “a lot of thinking now about what kinds of partnerships [they] want to set up with industry.” Companies whose intentions align with those of Norman and the people working within his lab would reflect his goal of doing “things that benefit society broadly, and not just some company’s bottom line.”
Haley Champion is a News contributor for the ‘Prince.’
Please send corrections to corrections@princeton.edu.
