Study: Princeton students have poorer cognitive abilities than peers, though methodology is disputed

While MIT, Harvard, Stanford and Yale all made the top five in a new study ranking U.S. colleges according to their students’ cognitive abilities, Princeton ranked 39th, behind Bucknell and Northeastern. Some psychologists, however, are questioning the methods of the study that ranks Princeton poorly.

The study was performed by Lumosity, a website that focuses on improving users’ cognitive abilities through online games. The organization was co-founded by current chief executive officer Kunal Sarkar ’00 and chief scientific officer Michael Scanlon ’01.

The training program features games that target cognitive areas such as memory, speed and problem solving. One game titled “Eagle Eye” quickly flashes a number and a bird in front of a background photo anywhere on the screen. The user must then locate the exact position of the bird and recall the number. Another game called “Penguin Pursuit” tests memory and directional orientation.

Using these games, Lumosity has gathered data from 30 million registered members, and about 60,000 of the members since 2007 were determined to be college students. The ranking was based on the scores of the first game a student played in each of the five areas tested. The final results ranked 411 schools where at least 50 students participated, thus eliminating some smaller schools like the California Institute of Technology from the study.

The rankings were then compiled using each school’s median grand index, a measurement Lumosity researchers developed. Princeton ranked sixth out of the Ivy League schools.

Though Lumosity believes its games improve brain function, psychology professor Andrew Conway, who studies general intelligence and is the principal investigator at the University’s Human Working Memory Lab, said he disagrees.

“The studies that [Lumosity] links to — and when I say 'study,' I mean a peer review journal article — none of them are studies on healthy young adults,” Conway noted. The studies have shown that children with developmental disorders benefit from playing the games, Conway said, but there is no evidence that healthy college students benefit as well.

“In the [study] that we’re referring to and on their website, there is no information provided about reliability or the validity of those brain games,” Conway added. “What that means is we have no way of knowing if they’re even accurately assessing memory, attention, speed or flexibility.”

Lumosity public relations representative Erica Perng said the whole program is designed to train users’ “core cognitive abilities.” She said the organization takes pride in having a research and development team that works to frequently improve its games.

“We’re based in science; we’re based in neuroscience,” Perng said, distinguishing the games on Lumosity’s site from regular video games. “A lot of video games are created for entertainment value.”

Lumosity also compiled data looking at the schools’ performances in individual categories, like memory, speed, attention, problem solving and flexibility. Princeton did not rank in the top 25 except in problem solving, coming in seventh behind MIT, Yale and Harvard.

The study acknowledges several limitations, including the method of selecting college students. Lumosity identified users as college students through the students' use of university email addresses or the university IP address.

Get the best of the ‘Prince’ delivered straight to your inbox. Subscribe now »

Furthermore, the study is based on the voluntary participation of users who subscribe to Lumosity. These individuals may not be representative of the general student population at each school.

Conway noted that these flaws raised questions about the validity of the study as a whole.

“This isn’t a training study. They didn’t have students do an intelligence test, then play their brain games and then do an intelligence test again. They just looked at whoever happened to go to their website, and all they needed was 50 students from a university,” Conway said. “It could have been university employees; it could have been anyone. That’s the other thing. We don’t know who’s in the sample.”

Yet the study aimed to analyze student intelligence from a new angle, according to Daniel Sternberg, who performed the study. He said these spontaneous games could provide an alternative measure of intelligence that differs from rankings based on students’ scores on standardized tests like the SAT or ACT.

Sternberg said these results may help students choose which institutions to attend, by providing a new type of ranking based on different criteria than traditional college rankings, like U.S. News & World Report or Forbes.

“Right now, most of the measures of universities and students at universities are based on academic performances and academic characteristics,” Sternberg said. “While we expect those to be related to cognitive measures, no one ever looked at these direct measures of cognitive performance — things like memory, speed of processing, attention — within particular universities and comparing them to each other.”

Conway, on the other hand, said the results are misleading and do not reflect individual students’ levels of intelligence.

“These are ranking colleges,” Conway said, referring to Lumosity’s study. “They’re not ranking individual students within a college. So these correlations that they’re reporting between their Lumosity game scores and college SAT scores are at the level of the university, not at the level of a student.”

Though this is the first year that Lumosity conducted a study focused on college students, it hopes to do more research in the coming years that may take these shortcomings into account.

“Research is really important to us,” Perng said. “At the core of it, we’re a neuroscience company and we’re always trying to learn more things.”