Student analysis of room draw sparks renewed conversation

In the most recent development concerning University student room draw, Adam Chang ’20 and Yang Song ’20 found that draw groups with more students are more likely to receive earlier draw times, draw ordering is the same across 2018 and 2019 for those in the same draw groups, and students other than current juniors are affected by the randomization error previously disclosed by the University. Some of these conclusions, which were initially sent to the RockyWire listserv, refute claims made by University Housing.

Chang and Song gathered data on room draw order and analyzed it using Python. Their analysis revealed that larger room draw groups, on average, received earlier draw times, as shown in the group size plots below. Upperclass students in the 2019–20 upperclass draw who drew in larger groups had more favorable average positions than those who drew in smaller groups. The correlation between group size and average position among both seniors and juniors approached linearity.

Group size correlates with average position among juniors in room draw. Courtesy of Yang Song and Adam Chang.

Group size correlates with average position among seniors in room draw. Courtesy of Yang Song and Adam Chang.

“A weighted average is calculated by adding the points for each member and dividing by the number of members in the group,” according to the Housing website. This language indicates that draw group size should not affect the group’s draw time.

Chang and Song stated that the problem was likely caused by Housing making a simple mistake.

“The graphs above seemed rather linear … so we had a hunch that individual students were being drawn,” they wrote in their analysis.

They surmised that Housing had two options — they could have drawn by group, meaning that “each group would be chosen with equally likely probability,” or by person, meaning that an eight-person group would be eight times as likely to be chosen before an individual student drawing for a single.

The second choice would have resulted in the housing mistakes that Chang and Song found, according to the simulation presented in their report.

Chang and Song ran a simulation showing draw by person rather than by group. Courtesy of Adam Chang and Yang Song.

Get the best of ‘the Prince’ delivered straight to your inbox. Subscribe now »

This pattern persists across residential college draws as well, as demonstrated in the plots below.

Residential college draws show the same pattern of favoring larger groups. Courtesy of Adam Chang and Yang Song.

The analysis also indicates that hundreds of groups this year retained the same draw orders that they received last year, and that upperclass and independent room draws are in the same order each year, as indicated by the plot below.

“We see that the plot is strictly increasing, meaning that if one group had a better draw time than another group in 2018, then it will for certain have a better draw time than the same group in 2019,” Song wrote in a blog post on Medium, describing a plot “showing the relative position of 2018 draw groups vs 2019 draw groups (the same group across two years).”

“Upperclass and independent draw are in the same order each year,” Chang and Song wrote in their analysis. “This is also true for [Residential] College/Upperclass draws, and so on, in the same year.”

Upperclass and independent draw correlations. Courtesy of Yang Song and Adam Chang.

Chang and Song also suggest that current and rising juniors, who are referenced in the official Housing email, were affected by the randomization error, as demonstrated in the plot above.

"The trend is less obvious, since there are fewer students (and even fewer who stay in the same draw groups), but there still seems to be some trend on the diagonal, meaning earlier draw times in Forbes last year = earlier draw times in Upperclass this year," Chang and Song wrote.

“A lot more people should receive $1000,” they concluded.

Deputy University Spokesperson Mike Hotchkiss referred to Dorian Johnson’s earlier comment about Housing working with CBORD, the vendor that provides software to manage the draw process. Hotchkiss said the cause of the issue is not yet known.

While Chang participated in room draw, Song is a Dormitory Assistant (DA) and selected his room in a separate room draw for DAs, so he was unaffected by this year’s randomization error. He suspected room draw was biased in previous years but had never run any analyses until this year.

On Monday, the two ORF 350: Analysis of Big Data classmates were eating at Colonial Club when they saw the article by The Daily Princetonian about reactions to room draw being unfair. Chang then suggested that the two could turn the news into a data project.

They quickly realized that all of the room draw data was online and publicly available — and the URL links themselves could be changed to acquire draw lists not otherwise listed on the MyHousing website.

The ‘Prince’ downloaded Chang and Song’s code and independently confirmed the validity of their data analysis.

HOUSING’S RESPONSE TO STUDENTS

On April 8, Amy Cass ’20 reached out to the Housing office with room draw concerns. She had noticed similarities in the 2018 and 2019 ordering of draw groups around her own.

“I was appalled to see that this year’s list was almost identical to last year's list — anyone who drew with the same people received roughly the same position within their grade as last year,” Cass wrote in an email to Housing.

“Furthermore, there is a clear lack of transparency given that the system presents itself to students as randomized and therefore equalizing when it is not,” Cass continued in the email.

She asked the Housing office to “correct this injustice promptly.”

Cass eventually forwarded her email on April 10 to Associate Director for Student Housing Angela Hodgeman, who responded to Cass’s concerns on April 16, five days after she acknowledged having received the message.

In her response, Hodgeman stated that Housing had run its own statistical analysis on this year’s draw but believed the results to be inconclusive.

“Once we hear back from the vendor we will be happy to share with you whatever information they give to us regarding the room draw randomization algorithm,” Hodgeman wrote.

Cass eventually received an email similar to the one sent to all students on April 25 by Director of Housing Dorian Johnson.

After receiving the April 25 email, Devin Plumb ’21 reached out to Housing as well and received a similar response.

“The top draw times for 2020 and 2021 are almost exclusively large groups; the bottom draw times are almost exclusively singles,” Plumb wrote in his email to Housing. “It is theoretically possible this is simply random, but highly statistically improbable.”

“All groups have a group number including students who apply alone. The size of the group is not considered in the randomization algorithm,” Hodgeman wrote on the same day in response to Plumb’s concerns.

Chang said that he’s been consistently unhappy with Housing’s handling of the issue as well.

“I think there’s a pattern of negligence. People have raised the issue, and Housing failed to hear people out,” Chang said.

The Housing office has also removed all non-upperclass 2019 room draw times and edited the 2019 upperclass draw information to remove independents and draw group numbers.

When asked about why non-upperclassmen room draw lists were removed from the MyHousing website, Hotchkiss said that the upperclass draw is the only open draw, and all previous draw rounds have been closed.

He added that independents have been removed from the upperclass draw time documents, as all students who chose rooms in earlier room selections, including independent and Spelman draws, automatically forfeit their upperclass draw time and thus are removed from the list.

Song believed the removal of this information did not occur last year. The links to last year’s later draw time information, including upperclass draw times, showed no evidence of redaction at the time of publication.

LOOKING TO THE FUTURE

Students agreed that Chang and Song’s analysis merits further action from Housing, as well as an investigation into how long the room draw process has been broken and how the University intends to fix these issues in the future.

“The cleanest answer would be a redraw. If the Housing Office feels that is impractical, then financial compensation for affected students might be the best alternative,“ Plumb said. “But clearly not all students were affected equally. Exclusively and uniformly reimbursing groups in the bottom half of draw times seems very arbitrary.”

Although Cass agreed that redoing room draw would be an optimal solution, she was glad that Housing has offered financial compensation.

Cass mentioned, however, that she believes her original email to Housing, sent on April 8, was received early enough to potentially have redone room draw.

While many students conceded the small likelihood of redoing room draw in May, some called for an investigation and more action from Housing to prevent problems in the future.

Chang stated that one of his motivations was to “inform and rile people up, because housing has swept things under the table.” He appears to have succeeded in tapping into student frustrations and raising student engagement surrounding the issue.

“[Chang and Song’s] report seems well written and well produced, and releasing the work publicly in a Jupyter notebook is a high standard for reproducing their analysis,” said Robert Liu ’20, who received Chang’s email through the Forbes ReInnformer listserv and forwarded that email to the WhitmanWire and ButlerBuzz listservs.

“Based on the presented analysis, it seems that the room draw process has been broken for the last few years at least,” Liu continued. “I would like to see a report from Housing with similar levels of competence as the student housing report.”

Many students have taken to the Facebook group Tiger Confessions to express their frustrations, both before and after the student analysis.

“Isn’t it a violation of the fair housing act [sic] for larger groups to have systematically higher draw positions despite housing literally lying to us that each group no matter the size is treated equally? If people knew this why would anyone in the [sic] right mind draw in a single group? Housing is a $10k scam …” said a post on Tiger Confessions.

Another post said that Housing “should redraw with OIT randomizing the list not this shit contractor. there [sic] are so many more inequities, lies, and mistakes plaguing draw times than those that are being compensated for currently. if [sic] housing actually compensated everyone wronged by the “randomization,” they’ll be losing a hell of a lot more than the current $220,000 pledged in the email.”

Students are especially interested in how Housing will ensure that this problem doesn’t occur in the future, including the possibility of switching housing contractors.

“The housing department tried to pass responsibility off to a third party in their email, but ultimately it's the Housing Department’s responsibility to ensure that the company they hired actually does their job,” David McElroy ’21 said.

Plumb expressed similar sentiments, saying, “It doesn’t matter who we directly hold responsible. What matters is dealing with what happened and preventing it from happening again. And in that sense, Housing is responsible.”

Song concluded that the campus community must be involved in future changes.

“I think the people who were affected in the worst way should have some say into what happens to the system in future years,” Song said.