Nate Silver Discusses the Art of Prediction

The well-known statistician Nate Silver spoke yesterday to a booked Marian Miner Cook Athenaeum at Claremont McKenna College. Silver, who gained national prominence by predicting the results of the 2008 and 2012 national elections, discussed approaches to making predictions based on large quantities of data.

At the talk, Silver examined the topic of prediction by discussing examples such as weather, sports, poker, World War II, and the 2008 housing crisis. 

He emphasized taking a “philosophical” approach to analyzing data.

“A lot of people are really quite lazy … because people think you buy some software or hire a consultant and you can solve all your problems,” Silver said. 

Silver is currently the editor-in-chief of ESPN's FiveThirtyEight blog and a Special Correspondent for ABC News. He first drew attention for developing a system for predicting the performance of Major League Baseball Players. 

During his speech, Silver stressed the necessity to take human bias into account.  

He discussed a point that Facebook’s chief operating officer Sheryl Sandberg raises in her book Lean In, in which Sandberg cites a study that examines how executives responded to identical job applications submitted by female and male candidates.  

“It was the people who said they would never be biased at all who were,” Silver said. “It’s better to say, here’s my point of view, here’s where I come from, and I’m going to try to be honest.” 

“We all have one perspective in a very large world, and we have to be honest about where we’re coming from,” he said.

He also addressed the issue of false positives, which occur when people conclude a meaningful effect or relationship when in fact the data would not support those conclusions.

“Our brains are trained to perceive meaning in random numbers,” he said. “You show people random data, they think it’s made by a human, and vice versa.” 

Silver described his methodology for predicting the 2008 and 2012 presidential elections: averaging the polls to reduce the effect of outliers, determining the ways a candidate could earn 270 electoral votes, and thinking in terms of probability. 

Silver correctly predicted the results of all 50 states in the 2012 election. 

“From my point of view we got a little bit lucky,” Silver said, referring to narrowly decided swing states such as Florida. 

He claimed the Obama campaign used data much more effectively than the Romney campaign in the past election. 

“I think it kind of stems from a kind of arrogance,” he said, explaining that the Romney campaign relied too heavily on the premise that the strength of its message would win voters and thus did not concentrate enough on convincing swing voters.