Smart Quarterbacks, the Super Bowl, and SAT Scores

This weekend, when the Pittsburgh Steelers take on the Green Bay Packers in Super Bowl XLV, it’ll be a match-up pitting a very smart quarterback against, well, a guy who’s not exactly the sharpest knife in the drawer.

If you’re like most of the world, you probably don’t perceive Ben Roethlisberger to be very smart.  He attended Miami University, but information about what he studied is considerably harder to find, and few would call him intelligent. After all, he rides his motorcycle without a helmet, frequently fraternizes with underage co‑eds, and associates with people who occasionally urinate in public. So it will come as no surprise that Roethlisberger scored lower on the Wonderlic test — the 50‑question, 12‑minute exam administered by the National Football League to measure the problem-solving ability of players who will enter the draft — than Aaron Rodgers.

The maximum possible score on the Wonderlic test is 50. Roethlisberger scored 25, Aaron Rodgers scored 35. (Wanna know how you compare? Try a sample Wonderlic test for yourself.)

So, does this mean that Rodgers has an advantage in Sunday’s game? Not necessarily.

Below is data from the last ten Super Bowls. The winning quarterback is listed first, and his Wonderlic score is given in parentheses. (Sorry, I couldn’t locate the Wonderlic score of Brad Johnson.) But for the other nine games, the team whose quarterback had a higher Wonderlic score won four times, the team whose quarterback had a lower Wonderlic score won four times, and last year, the two quarterbacks had the same score.

Super Bowl XXXV – 1/28/01
Trent Dilfer, Baltimore Ravens – Fresno State (22)
Kerry Collins, New York Giants – Penn State (30)

Super Bowl XXXVI – 2/3/02
Tom Brady, New England Patriots – Michigan (33)
Kurt Warner, St. Louis Rams – Northern Iowa (29)

Super Bowl XXXVII – 1/26/03
Brad Johnson, Tampa Bay Buccaneers – Florida State (unavailable)
Rich Gannon, Oakland Raiders – Delaware (27)

Super Bowl XXXVIII – 2/1/04
Tom Brady, New England Patriots – Michigan (33)
Jake Delhomme, Carolina Panthers – Louisiana-Lafayette (32)

Super Bowl XXXVIX – 2/6/05
Tom Brady, New England Patriots – Michigan (33)
Donovan McNabb, Philadelphia – Syracuse (14)

Super Bowl XL – 2/5/06
Ben Roethlisberger, Pittsburgh Steelers – Miami, Ohio (25)
Matt Hasselbeck, Seattle – Boston College (29)

Super Bowl XLI – 2/4/07
Peyton Manning, Indianapolis Colts – Tennessee (28)
Rex Grossman, Chicago Bears – Florida (29)

Super Bowl XLII – 2/3/08
Eli Manning, New York Giants – Ole Miss (39)
Tom Brady, New England Patriots – Michigan (33)

Super Bowl XLIII – 2/1/09
Ben Roethlisberger, Pittsburgh Steelers – Miami, Ohio (25)
Kurt Warner, Arizona Cardinals – Northern Iowa (29)

Super Bowl XLIV – 2/7/10
Drew Brees, New Orleans Saints – Purdue (28)
Peyton Manning, Indianapolis Colts – Tennessee (28)

As it turns out, the average Wonderlic score of an NFL player is 20, while the average score of an NFL quarterback is 24. Only one Super Bowl quarterback in the past ten years had a Wonderlic score below the league average. That was Donovan McNabb (14) in 2005. So while a higher Wonderlic score may not imply Super Bowl success, it does seem that quarterbacks who make it to the Super Bowl have above average scores.

Of course, a football team has more than just one player, so it might be more informative to look at the Wonderlic scores for every player on a team. Sadly, I don’t have that kind of time, but such an analysis was done at least once. The Denver Broncos defeated the Green Bay Packers in Super Bowl XXXII; the average Wonderlic score for the Broncos was 20.4, while the average score for the Packers was 19.6.

The Wonderlic test fascinates me. While it may not be the best predictor of success in the NFL, many companies use it to assess prospective employees’ problem-solving abilities. And it got me to thinking — if the Wonderlic test is adequate to predict job success, could it also be used to predict college success?

Consequently, I sought to answer the following question: Could the Wonderlic test be as good a predictor of college success as the SAT?

Unfortunately, acquiring data to analyze this question is no small task. Wonderlic scores of many NFL players are readily available online, but other companies aren’t willing to release the scores of their employees. (Truth be known, the NFL isn’t really willing to release its employees’ scores, either, but players’ scores are interesting trivia for the public, so sports reporters find ways to uncover them.) In a quick search, I was able to locate the Wonderlic scores of scads of NFL players. However, unearthing the college GPA and SAT scores of those players was exorbitantly difficult. I found all three numbers for just six players online (see table below). I tried to acquire the numbers for other players over the phone, but I met with limited success. A typical conversation went something like this:

Woman in Registrar’s Office at University of Virginia: Hello.
Me: Uh, good afternoon, ma’am. I’m trying to locate the GPA and SAT scores of one of your former students.
Woman: Whose information are you looking for, sir?
Me: Matt Schaub.
Woman: And you are?
Me: Patrick Vennebush.
Woman: Are you related to Mr. Schaub?
Me: Um, no, ma’am.
Woman: Are you a prospective employer?
Me: No, ma’am.
Woman: So… why do you need Mr. Schaub’s information?
Me: Well, see, I’m comparing professional football players’ scores on the Wonderlic test…
Woman: The what?
Me: The Wonderlic test. It’s a test they give to professional football players to determine their problem‑solving ability.
Woman: Hold on — Mr. Schaub is a professional football player?
Me: Yes, ma’am. He played quarterback for the University of Virginia from 1999 to 2003, and now he plays for the Houston Texans.
Woman: So, why do you need Mr. Schaub’s GPA and SAT scores?
Me: Well, I’m trying to determine if the Wonderlic test could be used as a predictor of college success. I need Mr. Schaub’s GPA and SAT scores to see if the Wonderlic test was as accurate as the SAT in predicting how well he did in college.
Woman: Well, I can’t just go around giving out information about former students to total strangers.
Me: Yes, I understand, ma’am, but I’m not going to publicize the information. I just want to analyze it.
Woman: And what will you do with your analysis?
Me: Well, I was planning to post the results on my blog.
Woman: So, you write a sports blog?
Me: Well, no, ma’am. It’s actually a math blog.
Woman: A math blog that focuses on sports?
Me: Um, well, no.
Woman: Then what kind of math blog is it?
Me: Well, actually, it’s a blog about math jokes.
Woman: About what?
Me: Math jokes.
Woman: [click]

Several other calls met a similar fate. Consequently, I only have Wonderlic, GPA and SAT scores for six players. But, whatever. Let’s roll with it and see what happens. The three numbers for each player are shown below.

Player	College	Wonderlic	GPA	SAT
Tim Tebow	Florida	22	3.66	890
Brady Quinn	Notre Dame	29	3.00	1030
Peyton Manning	Tennessee	28	3.61	1030
Aaron Rodgers	California	35	3.60	1300
Myron Rolle	Florida State	33	3.75	1340
Ryan Fitzpatrick	Harvard	48	3.20	1580

From this limited sample, three pair-wise correlations were calculated:

• SAT and GPA: r = ‑0.14
• Wonderlic and GPA: r = ‑0.36
• Wonderlic and SAT: 0.95

There’s not a very strong correlation between SAT and GPA. But here’s the thing: the correlation between SAT and GPA for this set of six football players isn’t that much worse than the correlation between SAT and GPA reported in Validity of the SAT for Predicting First-year College Grade Point Average, a study of 151,316 students at 726 four‑year institutions undertaken by the College Board; in that study, r = 0.29.

There’s not a very strong correlation between Wonderlic and GPA, either, but it’s stronger than the correlation between SAT and GPA for the six football players above and for the 151,316 students in the College Board study.

There is, however, a very strong correlation between Wonderlic and SAT, which is perhaps just another way of saying that both tests are equally lousy at predicting college success.

Of course, there are all kinds of reasons that this analysis might be invalid:

• the sample is too small;
• it is difficult to compare GPA from school to school, since it might be more difficult to earn a 3.20 at an Ivy League college than at a public university;
• it is difficult to compare GPA between students within a school, since it might be more difficult to earn a 3.20 in electrical engineering than, say, in parks and recreation;
• and, the grades of college football players may be artificially inflated.

Still, I think I’m onto something here. Wouldn’t it be great if we could replace the four‑hour SAT with the 12‑minute Wonderlic test? The marketing of it would be easy. For school administrators, simply tout a stronger correlation to college success than the SAT, and mention significantly lower costs. For students, simply state, “You can finish the Wonderlic in 5% of the time it takes to complete the SAT! You won’t have to give up your entire Saturday!” Now, wouldn’t that be grand?