Results of a Wonderlic-SAT Comparison

Eli Manning and Tom Brady are arguably the smartest pair of quarterbacks to face each other in a Super Bowl. That’s not just hyperbole; there’s data to support it. Manning scored a 39 on the Wonderlic test, and Brady scored a 33, giving them an average score of 36. That’s the highest average ever for the starting quarterbacks in a Super Bowl.

The two starting quarterbacks for Super Bowl XLVII, Colin Kaepernick and Joe Flacco, are no intellectual slouches, either. Flacco scored a respectable 27 on the Wonderlic, and Kaepernick rocked the test with a 37, placing him two standard deviations above the norm. That puts him in the 97th percentile. If he wins the Super Bowl this Sunday, he’ll be the second-smartest quarterback to do so.

Last week, I asked readers to supply me with data for a research project. The Wonderlic test is used by the National Football League to measure the problem-solving abilities of prospective players. The SAT (and the ACT) have long been used as college entrance exams, and both claim to predict college success. My hypothesis is that the Wonderlic — a 12-minute, 50-question test — would be equally good at predicting college success.

The following presents the (I) results, (II) limitations of the research, and (III) some notes about the methodology. (Sorry, I don’t mean to be pretentious or to imply false erudition by using Roman numerals. I just know that some folks are interested in (I) but could give a rat’s butt about (II) or (III), so I thought dividing this post into sections might be helpful. Hopefully by using the phrase “rat’s butt,” I’ve removed all sense of pretense.)

I. Results

Neither the SAT nor the Wonderlic are good at predicting college success, but to my surprise, the SAT is better than the Wonderlic.

The following correlation coefficients resulted when three pair-wise correlations were performed:

• Wonderlic and GPA: r = 0.0086
• SAT and GPA: r = 0.0506
• Wonderlic and SAT: r = 0.2897

When comparing the Wonderlic and college GPA (n = 46), the correlation coefficient was r = 0.0086, meaning that roughly 9% of the variance of college GPAs can be explained by Wonderlic scores.

When comparing the SAT and college GPA (n = 41), the correlation coefficient was r = 0.0506, meaning that roughly 22% of the variance of college GPAs can be explained by SAT scores.

When comparing the Wonderlic and SAT (n = 44), the correlation coefficient was r = 0.2897, meaning that roughly 54% of the variance of college GPAs can be explained by Wonderlic scores.

Though not quite as strongly, these results corroborate my previous findings that neither the SAT nor the Wonderlic is a very good predictor of college success, but both are pretty good predictors of scores on other standardized tests.

II. Limitations

A number of factors discredit the validity of this research, among them:

• Voluntary Response Bias. The majority of respondents were above average in all categories. Additional data is needed from individuals who scored poorly on the SAT/ACT or Wonderlic or who had below-average college GPAs.
• Sample Size. It is difficult to draw conclusions from a sample of just 54 individuals.
• Timing. Those who responded often took the Wonderlic many years after taking the SAT. This is an issue with data from NFL prospects, too; they take the SAT prior to entering college, but they take the Wonderlic at least three years later. Certainly, those years of experience would influence the results.
• Consistency. College GPA is not transferrable. Without a doubt, earning a 3.1 GPA at Harvard University is more impressive than holding a 3.9 at the Univerity of the District of Columbia. Even within the same university, there can be discrepancies; it’s likely more difficult to hold a high GPA if your major is electrical engineering than, say, parks and recreation. Unfortunately, it’s one of the only means of comparing two students from different schools, apart from reputation of the issuing institution.

Consequently, this research should be taken in the spirit it was intended. It it not academic research. It was merely a tongue-in-cheek attempt to show that neither the SAT/ACT nor the Wonderlic test are terribly good at predicting college success.

That said, this analysis could serve as the impetus for an academic research project. By gathering Wonderlic scores from high school students at the same time that they take the SAT, and then tracking them to determine their success in college, the viability of the Wonderlic test as a college entrance exam could be determined. (It should be noted that the Wonderlic Personality Test (WPT) is used by the NFL when evaluating prospective players, but scores on the Wonderlic Basic Skills Test (WBST) are already accepted by some colleges.)

III. Notes About the Method

For consistent comparison, all college exam scores were converted to a scale based on the old SAT (out of 1600). ACT scores were converted using results of a concordance study conducted by the ACT and the College Board. Converting scores from the new SAT to the old SAT used the method described below.

Because the maximum score on the new SAT is 2400 and the maximum score on the old SAT was 1600, the following conversion formula might seem reasonable:

2/3 × new SAT = old SAT

However, there are two reasons that won’t work. First, in addition to covering the same math topics as the old SAT, the new SAT also covers Algebra II. Second, the writing section has proven to be the hardest part of the new test; the average score on the writing section is 493, since its inception in 2005; by comparison, the average scores for math and reading are 516 and 501, respectively, during the same time period.

Using scores from 2000-11, it seems that approximately 67.3% of a student’s score on the new SAT comes from the math and reading sections; the writing section only accounts for about 32.7% of the student’s total score. Second, the average score on the old SAT from 2000-05 was 1024, whereas the average combined score for the math and reading sections on the new SAT from 2005-11 was 1017, which means that the average score on the old SAT was about 0.7% higher than the average combined score on the math and reading sections of the new SAT.

Consequently, for any respondent who listed a new SAT score, I multiplied their score by 0.673 to find their score on just the math and reading sections, and then I multiplied by 1.007 to account for the higher average score on the old SAT. This is obviously an imperfect system. That said, one of the respondents told me that his combined math/reading score on the new SAT was 1390, and this formula yielded an old SAT estimate of 1410. Since the old SAT score should be slightly higher, it seems that the formula is reasonable. I therefore used this formula for all respondents who listed a new SAT score, of which there were only two.

No changes were made to the college GPAs, despite the inherent flaws described above.

Once the data was in comporable form, my good friend Excel was used to perform a linear regression and determine the correlation coefficient.

SAT, Wonderlic, and GPA

Yeah, that’s right. I’m as smart as Greg McElroy, and I’m smarter than Blaine Gabbert. Based on Wonderlic scores, anyway.

I scored a 43, so I guess that means I’m smarter than Phil Simms, too.

The Wonderlic test has been administered to NFL players since the 1970’s, and Phil Simms scored the lowest of all Super Bowl quarterbacks who have taken the test. He led the New York Giants to victory in Super Bowl XXI, but he only got a 10 (out of 50). A score of 10 indicates that a person is literate, but “that’s about all we can say,” according to Charles Wonderlic, Jr. And to think that Phil’s words will be heard by 100 million people when he announces Super Bowl XLVII on Feburary 3.

Ever wonder which NFL players are intellectually inferior to you? You can find out by taking the mini-Wonderlic test at www.nicholascreative.com/footballiq.

And I sure wish you would. Since conducting “research” for my Smart Quarterbacks post two years ago, I’ve been somewhat obsessed by the idea that the Wonderlic — a 12‑minute, 50‑question test — will someday unseat the SAT — an onerous, expensive, 5‑hour ordeal consisting of 10 sections — as the entrance exam for college.

In my wildest fantasies, I envision a world with no entrance exams. Not without reason, mind you. It’s just that the SAT is better at measuring natural intelligence than predicting college success. Then again, so is the Wonderlic. But the Wonderlic takes only 4% as long to complete, so high school students wouldn’t have to sacrifice an entire morning to complete it.

To prove my point, I attempted to gather SAT scores, Wonderlic scores, and college GPAs for a handful of NFL players, and this data has supported my hypothesis — that the Wonderlic test is just as accurate as the SAT at predicting college success (which is to say, it is equally abysmal). But the sample size is far too small to be reliable.

Data. I need more data. Data, data, data.

That’s where you come in.

If you’re willing, please enter your SAT (or ACT) score, your college GPA, and your Wonderlic score in the form below. (If you can’t see the form below, try this Google form.) All of this data will be held confidential. I will only use it to conduct an analysis.

You probably remember your SAT or ACT score and your college GPA. But you probably don’t know your Wonderlic score, because most people have never taken the Wonderlic test. you can take the mini-Wonderlic at www.nicholascreative.com/footballiq to get a fair approximation of your Wonderlic score.

Thanks in advance for your help. I’ll collect data through Wednesday, January 30, and I’ll post my analysis before kick-off on Super Bowl Sunday.

Smart Quarterbacks, the Super Bowl, and SAT Scores

This weekend, when the Pittsburgh Steelers take on the Green Bay Packers in Super Bowl XLV, it’ll be a match-up pitting a very smart quarterback against, well, a guy who’s not exactly the sharpest knife in the drawer.

If you’re like most of the world, you probably don’t perceive Ben Roethlisberger to be very smart.  He attended Miami University, but information about what he studied is considerably harder to find, and few would call him intelligent. After all, he rides his motorcycle without a helmet, frequently fraternizes with underage co‑eds, and associates with people who occasionally urinate in public. So it will come as no surprise that Roethlisberger scored lower on the Wonderlic test — the 50‑question, 12‑minute exam administered by the National Football League to measure the problem-solving ability of players who will enter the draft — than Aaron Rodgers.

The maximum possible score on the Wonderlic test is 50. Roethlisberger scored 25, Aaron Rodgers scored 35. (Wanna know how you compare? Try a sample Wonderlic test for yourself.)

So, does this mean that Rodgers has an advantage in Sunday’s game? Not necessarily.

Below is data from the last ten Super Bowls. The winning quarterback is listed first, and his Wonderlic score is given in parentheses. (Sorry, I couldn’t locate the Wonderlic score of Brad Johnson.) But for the other nine games, the team whose quarterback had a higher Wonderlic score won four times, the team whose quarterback had a lower Wonderlic score won four times, and last year, the two quarterbacks had the same score.

Super Bowl XXXV – 1/28/01
Trent Dilfer, Baltimore Ravens – Fresno State (22)
Kerry Collins, New York Giants – Penn State (30)

Super Bowl XXXVI – 2/3/02
Tom Brady, New England Patriots – Michigan (33)
Kurt Warner, St. Louis Rams – Northern Iowa (29)

Super Bowl XXXVII – 1/26/03
Brad Johnson, Tampa Bay Buccaneers – Florida State (unavailable)
Rich Gannon, Oakland Raiders – Delaware (27)

Super Bowl XXXVIII – 2/1/04
Tom Brady, New England Patriots – Michigan (33)
Jake Delhomme, Carolina Panthers – Louisiana-Lafayette (32)

Super Bowl XXXVIX – 2/6/05
Tom Brady, New England Patriots – Michigan (33)
Donovan McNabb, Philadelphia – Syracuse (14)

Super Bowl XL – 2/5/06
Ben Roethlisberger, Pittsburgh Steelers – Miami, Ohio (25)
Matt Hasselbeck, Seattle – Boston College (29)

Super Bowl XLI – 2/4/07
Peyton Manning, Indianapolis Colts – Tennessee (28)
Rex Grossman, Chicago Bears – Florida (29)

Super Bowl XLII – 2/3/08
Eli Manning, New York Giants – Ole Miss (39)
Tom Brady, New England Patriots – Michigan (33)

Super Bowl XLIII – 2/1/09
Ben Roethlisberger, Pittsburgh Steelers – Miami, Ohio (25)
Kurt Warner, Arizona Cardinals – Northern Iowa (29)

Super Bowl XLIV – 2/7/10
Drew Brees, New Orleans Saints – Purdue (28)
Peyton Manning, Indianapolis Colts – Tennessee (28)

As it turns out, the average Wonderlic score of an NFL player is 20, while the average score of an NFL quarterback is 24. Only one Super Bowl quarterback in the past ten years had a Wonderlic score below the league average. That was Donovan McNabb (14) in 2005. So while a higher Wonderlic score may not imply Super Bowl success, it does seem that quarterbacks who make it to the Super Bowl have above average scores.

Of course, a football team has more than just one player, so it might be more informative to look at the Wonderlic scores for every player on a team. Sadly, I don’t have that kind of time, but such an analysis was done at least once. The Denver Broncos defeated the Green Bay Packers in Super Bowl XXXII; the average Wonderlic score for the Broncos was 20.4, while the average score for the Packers was 19.6.

The Wonderlic test fascinates me. While it may not be the best predictor of success in the NFL, many companies use it to assess prospective employees’ problem-solving abilities. And it got me to thinking — if the Wonderlic test is adequate to predict job success, could it also be used to predict college success?

Consequently, I sought to answer the following question: Could the Wonderlic test be as good a predictor of college success as the SAT?

Unfortunately, acquiring data to analyze this question is no small task. Wonderlic scores of many NFL players are readily available online, but other companies aren’t willing to release the scores of their employees. (Truth be known, the NFL isn’t really willing to release its employees’ scores, either, but players’ scores are interesting trivia for the public, so sports reporters find ways to uncover them.) In a quick search, I was able to locate the Wonderlic scores of scads of NFL players. However, unearthing the college GPA and SAT scores of those players was exorbitantly difficult. I found all three numbers for just six players online (see table below). I tried to acquire the numbers for other players over the phone, but I met with limited success. A typical conversation went something like this:

Woman in Registrar’s Office at University of Virginia: Hello.
Me: Uh, good afternoon, ma’am. I’m trying to locate the GPA and SAT scores of one of your former students.
Woman: Whose information are you looking for, sir?
Me: Matt Schaub.
Woman: And you are?
Me: Patrick Vennebush.
Woman: Are you related to Mr. Schaub?
Me: Um, no, ma’am.
Woman: Are you a prospective employer?
Me: No, ma’am.
Woman: So… why do you need Mr. Schaub’s information?
Me: Well, see, I’m comparing professional football players’ scores on the Wonderlic test…
Woman: The what?
Me: The Wonderlic test. It’s a test they give to professional football players to determine their problem‑solving ability.
Woman: Hold on — Mr. Schaub is a professional football player?
Me: Yes, ma’am. He played quarterback for the University of Virginia from 1999 to 2003, and now he plays for the Houston Texans.
Woman: So, why do you need Mr. Schaub’s GPA and SAT scores?
Me: Well, I’m trying to determine if the Wonderlic test could be used as a predictor of college success. I need Mr. Schaub’s GPA and SAT scores to see if the Wonderlic test was as accurate as the SAT in predicting how well he did in college.
Woman: Well, I can’t just go around giving out information about former students to total strangers.
Me: Yes, I understand, ma’am, but I’m not going to publicize the information. I just want to analyze it.
Woman: And what will you do with your analysis?
Me: Well, I was planning to post the results on my blog.
Woman: So, you write a sports blog?
Me: Well, no, ma’am. It’s actually a math blog.
Woman: A math blog that focuses on sports?
Me: Um, well, no.
Woman: Then what kind of math blog is it?
Me: Well, actually, it’s a blog about math jokes.
Woman: About what?
Me: Math jokes.
Woman: [click]

Several other calls met a similar fate. Consequently, I only have Wonderlic, GPA and SAT scores for six players. But, whatever. Let’s roll with it and see what happens. The three numbers for each player are shown below.

Player	College	Wonderlic	GPA	SAT
Tim Tebow	Florida	22	3.66	890
Brady Quinn	Notre Dame	29	3.00	1030
Peyton Manning	Tennessee	28	3.61	1030
Aaron Rodgers	California	35	3.60	1300
Myron Rolle	Florida State	33	3.75	1340
Ryan Fitzpatrick	Harvard	48	3.20	1580

From this limited sample, three pair-wise correlations were calculated:

• SAT and GPA: r = ‑0.14
• Wonderlic and GPA: r = ‑0.36
• Wonderlic and SAT: 0.95

There’s not a very strong correlation between SAT and GPA. But here’s the thing: the correlation between SAT and GPA for this set of six football players isn’t that much worse than the correlation between SAT and GPA reported in Validity of the SAT for Predicting First-year College Grade Point Average, a study of 151,316 students at 726 four‑year institutions undertaken by the College Board; in that study, r = 0.29.

There’s not a very strong correlation between Wonderlic and GPA, either, but it’s stronger than the correlation between SAT and GPA for the six football players above and for the 151,316 students in the College Board study.

There is, however, a very strong correlation between Wonderlic and SAT, which is perhaps just another way of saying that both tests are equally lousy at predicting college success.

Of course, there are all kinds of reasons that this analysis might be invalid:

• the sample is too small;
• it is difficult to compare GPA from school to school, since it might be more difficult to earn a 3.20 at an Ivy League college than at a public university;
• it is difficult to compare GPA between students within a school, since it might be more difficult to earn a 3.20 in electrical engineering than, say, in parks and recreation;
• and, the grades of college football players may be artificially inflated.

Still, I think I’m onto something here. Wouldn’t it be great if we could replace the four‑hour SAT with the 12‑minute Wonderlic test? The marketing of it would be easy. For school administrators, simply tout a stronger correlation to college success than the SAT, and mention significantly lower costs. For students, simply state, “You can finish the Wonderlic in 5% of the time it takes to complete the SAT! You won’t have to give up your entire Saturday!” Now, wouldn’t that be grand?