Results of a Wonderlic-SAT Comparison

Eli Manning and Tom Brady are arguably the smartest pair of quarterbacks to face each other in a Super Bowl. That’s not just hyperbole; there’s data to support it. Manning scored a 39 on the Wonderlic test, and Brady scored a 33, giving them an average score of 36. That’s the highest average ever for the starting quarterbacks in a Super Bowl.

The two starting quarterbacks for Super Bowl XLVII, Colin Kaepernick and Joe Flacco, are no intellectual slouches, either. Flacco scored a respectable 27 on the Wonderlic, and Kaepernick rocked the test with a 37, placing him two standard deviations above the norm. That puts him in the 97th percentile. If he wins the Super Bowl this Sunday, he’ll be the second-smartest quarterback to do so.

Last week, I asked readers to supply me with data for a research project. The Wonderlic test is used by the National Football League to measure the problem-solving abilities of prospective players. The SAT (and the ACT) have long been used as college entrance exams, and both claim to predict college success. My hypothesis is that the Wonderlic — a 12-minute, 50-question test — would be equally good at predicting college success.

The following presents the (I) results, (II) limitations of the research, and (III) some notes about the methodology. (Sorry, I don’t mean to be pretentious or to imply false erudition by using Roman numerals. I just know that some folks are interested in (I) but could give a rat’s butt about (II) or (III), so I thought dividing this post into sections might be helpful. Hopefully by using the phrase “rat’s butt,” I’ve removed all sense of pretense.)

I. Results

Neither the SAT nor the Wonderlic are good at predicting college success, but to my surprise, the SAT is better than the Wonderlic.

The following correlation coefficients resulted when three pair-wise correlations were performed:

• Wonderlic and GPA: r = 0.0086
• SAT and GPA: r = 0.0506
• Wonderlic and SAT: r = 0.2897

When comparing the Wonderlic and college GPA (n = 46), the correlation coefficient was r = 0.0086, meaning that roughly 9% of the variance of college GPAs can be explained by Wonderlic scores.

When comparing the SAT and college GPA (n = 41), the correlation coefficient was r = 0.0506, meaning that roughly 22% of the variance of college GPAs can be explained by SAT scores.

When comparing the Wonderlic and SAT (n = 44), the correlation coefficient was r = 0.2897, meaning that roughly 54% of the variance of college GPAs can be explained by Wonderlic scores.

Though not quite as strongly, these results corroborate my previous findings that neither the SAT nor the Wonderlic is a very good predictor of college success, but both are pretty good predictors of scores on other standardized tests.

II. Limitations

A number of factors discredit the validity of this research, among them:

• Voluntary Response Bias. The majority of respondents were above average in all categories. Additional data is needed from individuals who scored poorly on the SAT/ACT or Wonderlic or who had below-average college GPAs.
• Sample Size. It is difficult to draw conclusions from a sample of just 54 individuals.
• Timing. Those who responded often took the Wonderlic many years after taking the SAT. This is an issue with data from NFL prospects, too; they take the SAT prior to entering college, but they take the Wonderlic at least three years later. Certainly, those years of experience would influence the results.
• Consistency. College GPA is not transferrable. Without a doubt, earning a 3.1 GPA at Harvard University is more impressive than holding a 3.9 at the Univerity of the District of Columbia. Even within the same university, there can be discrepancies; it’s likely more difficult to hold a high GPA if your major is electrical engineering than, say, parks and recreation. Unfortunately, it’s one of the only means of comparing two students from different schools, apart from reputation of the issuing institution.

Consequently, this research should be taken in the spirit it was intended. It it not academic research. It was merely a tongue-in-cheek attempt to show that neither the SAT/ACT nor the Wonderlic test are terribly good at predicting college success.

That said, this analysis could serve as the impetus for an academic research project. By gathering Wonderlic scores from high school students at the same time that they take the SAT, and then tracking them to determine their success in college, the viability of the Wonderlic test as a college entrance exam could be determined. (It should be noted that the Wonderlic Personality Test (WPT) is used by the NFL when evaluating prospective players, but scores on the Wonderlic Basic Skills Test (WBST) are already accepted by some colleges.)

III. Notes About the Method

For consistent comparison, all college exam scores were converted to a scale based on the old SAT (out of 1600). ACT scores were converted using results of a concordance study conducted by the ACT and the College Board. Converting scores from the new SAT to the old SAT used the method described below.

Because the maximum score on the new SAT is 2400 and the maximum score on the old SAT was 1600, the following conversion formula might seem reasonable:

2/3 × new SAT = old SAT

However, there are two reasons that won’t work. First, in addition to covering the same math topics as the old SAT, the new SAT also covers Algebra II. Second, the writing section has proven to be the hardest part of the new test; the average score on the writing section is 493, since its inception in 2005; by comparison, the average scores for math and reading are 516 and 501, respectively, during the same time period.

Using scores from 2000-11, it seems that approximately 67.3% of a student’s score on the new SAT comes from the math and reading sections; the writing section only accounts for about 32.7% of the student’s total score. Second, the average score on the old SAT from 2000-05 was 1024, whereas the average combined score for the math and reading sections on the new SAT from 2005-11 was 1017, which means that the average score on the old SAT was about 0.7% higher than the average combined score on the math and reading sections of the new SAT.

Consequently, for any respondent who listed a new SAT score, I multiplied their score by 0.673 to find their score on just the math and reading sections, and then I multiplied by 1.007 to account for the higher average score on the old SAT. This is obviously an imperfect system. That said, one of the respondents told me that his combined math/reading score on the new SAT was 1390, and this formula yielded an old SAT estimate of 1410. Since the old SAT score should be slightly higher, it seems that the formula is reasonable. I therefore used this formula for all respondents who listed a new SAT score, of which there were only two.

No changes were made to the college GPAs, despite the inherent flaws described above.

Once the data was in comporable form, my good friend Excel was used to perform a linear regression and determine the correlation coefficient.