In my last post, I posited some explanations for grade inflation at elite colleges in recent years and decades, and mentioned that this poses an information problem for employers and graduate schools. There are other costs as well: it drives students to take easier classes rather than better classes, making students less productive than they otherwise would have been. It’s hard to calculate what these costs are, but it’s safe to say that this is a bad thing.
Here, I propose a simple solution that addresses both the information problem and the selection problem. It’s based on several stats that are used in baseball analysis to compare players from different eras.
I ‘ll use as an example a statistic called OPS+, though virtually any stat in baseball can be adjusted this way. OPS is a stat that combies two other stats, On-Base Percentage (OBP) and Slugging Percentage (SLG). (Yes, it adds unlike quantities. Deal with it.). This stat is designed to measure offensive production by batters. Because offense varies over time, comparing players from different eras can’t be done straight up. Steroid era stats don’t compare to dead ball era stats in any meaningful way: league-average OBP was .345 at the peak of the former and under .300 at the bottom of the latter. To compare two, eras, OPS+ compares a player’s percentages to the league averages and then creates an index that lets you compare players from different eras, with 100 indicating “league average” and higher numbers indicating better seasons. The formula is this:
OPS+ = 100 x (OBP/lgOBP+SLG/lgSLG-1)
(where lgOBP and lgSLG indicate league average for the statistic)
This translates well to calculating GPAs for people who took different classes (for now, let’s stay within the same college.) For a particular student who took classes 1 and 2, the formula would be:
GPA+ = 100 x [(GP1/clGPA1+GP2/clGPA2)/(number of classes)]
(where clGPA1 and clGPA2 indicate class average values in classes 1 and 2)(we also have to adjust for the fact that these aren’t rate stats, hence the difference)
Let’s say the student got a B in each class. His GPA would be 3.0. His GPA+, however, would depend on what grades other people in the same classes received. If the average grade in each class was a C, then:
GPA+ = 100 x [(3.0/2.0+3.0/2.0)/2] = 150
This tells us that the student did 50% better than the average students in two hard clases. Comparatively, if the average grade were a Harvard A-:
GPA+ = 100 x [(3.0/3.7+3.0/3.7)/2] = 81
In this case, the student did 19% worse than the average student in two gut classes. Now expand this to every student for every class, and list their GPA+ on their transcript. (Perhaps it’d be useful to provide the average grade for each class next to the grade the student attained.) Now anyone reading the transcript has a far better measure of whether the student did well in easy classes or challenged him/herself, with the appropriate hit to the GPA. This also removes the incentive to take easy classes only, since a hard class will reflect itself in an appropriately low denominator.)
The above changes will permit comparisons of students at the same school that take different course loads. This is helpful to a degree, and it lets students take classes they need rather than classes they want, but it doesn’t solve the problem of comparing students from different schools. Of course, the current system doesn’t really do that either, and I don’t think it’s a huge problem for people to differentiate between graduates from Stanford and Idaho State (no disrespect to the Bengals).
That issue is solved in OPS+ by adjusting for park effects – certain parks are smaller, others have better backgrounds, still others have wind that blows out more frequently, so some parks are easier to hit in than others. Park effects are calculated by comparing runs a team scores at home and runs they score on the road, and using the ratio between the two to calculate the park effects. This is difficult to accomplish in colleges, since so few students take similar classes at more than one school, but an interested party could still get additional information. For example, a particular employer or graduate school could track how graduates of different schools with similar GPAs turn out as employees or doctoral candidates. If Stanford engineers are 20% more productive at your company than Idaho State engineers of the same GPA, then you should give Stanford applicants an implicit 20% boost in GPA relative to ISU applicants.
With enough data, you can generate these ratios for many schools, and even for particular fields. Park effects can be calculated separately for, say, home runs (which benefit from small parks) and triples (which benefit from large parks and unusual design), or even for left-handers and right-handers. One could imagine calculating separately the implicit differences between Stanford English majors, ISU English Majors, Stanford math majors, and ISU math majors applying to your program.
Of course, few organizations will get enough data to figure this out,* since much output can’t be measured well enough to compare graduates, and you need large sample sizes to make meaningful conclusions. This would be difficult to put together from existing data sets, too, so most likely industries would have to combine data for this to work. Considering the up-front costs, the uncertain benefits, and the collective action problems inherent in rivals working together, I’m not sure we’ll any such analysis any time soon. I wouldn’t definitely say no, though, as identifying market inefficiencies and recruiting talent are going to matter more and more.
*Except the NSA.