‘Economic Value’ Paper a Mistrial at Best

Part 1 of 2

I’m responding to Simkovic and McIntyre not because their Am Law Daily article contains even one valid criticism of my original article but for two separate reasons. First, although most of the weaknesses of “The Economic Value of a Law Degree” (“Economic Value”) were uncovered promptly after its publication, surprisingly there are still a few substantive matters I haven’t seen raised elsewhere. Second, I expect “Economic Value” or its authors are going to pop up again sooner or later, and my life would be a lot easier if there were a reference post to link to detailing its flaws. Bonus, I’m a masochist, so while I can’t promise this will be the last time I write on this topic, I promise I will try to try.

We begin.

Criticism 1:

“In a column published on July 25, 2013, Am Law Daily contributor Matt Leichter repeated many misrepresentations of our research that originally appeared on Above the Law—even after Above the Law posted corrections to its article and after we refuted many of these misrepresentations.”

False: All the representations I used from the Above the Law article survived the correction: The authors of “Economic Value” did not account for graduates after 2008 (page 13), include tuition costs in their lifetime value calculations (Table 7), or distinguish among graduates from different law schools (pages 49-50).

Secondly, nothing in the ATL post was relevant to my argument that “Economic Value” did not show that legal education causes higher earnings and was a non sequitur for legal education reform, nor was it meant to be. It was meant to show that the “Internet promptly erupted” when “Economic Value” first appeared.

The authors’ lack of basic reading comprehension skills is disappointing.

Criticism 2:

“Here are a few—though, by no means all—of Leichter’s misrepresentations. He claims, for instance, that we ‘assume law school pays off equally for non-lawyers’ when we do not make assumptions; rather, we measure the earnings premium regardless of occupation.”

False: Simkovic and McIntyre’s human capital theory requires them to assume that law school pays off equally for non-lawyers. Their theory states that the knowledge and skills imparted in law school improve graduates’ workplace performance as reflected by their higher earnings, even if they don’t work as lawyers. Importantly, people attempting to learn legal doctrines and skills outside of law school would be hopelessly confused without law professors’ guidance. Thus, insisting that one is “measuring a premium” is not a theory that explains why legal education causes higher earnings.

Without explanation, the authors reject signaling theory, an alternative that appears in an article by David Card cited in footnote 4 (PDF) of “Economic Value.” Card’s article displays a chart, though outdated, showing that professional school graduates make significantly more money than the trend predicts. Card identifies this phenomenon as the “sheepskin effect,” which means the additional earnings degree-holders obtain over what they would have earned with just an additional year of schooling.[i] Given that most states require a law degree to practice law, the sheepskin effect, which doubles here as a “licensed professional effect,” is probably very pronounced. The utility of the 58,000^th minute of law school that the ABA mandates is very high indeed.

At this point, we should declare “Economic Value” a mistrial. The authors overeagerly tested the data without thinking through the theory discussed in their own citations. A “premium” cannot be measured until someone compares the earnings of law school graduates to the earnings of those who’ve completed all or nearly all the required law school coursework but didn’t graduate. If the earnings of both groups are the same, then the authors have a basis for claiming that law school increases people’s earnings based on the human capital theory.[ii] This study still wouldn’t necessarily invalidate all alternative theories, like law schools putting zorpqaq in their water fountains that makes people more productive, but it’d be a significant improvement. If the data on dropouts aren’t available, then for the sake of the highly indebted students we should assume that law school does not build much human capital.

But a mistrial is too generous for “Economic Value” because it still omits independent variables that increase lawyers’ earnings over non-lawyers’. Skipping ahead in their response, Simkovic and McIntyre contradict themselves on why graduates’ occupations matter:

“The literature on two forces often cited as major threats to lawyer earnings—outsourcing and automation—finds that jobs that require judgment and decision making have been less susceptible to these pressures than those that can be reduced to simple, repetitive, rule-based tasks. Several economists have argued that the tasks performed by educated professionals such as doctors and lawyers are harder to outsource or automated [sic] than jobs performed by those with less education.”

In other words, an occupation’s wages are determined not by education but by mundane market forces like the demand for the goods and services produced by the occupation and the supply of workers capable of meeting that demand. Demand for lawyers tends to be more inelastic than in other occupations because lawyers require “judgment” and “decision making,” which increase their wages. Adding the fact that wealthy people and corporations tend to spend heavily on legal services to signal their resources establishes a pretty clear explanation for why Simkovic’s and McIntyre’s law school graduates had higher incomes than similar college graduates. Education likely had little to do with it. A barista-barrister will not be paid a higher wage than a barista-bachelor for learning the material covered in law school.

Consequently, elasticity of labor demand in occupations also explains why it matters that so many law school graduates don’t work as lawyers, an observation Simkovic adamantly argues is inappropriate to investigate because “occupation is an outcome variable, not a pretreatment covariate.” But this is just econometrician-speak for “But if you reject our theory then we can’t use our sophisticated mathematical methodology!” Too bad: It’s not my problem that Simkovic and McIntyre adopted a theory that doesn’t account for all the variables that affect occupational wages (like the elasticity of labor demand)—variables the authors only acknowledge opportunistically to dismiss mass law graduate unemployment as a cyclical rather than structural phenomenon (e.g. page 36). If they really believed their theory, Simkovic and McIntyre would write that “judgment” and “decision making” don’t matter at all because legal education alone boosts law graduates’ incomes equally. The authors themselves cannot escape the implications stemming from most lawyers having gone to law school and many law school graduates working as lawyers and other professionals.

I should add that although “Economic Value” states in footnote 10 that “exploratory results” show that the non-lawyer law graduates in their data still earn more than similar college graduates, this is probably yet another composition fallacy that lumps together the earnings of 55-year-old Fortune 500 CEOs with 25-year-old bartenders who don’t have much workplace autonomy and whose law degrees are effectively worthless. I hope the next time researchers use the SIPP data, they tell us who the people in the datasets are instead of water-boarding them with econometric tools.

[Note: I expanded this argument in an article on The American Lawyer, “‘Human Capital’ Theory Doesn’t Explain Law Grad Earnings,” June 19, 2015.]

Criticism 3:

“[Leichter] claims that we ignore distributional data when we report it.”

False: I claimed Simkovic and McIntyre omitted dispersal of incomes, not “distributional data,” which the authors describe as the “relative advantage” law school graduates have over similar college graduates (page 32). The relative advantage means that at any given percentile, law school graduates will earn more than similar college graduates (unless they both earned nothing). This is no surprise given that lawyers earn more than non-lawyers and lawyers rarely lack law degrees, biasing law grads’ incomes upward due to reasons other than education. The authors’ methodology mathematically trivializes law graduate underemployment by design, which is why I wrote, “Their argument boils down to a circular claim: Law school pays off because it’s impossible for law school to not pay off.”

If the authors included dispersal of incomes (and occupation information), we would have been able to see why, for example, the mid-20s law school graduates in the bottom of figure 4 had such paltry earnings.

How often do law school graduates go from earning less than $20,000 annually to nearly $80,000 a few years later? This is why absolute incomes matter and why we need to know their dispersal.

To Be Continued…

---

[i] A subtlety in Card’s chart: Male professional school graduates aged 40-45 in 1990 had fewer than 18 years of education on average yet showed a significant earnings boost above the trend nonetheless. Nowadays, professional school graduates probably have 19 or more years of formal education. This alone should raise suspicions that the sheepskin effect is boosting professional school graduates’ earnings, not education. Corollary: Shortening formal professional education requirements will not substantially impair the delivery of professional services.

Also, Card’s chart significantly understates the sheepskin effect’s impact on bachelor’s degrees. For example, one study found a 27.8 percent bonus to earnings over just the additional year of schooling, and even a negative effect on the 17th year, which would usually coincide with the first year of law school.

[ii] Additionally, they can test the earnings of law school dropouts—say, anyone who’s completed at least a year—against those of demographically similar college graduates on the hypothesis that if legal education generates a premium, then the dropouts earnings ought to be noticeably higher than the college graduates’.