I apologize for being last to the potluck for AccessLex Institute’s study, “Examining Value, Measuring Engagement,” which surveys law-school graduates to investigate their long-term outcomes. Please enjoy my store-bought pasta salad that I’ll abandon with my hosts.
I always take seriously any sincere exploration of the long-term value of law degrees. (Okay, the same goes for insincere ones, just for different reasons.) But how seriously one should take “Examining Value” depends on what one thinks of opinion surveys. For answering value-of-a-law-degree questions I see them as an inferior form of data. Valid, but not the best approach. Opinion data are essentially aggregates of respondents’ moods, making them subjective, contradictory, and volatile.
For example, “Examining Value” sometimes divides its respondents between pre-recession and post-recession graduates, and as expected, post-recession graduates report longer job searches and dimmer opinions on the value of their law degrees. Graduates with more law-school debt tend to believe they would not go to law school if they were Groundhog Daying their lives.
Law-school graduates do stray from my expectations in a few places. One is their willingness to recommend law degrees to people like them, e.g. 53 percent of post-recession graduates. Given how much people without law degrees discourage others from going to law school, the finding is surprising up to the point that one considers the enormous power of selection bias and choice-supportive bias. I’m also surprised that so few J.D.s believe their law degrees were not worth the cost (only 4 percent (of all law grads)), an empirical question that can be tested against graduates’ actual circumstances. However—and this is an important shortcoming of “Examining Value”—the study frequently declines to post percentages of its survey results by graduation year, dropping only side comments like, “Students who graduated during or after the Great Recession are less likely than earlier graduates to strongly agree that their degree was worth the cost, even when controlling for student debt.”
Thanks for the detailed insight, AccessLex Institute.
But the shortcomings only start there. One question AccessLex Institute didn’t bother asking was, “How much money do you make?” For all the study’s focus on student-loan debt, you’d think that it would take graduates’ incomes into account as well. Then, of course, there’s its uninterest in defining what a “good job” is. Sure it took 26 percent of post-recession grads more than a year to find one, but we still don’t know what they are or if they have anything to do with the skills and knowledge they obtained in law school that they couldn’t’ve gotten from reading a book.
Finally, the title to this post promised you some strange arithmetic, and here it is. Figure 14 asks the relevant respondents to choose one of a number of reasons they no longer practice law:
Rounding errors and omitting a few percentage points of unknown responses are okay, but one-quarter of non-practicing grads didn’t list a reason. That’s pretty significant, and it would be interesting to see how that corresponds with their debts and incomes, which, again, AccessLex Institute didn’t ask for. These non-practitioners in Figure 14 account for 37 percent of J.D. participants, an alarmingly high proportion that calls the survey’s primary findings into question. How can respondents say their law degrees have value if so many of them aren’t using them at work?
The best answer, further down, is “analytical skills,” but naturally the survey didn’t ask any respondents, much less experts, if they could obtain those by alternative means.
“Examining Value” can be found via the ABA Journal‘s article on it.
If the purpose of AccessLex Institute’s study was to find the current perceived value of a law degree, it’s done a mediocre job. Perceptions don’t say a lot that we don’t know, and even so they’re often contradictory and prey to cognitive biases. It’s only when researchers try to dig into the causes of those contradictions that these types of studies provide genuine insights.
The Last Gen X American 