An Overview of “A Study of the Relationship Between Law School Coursework and Bar Exam Outcomes”

Robert Kuehn, Professor of Law at Washington University School of Law in St. Louis, and David Moss, Associate Clinical Professor at Wayne State University Law School, recently conducted a large-scale study looking at the relationship between clinical/experiential or bar subject-matter courses and bar passage success in a paper entitled, “A Study of the Relationship Between Law School Coursework and Bar Exam Outcomes.” As a law student currently enrolled in a clinic, this study immediately piqued my interest.

This study was in response to fear that bar passage rates were down because of rising enrollment in “experiential courses” as opposed to “bar-subject courses.” Law schools began pushing students to enroll in more of these bar-subject courses to correct this so-called issue. However, Professor Kuehn and Professor Moss observed that there was no evidence to suggest that taking more bar-subject courses was appropriate advice for all students. Their study looked at this missing evidence between bar-subject courses and experiential learning and bar exam outcomes for ten years between two law schools: Washington University School of Law (WashU) and Wayne State University Law School (Wayne State). Both schools only require the designed first year courses and the upper-level writing courses mandated by ABA accreditation standards.

Previous studies performed in states like Texas, Colorado, and California looked at the effects of coursework and bar passage rates. These studies did not support the claim that taking more bar-tested law school course improve chances of passing on the first attempt. Notably, a study done in Indiana concluded, “simply forcing lower-performing students to take more upper division bar-subject courses will not solve the bar examination failure problem.”

The first goal of the present study was to determine whether a graduate’s enrollment in elective experiential courses was related to first-time bar passage success. Next, it was to assess whether enrollment in elective courses that cover bar subjects was related to bar success.

Data was collected from law school graduates from 2006-2015. The following table outlines the number of graduates with LSAT scores and bar passage rates between the two schools:

The next table looked at graduate characteristics such as undergraduate GPA, LSAT score, 1L GPA, and law GPA and their correlation with bar passage:

It wasn’t until 2005 that the ABA began requiring graduates to receive professional skills instruction with as little as one credit satisfying the requirement. In 2014, the ABA changed this to require six credits beginning with 2019 graduates. The study authors decided to track enrollment in skills courses versus bar passage over this time period.

The table above reveals a solid line depicting that average bar passage percentages were steady from 2006-2013 (this is when experiential course enrollment increased by over 50%). During the significant rise in experiential enrollment, bar passage percentages were largely steady. “Therefore, efforts to link declining nationwide bar passage rates to the rise in experiential course enrollment are not supported by national statistics.” A more likely contributing cause for bar passage declines since 2014 is weaker credentials of incoming 1Ls.

At WashU, it was found that while taking at least the average number of bar courses is associated with increased likelihood of passing the bar, there was no statistically significant increase in bar passage associated with bottom-quartile LGPA graduates who took more than the school’s average. This was similar with graduates in the bottom half of their class at Wayne State. Results for both schools indicate that graduates in bottom quartile who take fewer than the average number of bar courses at their school were associated with a significant increase in bar failure. Further, at both schools, students entering with scores lower than 150 were associated with pass rates significantly below the school’s average.

This study concluded that the claim that the dramatic decline in bar passage rates is due to law students taking more experiential courses or fewer bar-related courses is not supported. It characterized efforts to cap experiential credits in order to improve bar passages rates are “misguided,” warning that schools should not expect that “mere exposure” to more bar courses will significantly improve bar passage rates.

Also see “Legal Skills Prof Blog” and “TaxProf Blog” for more posts on this study

Fostering Student Success: Part II -Possible Actionable Steps to Encourage Growth Mindsets

The opinions expressed within this article are the personal opinions of the author.

By Sara J. Berman, Director of Academic and Bar Success Programs at the AccessLex Institute’s Center for Legal Education Excellence; SSRN author page https://ssrn.com/author=2846291

As was detailed in Part I of this post on Fostering Student Success, we must meaningfully reward those who do the hard work and actually end up achieving the requisite skills and doctrine mastery at some point (any point!) before graduation. Those who take a bit longer to catch on must be given true opportunities to improve so that they see struggling to learn as evidence of powerful grit and a stepping stone to a lifetime as a successful professional, rather than a predictor of future failure.  Below are a number of possible actionable steps we might consider piloting and studying.

First, we might encourage growth mindsets by listing grades as AGP (annual grade points) rather than cumulative GPA (grade point average). Every year would provide a new, level playing field for students, and, employers would readily see whose grades increased, and by how much each year. (Note: Scholarship comprehensively critiquing grading and class ranking systems dates back some time. [1] The suggestions here simply point to “low-hanging fruit” interventions.) A natural criticism of this approach is that first-year courses tend to be required and are thus an apples-to-apples comparison, while upper-division courses vary widely and often have looser grading policies. Too many 2L students who see Cs turn to Bs falsely attribute this “improvement” to their own effort when grade increases actually stem from “easier” courses and/or more lenient grading.  Nonetheless, there could be a great psychological benefit to having a “clean slate” each year, with new opportunities in 2L and 3L to be at the top of the class. Prestigious and financially generous awards could be given to students whose GPAs have increased the most from the first year to the third year. And, employers could still see grades in particular courses and full transcripts as desired.

Second, we could study the effect of eliminating class ranking altogether. Justified, as is GPA, by the “needs” of employers, class ranking also fosters a fixed mindset, competitively boxing students into “winners” (those at the top of the class) and “losers” –those at the bottom who  may internalize defeat and, far too often, treat low ranking as a predictor of bar exam failure (which in turn may become a self-fulfilling prophecy).[2] Are class rankings necessary? What pedagogical purpose do they serve? Some medical schools are moving to a pass/fail model[3] with less emphasis on relative rank.[4] This appears to be reducing some of the stress associated with mental health challenges in these similarly high-pressured graduate programs[5] without affecting academic performance or accomplishment.[6] Some (mostly elite) law schools do not rank students. Should others experiment as well?  The main advantage appears to be providing a triage system for potential employers, (e.g. “We only hire from the top 25% of the class.”). Yet recent studies[7] show that what many legal employers want in new lawyers includes so-called “soft skills,” not measured by grades or class rankings.[8]  If this is the case, might we better serve employers’ needs by creating rubrics to measure professionalism and practical lawyering skills?  Highlighting how much a student’s grades have improved from 1L to graduation could help employers measure resiliency, while actually encouraging improvement by stemming some of the “why bother” mentality of those who turn off after receiving low 1L grades.  

Third, let us endorse studies that pilot tests of non-cognitive skills, such as those LSAC is undertaking and those inspired by the Shultz and Zedeck studies.[9] And let us support and laud efforts to showcase (in part for potential employers) the wide range of student skills on display in lawyering competitions.[10]

Fourth, let us identify and study other creative ways to assist employers while breaking vicious, defeatism cycles that thrive in our current system. I have long encouraged graduating classes with the aspirational challenge of 100% bar passage, reminding them that while class ranking forced some to the top and others to the bottom, every graduate can pass the bar exam first time around.  (Recall the old joke: “Question: What do you call the person who was last in his class in medical school? Answer: Doctor!”).  I also urge law graduates to help each other –with a “rising tide lifts all boats” philosophy and with the learning science-backed truth that teaching another is often the best way to learn.

Fifth, we might pilot the administration of comprehensive exams at the end of each year of law school.  These would encourage students to review and be re-tested on key subjects, “building mental muscle” over time so that they learn to master materials they may only have understood superficially when first exposed.  Awards could be given to every student who achieved high scores on these “comps,” rewarding those who caught on later as well as those who caught on initially.

Sixth, we could develop a national pre-bar exam (what I call the “NPBE”), similar to the PSAT, which would allow 2L law students a high-stakes “practice exam” which schools could use as a diagnostic and formative assessment so that law graduates do not have to fail the bar exam in order to realize how much improvement they really need to pass, in skills, substance, time management, mindset, and more.[11] Like the PSAT with its National Merit Scholar incentives, the NPBE could award scholarships to those with low 1L grades who overcome challenges and perform exceptionally well on the NPBE.

Perfect pass rates are not impossible on the law school side (though I understand limitations that may result from certain jurisdictions’ cut scores), especially when considering cumulative rather than first-time bar passage, per the new ABA Standard 316.[12]  But widespread student success requires more than mouthing “grit” and “persistence” mantras.  We must actively foster institution-wide expression of and action supporting the belief that every student who is not academically dismissed can pass the bar exam.  We must equip all students who graduate from ABA law schools to pass the bar first time around.  And, if we truly hope to so equip our law students, their self-perceptions simply may not be allowed to become fixed after first semester grades. 


[1] Barbara Glesner Fines, Competition and the Curve, 65 UMKC L. Rev 879 (1997); Jay M. Feinman, Law School Grading, 65 UMKC L. Rev. 647, 656 (1997); Jerry R. Foxhoven, Beyond Grading: Assessing Student Readiness to Practice Law, 16 Clinical L. Rev. 335 (2009); Heather D. Baum, Inward Bound: An Exploration of Character Development in Law School, 39 UALR L. Rev. 25 (2016).

[2] Query whether research presented at AALS (January 2018) by Professor Robert R. Kuehn (Washington University in St. Louis) suggests this, given results of students with identical entering LSAT scores failing the bar where they were at the bottom of the class and passing where they were at the top of the class.

[3] Casey B. White and Joseph C. Fantone, Pass–fail Grading: Laying the Foundation for Self-Regulated Learning, 15 Advances in Health Sci. Educ. 469 (2010).

[4] John P. Bent et al., Otolaryngology Resident Selection: Do Rank Lists Matter? 144 Otolaryngology-Head & Neck Surgery 537 (2011).

[5] Daniel E. Rohe et al., The Benefits of Pass-Fail Grading on Stress, Mood, and Group Cohesion in Medical Students, 81 Mayo Clinic Proc. 1443 (2006); see also Robert A. Bloodgood et al., A Change to Pass/Fail Grading in the First Two Years at One Medical School Results in Improved Psychological Well-Being, 84 Acad. Med. 655 (2009); Francis Deng and Austin Wesevich, Pass-fail is here to stay in medical schools. And that’s a good thing, KevinMD.com (Aug. 3, 2016).

[6] B. Ange et al., Differences in Medical Students’ Academic Performance between a Pass/Fail and Tiered Grading System, 111 S. Med. J. 683 (2018).

[7] Alli Gerkman & Logan Cornett, Foundations for Practice: The Whole Lawyer and the Character Quotient, AccessLex Inst. Res. Paper Series No. 16-04 (2016).

[8] Bryant G. Garth, Notes on the Future of the Legal Profession in the United States: The Key Roles of Corporate Law Firms and Urban Law Schools, 65 Buff L. Rev. 287 (2017).

[9] Marjorie M. Shultz & Sheldon Zedeck, Predicting Lawyer Effectiveness: A New Assessment for Use in Law School Admission Decisions, CELS 2009 4th Ann. Conf. on Empirical Legal Stud. Paper (2009).

[10] Sherry Y. English, Cincinnati Law hosts nation’s first, only law student case competition, UC News (Jan. 10, 2019),https://www.uc.edu/news/articles/2019/01/n2059715.html.

[11]As I often say, would anyone mount a Broadway show without a dress rehearsal? Do athletes compete in the Olympics without high-profile pre-competition practice?  No!  Yet we wait until after law school and generally outsource to bar reviews the only sort of organized practice runs for the highest stakes law exam of all.

[12] Two Indiana law schools soar on ultimate bar passage rate, Ind. Law. (April 22, 2019),https://www.theindianalawyer.com/articles/50047-two-indiana-law-schools-soar-on-ultimate-bar-passage-rate.

%d bloggers like this: