Best Practices for Legal Education

Bar examination passage rates are down again in many states. Last year’s results led to accusations that exam administration caused the decline, and counteraccusations that schools were at fault for admitting less qualified students than in prior years. Determining the possible cause of this year’s decline is complicated by the addition of a new subject (Civil Procedure) to the Multistate Bar Exam. In response to the declines, some blame an easy scapegoat — too many electives (especially experiential courses) and too few bar-tested courses. While limiting experiential or clinical courses or credits or mandating more bar courses presents an easy way of appearing  to do something,  there is no available evidence that students who take more experiential or clinical courses do worse on  the bar exam, and only a limited, weak positive correlation between bar courses and bar exam success.

Fueling this finger pointing against experiential courses was a comment from the president of the National Conference of Bar Examiner s (NCBE) on factors that could explain the decline in bar passage percentages: “In addition, the rise of experiential learning may have crowded out time for students to take additional ‘black-letter’ courses that would have strengthened their knowledge of the law and their synthesis of what they learned during the first year.”¹ She suggested another factor could be that schools are requiring fewer bar courses, “thereby permitting students to miss (or avoid) core subjects that will appear on the bar exam.” A possible connection between clinical courses and declining bar scores was also later raised by the NCBE’s director of testing and research.²

Unfortunately for the debate over the causes of bar exam failure and what schools might do to address the problem, these statements were made without reference to any supporting evidence. Indeed, none exists. In response to my inquiry whether there was any empirical basis for asserting that students with more experiential coursework perform, on average, worse on the bar exam or that taking more bar courses will increase  a student’s chances of success, the NCBE president replied that she was unaware of any research but would check with her testing staff. A follow up six months later confirmed there still was no supporting study to share.

I too am unaware of any published study examining the relationship between experiential or clinical coursework and bar passage. There are a number of studies showing the value of clinical courses in enhancing the practice skills and professional identity of students. But no data on the relationship of coursework to bar success include results for experiential courses. Studies do consistently find that law school grades and LSAT scores have the strongest relationship to bar exam success.³

Regarding a relationship between enrollment in bar courses and bar passage, published studies show no, or a small, positive relationship, but only for a narrow range of students. The earliest study sought to determine whether an Indiana bar admission rule mandating successful completion of courses in 14 subject matter areas was likely to increase the probability of passing the bar examination. Reviewing data from three administrations of the exam, the authors found “[n]o course or group of courses had any consistent relationship to success or failure on the bar exam.”⁴ They concluded:

The lack of consistent positive and significant relationships between taking or not taking bar- related courses and bar examination pass rates suggests that requiring these courses will not increase the likelihood that law school graduates, at risk of failure, will pass rather than fail the exam.

The most recent study reviewed coursework and bar results for students at St. Louis  University (SLU) and Hofstra University. Five years of bar results for SLU students showed:

No statistically significant relationship between the number of upper division, bar examination subject matter courses taken and bar examination passage for graduates who ranked in the first, second, or fourth quartiles or for graduates who ranked in the bottom 10 percent of their graduating class.⁵

There was a statistically significant relationship for students in the third quartile but only 4.1% of the difference in bar passage rates for that group could be explained by the number of bar examination su bject-matter courses taken while in law school; 95.9% of the difference in this quartile between those who passed and those who did not was due to other factors. A follow-up study with data provided by Hofstra similarly found no relationship between the number of bar subject-matter courses taken and bar passage for graduates who ranked in the first, second, or fourth quartiles and only a weak relationship for students in the third quartile.⁶The most likely explanation for these results is that students  in the top half of the class already pass the bar at sufficiently high rates that additional bar courses help very little, if any, and that students in the bottom quartile are so lacking in analytic skills that merely putting them in the audience of yet another lecture class fails to address the cause of, or provide a solution to, their problems.

So, while some authors claim, without empirical support, that bar courses will improve a student odds of passing the exam,⁷ and while schools continue to advise their students that the key to bar success is enrollment in bar courses, published studies do not support those claims.

But, all is not lost. Statistical analysis of bar performance at a number of schools has found that specially designed academic support and bar passage programs can improve passage rates, especially for students who have not performed well in law school.⁸ As one statistician told me, “accurately identifying [at]-risk [students] and then ensuring access to targeted, effective programs does make a difference.”

Of course these programs require a much greater commitment of a school’s resource than simply piling more bar courses on at-risk students. Yet, irresponsibly scapegoating experiential courses for bar failure or forcing students to take more upper-class bar courses as a purported solution is, as the authors of the most respected study warned, “overly simplistic” and “will not solve the bar examination failure problem.”⁹

Notes

^1. Erica Moser, President’s Page, THE BAR EXAMINER 4, 6 (Dec. 2014).
↩

^2.Mark A. Albanese, The July 201, MBE: Rogue Wave or Storm Surge, THE BAR EXAMINER 35, 46 (June 2015).↩

^3. See, e.g., Michael T. Kane, et al., Pass Rates and Persistence on the New York Bar Examination Including Breakdowns for Racial/Ethnic Groups, THE BAR EXAM- INER 6, 15 & n. 7 (Nov. 2007); Keith A. Kaufman, et al., Passing the Bar Exam: Psychological, Educational, and Demographic Predictors of Success, 57 J. LEGAL EDUC. 205, 214 (2007).
↩

^4. 4 Philips Cutright, et al., Course Selection, Student Characteristics and Bar Examination Performance: The Indiana University Law School Experience, 27 J. LEGAL EDUC. 127, 136 (1975).↩

^5. Douglas K. Rush & Hisako Matsuo, Does Law School Curriculum Affect Bar Examination Passage? An Empirical Analysis of Factors Related to Bar Examination Passage Between 2001 and 2006 at a Midwestern Law School, 57 J. LEGAL EDUC. 224, 233 (2007).↩

^6. Id. at 224.↩

^7. See, e.g., “Students should take most bar-tested courses during law school . . . . Although there is a lack of empirical evidence that taking bar courses correlates with bar success.” Denise Riebe, A Bar Review for Law Schools: Getting Students on Board to Pass Their Bar Exams, 45 BRANDEIS L. REV. 269, 308 (2006-07). Ironically, the footnote associated with the recommendation that students take most bar-tested courses contradictorily states: “Although law school professionals routinely advise students to take bar-tested courses, there does not appear to be any statistically verifiable support for the practice.” ↩

^8. See, e.g., Scott Johns, Empirical Reflections: A Statistical Examination of Bar Exam Program Interventions, 54 U. LOUIS- VILLE L. REV. (forthcoming 2016) (University of Denver); Mario W. Mainero, We Should Not Rely on Commercial Bar Reviews to Do Our Job: Why Labor- Intensive Comprehensive Bar Examination Preparation Can and Should Be a Part of the La w School Mission , at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2546001; Derek Alphran, et al., Yes We Can Pass the Bar. University of the District of Columbia, David A Clarke School of Law Bar Passage Initiatives and Bar Pass Rates – From the Titanic to the Queen Mary, 14 U.D.C. L. REV. 9 (2011)(U.D.C.); Linda Jellum & Emmeline Paulette Reeves, Cool Data on a Hot Issue: Empirical Evidence that a Law School Bar Support Program Enhances Bar Performance, 5 NEVADA L. REV. 646 (2005) (University of Richmond). ↩

^9. Rush & Matsuo, supra at 236. One of the authors of the study explained, “if you want students to pass bar exams, teach more logic, reasoning and test taking skills.” Doug Rush, Comment to Did You Know That “Bar Courses” Don’t Matter?, Best Practices for Legal Educ., July 25, 2008, http://bestpracticeslegaled.albanylawblogs.org/2008/07/25/335/#comment-289.↩

Filed under: Uncategorized |