Refuting the False Trope on Clinical Courses and Bar Passage

Posted on January 8, 2020 by rkuehnblog

Robert Kuehn, Washington University School of Law

It has been observed that “the fewer the facts, the stronger the opinion.” Until recently, this could be said about the possible influence of enrollment in clinical courses on a student’s likelihood of passing the bar examination. While there was a shortage of empirical studies on any possible relationship, there have been plenty of opinions on how taking those courses might be harmful, opinions often reflected in graduation restrictions on clinical courses and requirements for bar subject-matter courses.

But, there are now significantly more facts to refute those opinions. Two recent, large-scale studies have both found no relationship between the number of law clinic or externship courses or credits a law graduate took and her likelihood of passing the bar exam.

In a forthcoming article in the Journal of Legal Education, academic records of ten years of law school graduates of Washington University in St. Louis and Wayne State University were reviewed for any relationship between the number of law clinic, externship, or, more generally, experiential courses or credits and bar passage. After first accounting for the possible influence of law school grades on bar passage (the most significant predictor of bar success), the study found no correlation at either school between law clinic or externship enrollment and bar passage — no relationship between participation in a law clinic or externship and passage, none between the number of clinical courses and passage, none between the number of clinical credits and passage, and no evidence that students graduating with lower GPAs were disproportionately enrolling in those courses as a way to avoid doctrinal courses (another not uncommon trope). This lack of any relationship was in spite of increased enrollment in experiential courses at both schools over the ten-year period and decreased enrollment in courses teaching material tested on the bar (referred to as bar subject-matter courses).

The article notes that nationwide data on experiential course enrollment and bar passage also belie any claim the two are related. That data indicate that as enrollment in experiential courses was increasing from 2006-2016, bar passage percentages were fairly steady and that the recent decline in passage coincided with decreased, not increased, enrollment in those courses.

A recent study commissioned by the California State Bar found a similar lack of relationship between law clinic and externship courses and bar exam performance. The study reviewed law school coursework and performance on three July exams for over 7,500 bar applicants from eleven California schools. It found no relationship between the number of academic credits from law clinic courses and exam performance, either across all schools or even when reviewing schools separately. Similarly, there was no relationship between the number of externship or internship credits and performance, again when examined across all schools or within schools. The broad range of entering credentials at the eleven schools, and lack of a relationship even within those schools, indicates that the results should be applicable to most law schools, including those with lower LSATs and undergraduate GPAs for entering students.

The study results from Washington University/Wayne State and the California State Bar are similar to smaller studies at Texas Tech and the University of Denver that also reported no statistically significant relationship between enrollment in a law clinic or externship course and bar passage.

The Washington University/Wayne State and California State Bar studies further revealed that opinions about the value of bar subject-matter courses should be moderated. There were small correlations at both schools between the number of bar subject courses and bar passage. But this result (explaining less than 5% of the variability in bar outcomes) was only for low performing students and additional courses showed no marginal benefit once students took the school’s average number of bar courses.

The California State Bar study focused on whether taking a specific course was related to performance on the bar exam topic taught in those courses. It found that neither attendance nor performance in courses covering any of the 13 bar-related topics was related to performance on the corresponding California bar exam or Multistate Bar Exam content covering that subject.

Studies at other schools also indicate that enrollment in bar subject-related courses do not support broad claims about the benefit of taking those courses.

It is time to put away the misinformed trope of participation in law clinic and externship courses harming a student’s chances of passing the bar exam and let the facts do the talking. Law schools should recognize and students should be told they can obtain valuable preparation for the practice of law by enrolling in clinical courses without affecting their likelihood of passing the bar exam.

Filed under: Uncategorized | Tagged: , , , , , , , , , , , , , | Comments Off on Refuting the False Trope on Clinical Courses and Bar Passage

An Overview of “A Study of the Relationship Between Law School Coursework and Bar Exam Outcomes”

Posted on September 26, 2019 by Jessica N. Haller

Robert Kuehn, Professor of Law at Washington University School of Law in St. Louis, and David Moss, Associate Clinical Professor at Wayne State University Law School, recently conducted a large-scale study looking at the relationship between clinical/experiential or bar subject-matter courses and bar passage success in a paper entitled, “A Study of the Relationship Between Law School Coursework and Bar Exam Outcomes.” As a law student currently enrolled in a clinic, this study immediately piqued my interest.

This study was in response to fear that bar passage rates were down because of rising enrollment in “experiential courses” as opposed to “bar-subject courses.” Law schools began pushing students to enroll in more of these bar-subject courses to correct this so-called issue. However, Professor Kuehn and Professor Moss observed that there was no evidence to suggest that taking more bar-subject courses was appropriate advice for all students. Their study looked at this missing evidence between bar-subject courses and experiential learning and bar exam outcomes for ten years between two law schools: Washington University School of Law (WashU) and Wayne State University Law School (Wayne State). Both schools only require the designed first year courses and the upper-level writing courses mandated by ABA accreditation standards.

Previous studies performed in states like Texas, Colorado, and California looked at the effects of coursework and bar passage rates. These studies did not support the claim that taking more bar-tested law school course improve chances of passing on the first attempt. Notably, a study done in Indiana concluded, “simply forcing lower-performing students to take more upper division bar-subject courses will not solve the bar examination failure problem.”

The first goal of the present study was to determine whether a graduate’s enrollment in elective experiential courses was related to first-time bar passage success. Next, it was to assess whether enrollment in elective courses that cover bar subjects was related to bar success.

Data was collected from law school graduates from 2006-2015. The following table outlines the number of graduates with LSAT scores and bar passage rates between the two schools:

The next table looked at graduate characteristics such as undergraduate GPA, LSAT score, 1L GPA, and law GPA and their correlation with bar passage:

It wasn’t until 2005 that the ABA began requiring graduates to receive professional skills instruction with as little as one credit satisfying the requirement. In 2014, the ABA changed this to require six credits beginning with 2019 graduates. The study authors decided to track enrollment in skills courses versus bar passage over this time period.

The table above reveals a solid line depicting that average bar passage percentages were steady from 2006-2013 (this is when experiential course enrollment increased by over 50%). During the significant rise in experiential enrollment, bar passage percentages were largely steady. “Therefore, efforts to link declining nationwide bar passage rates to the rise in experiential course enrollment are not supported by national statistics.” A more likely contributing cause for bar passage declines since 2014 is weaker credentials of incoming 1Ls.

At WashU, it was found that while taking at least the average number of bar courses is associated with increased likelihood of passing the bar, there was no statistically significant increase in bar passage associated with bottom-quartile LGPA graduates who took more than the school’s average. This was similar with graduates in the bottom half of their class at Wayne State. Results for both schools indicate that graduates in bottom quartile who take fewer than the average number of bar courses at their school were associated with a significant increase in bar failure. Further, at both schools, students entering with scores lower than 150 were associated with pass rates significantly below the school’s average.

This study concluded that the claim that the dramatic decline in bar passage rates is due to law students taking more experiential courses or fewer bar-related courses is not supported. It characterized efforts to cap experiential credits in order to improve bar passages rates are “misguided,” warning that schools should not expect that “mere exposure” to more bar courses will significantly improve bar passage rates.

Also see “Legal Skills Prof Blog” and “TaxProf Blog” for more posts on this study

Filed under: Uncategorized | Tagged: , , , , , , , , , , , | 2 Comments »

DO LAW SCHOOLS ADEQUATELY PREPARE STUDENTS FOR PRACTICE? SURVEYS SAY . . . NO! – Robert Kuehn, Washington University School of Law

Posted on January 3, 2017 by rkuehnblog

Under ABA Accreditation Standard 301, law schools have two educational objectives: prepare their students “for admission to the bar and for effective, ethical, and responsible participation as members of the legal profession.” There has been much concern lately over declining bar passage rates, focusing attention on whether some schools are admitting students who may not be capable of passing the bar exam and whether a school’s program of legal education adequately prepares its graduates for the exam.

In focusing on the bar exam, it’s important not to lose sight of legal education’s primary duty of ensuring that law school prepares students for entry into the legal profession and a successful career. If studies of practicing lawyers and recent law graduates matter, it is clear that law schools are failing, even worse than in preparation for bar admission, to adequately prepare their students for legal practice.

A 2012 study by the National Conference of Bar Examiners (NCBE) analyzed the job activities of newly-licensed lawyers to determine which knowledge domains and professional skills and abilities are most significant to their job. Acquisition of professional skills and abilities were deemed significantly more important to newly-licensed lawyers than legal knowledge — 25 skills and abilities were deemed more important than the highest rated knowledge domain. The percentages of lawyers using these 25 skills in their work (all rated between 89% to 100%) also were all greater than the percentage using the highest rated knowledge domain (86%). Yet these skills and abilities generally are not developed in traditional doctrinal law classes but in the experiential and first-year legal writing courses that, under the ABA standards, need only account for ten percent of a student’s legal education.

These important skills and abilities are also a small part of the bar exam, which purports to measure competence to begin the practice of law. Although the NCBE study was promoted as the basis for further development of the exam, since the study’s completion the portion of the exam devoted to testing skills remains the same (the 3-hour Multistate Performance Test). The NCBE’s only apparent response to the study’s dramatic finding that professional skills and abilities are what new lawyers need most for competent practice was to add civil procedure (the study’s highest rated knowledge domain) to the Multistate Bar Exam.

A report released this year by Educating Tomorrow’s Lawyers reinforces the disconnect between legal education’s overwhelming focus on legal knowledge and the competencies new lawyers need. A study of more than 24,000 lawyers in 50 states sought to determine the foundations entry-level lawyers need to launch successful careers in the legal profession. The study found “that characteristics (such as integrity and trustworthiness, conscientiousness, and comment sense), as well as professional competencies (such as listening attentively, speaking and writing, and arriving on time), were far more important in brand new lawyers than legal skills.” Yet, again, only in clinical and first-year legal writing courses are there efforts in the law school curriculum to address the “soft skills” so necessary for the success of new lawyers.

These two studies mirror the findings of decades of earlier studies. In a 1978 study, mid-career lawyers rated the importance of 21 types of legal knowledge and skills in their daily work and the role of their law school training in attaining that knowledge or skill. With the exception of knowledge of statutory law, none of the eight areas of legal knowledge was considered very important for their work. In contrast, six of the methodology and skills areas were deemed of great importance.  When then asked to rate their law schools’ role in developing skills, two-thirds said their school had been “not helpful” or “played no role” in their ability to develop essential practice skills like interviewing, counseling clients, and negotiating; more than 40% said law school failed to train them to draft legal documents or effectively communicate orally.

A later study of Montana lawyers came to similar conclusions. It asked what level of competence a lawyer should have to perform in a professionally competent manner and what level of competence they observed first-year lawyers to have. The results indicated the need for greater emphasis in law school on the development of professional skills and the importance of character traits to a new lawyer’s successful transition to practice. A 1993 study of Chicago and Missouri lawyers found large gaps between the skills lawyers deemed most important to their practice and the attention law school paid to those skills, especially in the areas of oral and written communication, drafting legal documents, problem solving, negotiation, fact finding, counseling, and litigation. The lawyers believed that a majority of these practice skills could be learned in law school, if the focus of legal education were changed. A similar study of Minnesota lawyers found most did not believe they were well prepared immediately following law school on nine of seventeen important practice skills. Like respondents in other studies, those lawyers believed these skills can be effectively taught in law school.

More recent studies have not reflected any improvement in the role of legal education in preparing graduates for practice. The American Bar Foundation’s After the JD study tracks the careers of a sample of lawyers who passed the bar in 2000. It asked lawyers three and seven years of out of school if “law school prepared me well for my legal career.” On this fundamental objective of legal education, law schools failed miserably — 40% of lawyers after three years of practice and 50% after seven years said that law school did not adequately prepare them. Both groups overwhelmingly agreed that law school was too theoretical and unconcerned with real life practice. In another study of early-career lawyers, only 28% believed that law school prepared them to practice law.

Two studies by LexisNexis reinforce this view. In a 2009 survey, 90% of attorneys in private practice and corporate law offices said that law school does not teach the practical skills needed to practice law today. A similar study six years later found that legal education has not improved, contrary to the claims of some legal educators and regulators. In the 2015 survey, 95% of hiring partners and senior associates who supervise new attorneys responded that recently graduated students lack key practical skills at the time of hiring. The lawyers also believe its not a matter that law schools cannot teach these skills but that they simply refuse to do so: “Most attorneys involved with hiring and management of new lawyers agree practical skills can be effectively honed through clinics, internships, clerkships, and experience in actual or simulated application to a case.”

Law students who just graduated realize their need for more practical training — 87% say legal education needs to undergo significant changes to better prepare future attorneys; 97% favor a law school model that incorporates clinical experience. Judges agree. When asked what change would most benefit law schools, judges of all types of court (federal and state, appellate and trial) rated more coursework on practice-oriented skills the highest, far exceeding support for expanding the core curriculum.

Fifty years ago the dean of the University of Chicago School of Law stated that the aim of law school “is not to train lawyers, but to educate men [and women] for becoming lawyers.” If the attitude of the ABA and law school deans has changed since then, it isn’t reflected in the readiness of law school graduates for practice when over 90% of lawyers give legal education a failing grade. So while educators worry about declining grades on the bar exam, isn’t it also time to fix legal education’s longstanding failure to meet its duty to adequately prepare it students “for effective, ethical, and responsible participation as members of the legal profession”?

Filed under: Best Practices, Best Practices & Curriculum, Teaching Methodology | Tagged: | 4 Comments »

%d bloggers like this: