After All These Years: Another Bar Exam Over, Another Entering Class, but Still a Disconnect between the Licensing Exam and What We Need Lawyers to Be and Do

I was never a Journey fan but I truly am astonished that after all these years of preparing lawyers for practice, and after two years of an unprecedented undermining of  the rule of law in our nation, law schools still live with a disconnect between the profession’s  licensing exam and what business, government and society needs lawyers to be and do, which includes protecting  the rule of law. 

The National Law Journal recently discussed two new major studies which will analyze whether the current exam is the best measure of new lawyer competence.  The National Conference of Bar Examiners (NCBE) is in the midst of a three year study  to “ensure that the bar examination continues to test the knowledge, skills, and abilities required for competent entry-level legal practice in the 21st century.”  (Hmm, continues? that’s a bit biased) and has already held 30 listening sessions.  

The second study, “Building a Better Bar: Capturing Minimum Competence” is an initiative of  the Institute for the Advancement of the American Legal System in partnership with Ohio State Law Professor Deborah Merritt, and aspires to develop a “fair, evidence-based definition of minimum competence” to improve the current licensing process.  Funded by Access-Lex, the researchers:

will be holding 60 focus groups in 12 locations around the country. While these focus group participants will primarily be new lawyers, we will also hold a number of specialized groups with supervisors. Additional specialized groups will include only women and only people of color, as well as groups in rural areas; traditional job analyses can mask the views of these lawyers, yet their perspectives are essential to create a more fully representative view of minimum competence and how to test for it effectively. Through these focus groups, we will be able to capture key information from a diversity of perspectives and provide concrete data on the definition of minimum competence that the profession can use to improve the bar exam and how lawyers are licensed.

 

Readers may remember that IAALS has provided helpful research in the past through its Foundations for Practice  research, which identified the  competencies over 24,000 legal employers value in new hires (most of which go untested by the current licensing process) as well as the evaluation of the graduates of the Daniel Websters Honors alternative to the bar exam in “Ahead of the Curve:  turning Law Students into Lawyers

I suppose I should be delighted that more studies are being launched. They are addressing the exact issues so many of us have raised for decades. However, my reaction is uncharacteristically pessimistic.  (Readers here who have tolerated my enthusiastic use of exclamation points and emphasis will agree it is uncharacteristic).  Perhaps it is the August humidity. Perhaps, it is the sorrow surrounding our nation after a week of grief from senseless gun violence But more likely, it is the fact that I am feeling frustrated that we have already studied this to death! For example, working with state bar associations The Foundations for Practice Project already studied new lawyer competencies with 24,000 lawyers from all 50 states participating and found

… the foundations that entry-level lawyers need to launch successful careers in the legal profession.

In a first-of-its-kind survey, we asked, “What makes a new lawyer successful?” More than 24,000 lawyers from all 50 states answered.

What we learned is that new lawyers need more than IQ and EQ to be successful. They also need CQ: Character Quotient. In fact, 76% of characteristics (thinks like integrity, work ethic, common sense, and resilience) were identified by a majority of respondents as necessary right out of law school.

Beyond character, new lawyers are successful when they come to the job with a broad blend of legal skills, professional competencies, and characteristics that comprise what we call the “whole lawyer.”

So why is the NCBE, who clearly has a stake in the outcome, refusing to respond to the outcome of that 3 year old study but instead promising only to do its own study. JEESH! We tweak here and there, we add more pro bono or experiential requirements, but no one truly influential will admit that our insistence on anchoring the gateway to the profession to a timed, written exam instead of clinical excellence is the problem.

Starting as early as 2008, this blog has discussed the problems with the bar exam and its role as an unhelpful, anxiety producing, discriminatory, skewed, and unnecessarily speeded, gate-keeping device.  For a sporadic history of posts between then and now, in fairly chronological order, click on the links below.

Did You Know That “Bar Courses” Don’t Matter? 

New Article: No Excuses Left for Failing to Reform Legal Education

Working with State Bar Associations on Best Practices

Bar Passage and Best Practices for Legal Education

One BAR to rule them all?

The Daniel Webster Scholar Honors Program

NYSBA Task Force on the Future of the Legal Profession Report

New Requirements for Bar Exam Stress Clinical Education

Existential Crisis and Bar Exams: what is really cruelest?

The Bar Exam Inhibits Curricular Reform

NEW YORK STATE BAR ASSOCIATION VIGOROUSLY OPPOSES PROPOSAL TO BRING UBE TO NY THIS JULY

Preparing Students for the Multistate Bar Exam

Musings on the Bar Exam and Legal Education’s Attitude toward it

Bar Exam Musings, Part II: Skillfully Changing the Bar Exam Narrative

Experts in the Legal Field Question the Bar Exam…

What’s going on in California? “TFARR- recommended” 15 credits of competency training

New York Proposes “Experiential Learning Requirements” as Condition of Licensure: CLEA and NYS Bar Committee Respond

Examining the Bar

Keeping an experiential identity in bar passage reform

Whither Clinical Courses and Bar Passage – by Prof. Robert Kuehn

DO LAW SCHOOLS ADEQUATELY PREPARE STUDENTS FOR PRACTICE? SURVEYS SAY . . . NO! – Robert Kuehn, Washington University School of Law

Professor Merritt’s Blog post on attorney discipline and bar exam WORTH A READ!

Studying Better Ways to Test Bar Applicants for Minimum Competence: Another Reason to Care about the California Bar Exam (Besides the Cut Score Debate)

Scholarship on Bar Exam Alternatives Needed

ABA Commission on Future of the Profession & ABA Vote on Bar Passage Proposal

Drafting Exams With Test-Taking Speed in MindConcrete Suggestions for Bar Exam Reform

We have to talk about the bar exam

What can Law Schools Learn about Bar Passage from Medical Schools’ Approach to Studying Students Who Struggle with Licensing Exams?

More Resources Re Teaching, Learning, and Bar Passage

A Fresh Look at the Uniform Bar Examination

Letters raise concerns about changes to the bar pass accreditation standard

Time to Remedy the Ills Afflicting ABA Council’s Standard 316 Proposal

Are the Students Failing the Bar Exam Today Canaries in the Coal Mine warning us of a More General Need to Change Legal Education?

Shifting the Focus of Legal Education Back to Just That: Education

How Practice Tests Reduce Anxiety in Bar Preparation and the Exam

Quite a listing, huh? I suspect that the IAALS and Merritt project will provide us with extraordinarily helpful insights into measuring minimum competence. But political clout is also needed. Will this BLOG simply be adding more posts for years to come on the unfairness and inappropriateness of a slightly modified, unnecessarily stressful, timed, bar exam — a continued hazing tradition?  I hope the NCBE and other institutional influencers proves me wrong.

Leading Edge Conference: Facing and Forming Legal Education’s Future with Insights, Data and Inclusive Thinking

Last week, I was fortunate to attend the 6th annual Leading Edge Conference hosted by Wolters Kluwer (WK) in Riverwoods, Illinois. It was my first experience with this particular conference. Using an unconference format and with a balance of old-timers and new attendees, WK brought together approximately 30 “thought leaders” for two+ days of intense discussion. Participants included professors and deans from a wide variety of law schools, representatives from law related entities such as LSAC, NITA and IAALS, education or pro-bono related entrepreneurs, and digitalization pioneers.

In addition to the conference, WK hosts the Leading Edge Webinar Series and just announced its 2nd annual Leading Edge prize. Ten Thousand Dollars ($10,000) will be awarded to two winning teams “to help implement their visions of improving student outcomes or expanding educational opportunities for law students.” Proposals are due August 15th.

I left the conferences with many “take-aways,” that I am only beginning to fully digest, and with a better sense of the continuing challenges facing legal education and our profession. Bernard A. Burk, Jerome M. Organ and Emma B. Rasiel recently published in the Nevada Law Review “Competitive Coping Strategies in the American Legal Academy: An Empirical Study”. Their research examined the response of law schools “to the substantial fall off in both the number and the conventional qualifications of applicants to law school that began after 2010.”

The “Competitive Coping Strategies” research also explains why more law schools have not closed and emphasizes the “widened distance” between current students’ needs and current school resources. The study found that in the face of plunging applications to law school, “Reputationally stronger schools” generally chose to preserve their entering Class Profile. This meant “thousands of viable candidates remained available to other law schools, effectively preventing the closing of as many as twenty Reputationally Weaker schools.”

Second, the study points out the implications of shrinking Class Size and discounting Tuition to preserve entering class profile. “As a practical matter, then, law schools ‘invested’ in Profile rather than in expanding their faculties, facilities or their access to clinical and experiential education. We encourage discussion of the implications of this investment choice.”

Third, the study noted that “some Reputationally Weaker law schools perversely were able to maintain or raise their average Net Tuition” and “the students with the least promising prospects for obtaining or making any economically sustainable use of their law degrees are paying the highest prices to obtain them. These inequalities expanded significantly after 2010.”

Fourth, the study highlights the millions of dollars in forgone Tuition Revenue “unavailable to meet the needs of students who at many law schools are significantly less prepared” than their predecessors and suggests this widening gap underlies the declining Bar Exam pass rate.

We seem to have reached a plateau in declining admissions to law school. But that plateau is not a place for us to settle in and rest. There are too many hard questions about where we are now.

How do we address the inequalities which have expanded since 2010 in law schools? What is the value we provide to those with the “least promising prospects?” Is it immoral that those least likely to make “any economically sustainable use of their law degrees are paying the highest prices to obtain them?” or that they may be undertaking crippling debt to obtain a law degree?

On the other hand, if we narrow the pathway into law schools even further, rejecting any who come to law school less credentialed or less prepared, will we be rejecting the dreams and hopes of those who desire a professional pathway? Will we be rejecting many who will find an economically sustainable and good life for themselves? Will we be playing God with students from less advantaged backgrounds just because we don’t know who will make it and who won’t? Will we be eliminating first generation students in larger numbers? Will we be amplifying the lack of diversity in our profession?

And what about the role of law schools in the community at large at this moment in our nation’s fledgling history? Shouldn’t we continue to exist as community laboratories which encourage civil discussion, uphold the rule of law, critique unjust legal systems and decisions, work to sustain democratic institutions and constitutional checks and balances, and produce new ideas about the role of law and legal systems in society?

Finally, if we espouse the “public good” values of my last two paragraphs as arguments for the continued existence of the legal academy and law schools, then do we prioritize these values in our faculty hiring, our strategic plans, and our prioritization of resources?
So, I leave you as I left the conference, with more questions than answers, but with a firm sense that we must continue to ask these important questions.

(Note: the author had her lodging, food and flights paid. She was not paid to write or post anything about the conference. Besides, she is pretty opinionated and not easily swayed.)

More Resources Re Teaching, Learning, and Bar Passage

Thank you to Best Practices for Legal Education Blog for having me as a blogger this week.  I hope the examples I’ve provided about methods medical schools use to evaluate their curriculum, test the effect of new programs, and look for factors that affect success on licensing exams.  As I mentioned at the end of my last post, the most comprehensive source for research based information about bar passage programs as well as a source of funding for sources is AccessLex.  There is a growing literature of articles from schools which have implemented successful bar passage programs.  Here’s an article by Louis Schulze about his work at FIU.

You might also be interested in a series of articles from back in 2009-2010 when those at the front lines of legal education, first year faculty and legal writing and research faculty, began to see significant differences in performance between the students they were teaching and those in the past.  These articles provide information about how substantial changes to the k-college education system in the U.S.A. impacts law students’ transition to law school. This article by Rebecca Flanagan is a good overview.  Prof. Patricia Grande here.  A literature review of law learning strategies by Profs Jennifer M. Cooper and Regan A.R. Gurung.   One more by Profs Susan Stuart and Ruth Vance

Here are the proceedings of a 2014 Symposium entitled “Teaching the Academically Underprepared Law Student” and I invited readers to take advantage of the comments section of this blog to share other publications—including the many more recent ones.  My point here is historical, not bibliographical.  And here, as a quick reminder of one of the crucial skills the bar doesn’t test– research.  Caroline L. Osborne

Finally, something I’ve mentioned elsewhere, the new collaboration between LSAC and Khan Academy providing free, on-line, very high quality LSAT preparation may have something to offer law students.  The skills underlying LSAT performance, close reading and legal reasoning, are not immutable—students can get better at them after enrolling in law school and may find some time with these materials a helpful and interesting way to brush up on these skills.

 

 

What can Law Schools Learn about Bar Passage from Medical Schools’ Approach to Studying Students Who Struggle with Licensing Exams?

It’s not unusual for a provost or a colleague or a relative at Thanksgiving to ask a legal academic why law students have so much trouble passing the bar exam when the pass rates for medical students are usually in the high 90th percent.  The short answer to that question is that the two processes are completely different—and there’s no obvious trick, technique, or intervention that could convert our bar passage rates into their licensure passage rates.   For one thing, it’s the wrong question.  “Passing” the medical licensing exams is certainly important, but unlike the “all or nothing” process of passing the bar exam, the score achieved on Step 1 affects medical students’ entire career path.  But there is a lot to learn about the methods that medical schools use in studying the very few students who have trouble as well as how they evaluate the effect of changes to their curriculums on scores on the licensing exams.

Quick recap on professional licensing—future doctors take a series of three exams over the first six years of their undergraduate medical education and the start of their residency.  (more links in a post I wrote earlier this year here).  The exams are almost entirely national although the actual process of being licensed is conducted on a state by state basis.   Law students take a licensing exam in the state where they intend to practice upon graduation.  For purposes of this post, the closest analogy to the bar exam is the more academic Step One students take during their second year of medical school.  Like  our NCBE, the National Board of Medical Examiners which produces United States Medical Licensing Examination works with medical licensing boards and depends on their confidence.  It issues annual reports.

The focus of this post is on the methods that medical schools use to study the small number of their students who do have trouble passing the licensing the exams as well as the factors that can affect the scores students achieve.  I’ve tried to focus on articles outside of paywalls, and would certainly encourage you to conduct your own searches in the various data bases to which you have access.  There are several journals devoted directly to studying medical education—although these articles can pop up anywhere.

Medical educators use a wide range of research techniques to learn more about students who struggle with licensure exams.  Like us, medical schools would prefer students pass the first time and many articles like this one look for characteristics who fail the first time but eventually pass.  Others look for characteristics of students at risk for failure here and here  or even  what students think of the exam.    Another area for inquiry involves the role stress plays in the score students achieve.   In partnership with social scientists at our schools or in our communities, we too could be conducting studies to help us learn more about students who face difficulty passing the bar exam.  These studies can be part of graduate student work or may even be funded by groups like Access which is making money available to study bar passage.

 

The actual reason the medical school pass rates are so high, though, may not be all that helpful.

It’s not just because they are able to limit admission to students who have already demonstrated an ability to score very highly on the MCAT.  A test that is much more similar to step 1 than the bar exam is to the LSAT.  Indeed, medical schools have direct input in both the MCAT and the Licensing Exams—so when one changes, the other can too. And it’s not clear that anything in the curriculum makes a difference at all—the industry offering study aids and licensure prep courses dwarfs the bar prep and study aid market to a point where students often start studying for the licensing exams before the first day of medical school.

But if it is the curriculum, it’s important to remember the vast difference in time scale between medical and legal education.  We have students for three years post B.A. Medical schools in the U.S. plan their curriculum based on  8 plus years of increasingly specialized medical education.  They are therefore comfortable holding off on the direct teaching of practice skills for the first two years while they are aligning their curriculum with the content of the Step 1 exam.

Even Step 1, though, is far more focused on practice than on knowledge accumulation or deliberately confusing question formulations that characterize the bar exam. Step 2,  the second round of licensing exams prior to graduation medical school,  go past paper and pencil in that they actually test students’ ability to conduct exams and exercise medical judgement.  Another reason for the high pass rate is that most medical schools have stopped developing their own tests and instead use assessment instruments (shelf exams) provided by the same company that produces the exam.   Sure, there is grumbling and criticism about content & timing of the licensing exams, but medical schools work hard to make sure that their curriculums are aligned with the content of the exams.  Finally, medical education is extremely self-reflecting–they are constantly aware of the risks that come from confusing correlation and causation.  How do you know that a change in one part of the curriculum is the cause of a change in test scores?  You run Pearson correlations followed by stepwise linear regressions.  Seeing is not believing when comes to identifying factors that affect performance on licensure exams.   Look here, here, here, and here for studies evaluating curriculum changes.  They take nothing for granted—does attendance make a difference, does flipping classrooms really work? Does reducing the number of hours spend in the anatomy lab reduce USMLE scores?

Another standard practice in medical schools is curriculum mapping— an essential first step for any school that wants to understand what they are teaching—let alone make changes.   Like all maps, curriculum maps are DESCRIPTIVE, not PROSCRIPTIVE.  Here is   Harvard’s curriculum map, but you can find examples on the home page of just about every U.S. Medical School.This is a an article walking through how to map a curriculum.

So what’s helpful to us isn’t so much what medical schools are doing, but how they are evaluating themselves. 

In recap, neither I nor anyone else who has ever practiced law thinks it would be a good idea to emulate medical schools by fully aligning our curriculum with the bar exam so as to turn the three years of law school into one extended bar prep course.  Among other reasons, the material tested on the bar is quite static and doesn’t reflect the realities of today’s law practice.   It also wouldn’t make much sense for schools whose students take the bar exam in many different jurisdictions.   Also, the bar exam is just not equivalent to the three rounds of USMLE exams in actually testing both the knowledge and application of knowledge needed to be a successful lawyer.  If it was, we wouldn’t hear so many complaints about how students who have passed bar are never-the-less not “practice ready.”

Tomorrow—where can we get the help we need to find out this information, and who is going to pay for it?  Spoiler--Access Lex has a program.

We have to talk about the bar exam

Thank you very much to the team at Best Practices for Legal Education for inviting me to blog this week.  My particular thanks to Elizabeth Murad for administering the blog, Professor Mary Lynch, Kate Stoneman Chair in Law and Democracy & Director, Center for Excellence in Law and President & Dean Alicia Ouellette of Albany Law School for hosting this blog.  It is an honor to join such a distinguished group of scholars and teachers.

We knew it was going to be a bad bar year when on Sept 14, 2018 the NCBE announced that “the national average MBE Score for July 2018” had decreased “about 2.2. points from the July 2018 average.”  And, indeed, as states have announced the pass rates based on their own individual formula of MBE plus essays plus the MPT (multistate performance test) plus their own individualized questions, the results were bad.  A lot of our students failed the bar exam.  Pass rates were down in New York, in California, Georgia, Florida, in Texas, and lots of other places.  Yet at the same time, individual schools saw significant success in raising pass rates in the face of grim news all around them.  All of this makes for glib headlines and much handwringing, but in the context of a blog post on “Best Practices for Legal Education” it is more helpful to take a step back and assess the tools we, as legal educators, have available to us in addressing bar passage in our individual schools.  I do so from my Ph.D. studies in higher education as well as from my experience as a dean, associate dean, law professor, and medical school professor.

One of my main themes this week will be to argue for individualized problem solving.  If anyone comes to you with a product to solve all your bar passage problems, I hope after this week you will be able to ask some questions about the data on which they base their claims.    Because a productive discussion of bar exam passage really rests on two questions—1. Why aren’t the students at your law school passing the bar exam at the rate they “should” and 2. What should you do about it?

I am going to use this week to share with you some of the resources available to law schools, to individual faculty members, and even to law students who want to increase their chances of passing the bar the first time.  Along the way, I hope to address some of the unhelpful myths that have arisen and to endorse a very old idea borrowed from the then revolutionary 1960s era child rearing techniques of Dr. Benjamin Spock: These are your students—and you know more than you think do.  Trust your judgement.  Ask questions.  That doesn’t mean that you can do everything yourself—it’s fine to consult with experts, but in the end addressing bar exam passage issues is a school wide effort and everyone has relevant information to add and a valuable role to play.

To get started, it’s helpful to have an overview of the players.  As a matter of foundational Constitutional Law, each state retains the power to license and regulate professionals.  (more detail here).   As a result, every state and territory has its own process for setting criteria for bar passage.   Almost every state contracts with the National Conference of Bar Examiners which develops the annual exam, grades it, and spends a lot of time explaining itself.  If you have any interest in this topic, a free subscription to The Bar Examiner will quickly bring you up to speed.

Tomorrow–how a test from the 1950’s trips up today’s digital natives (or “Do we need a Tardis to match law school curriculum to the bar exam?”)

New York Proposes “Experiential Learning Requirements” as Condition of Licensure: CLEA and NYS Bar Committee Respond

Readers of this blog and followers of the NCBE’s expansion remember  that this past Spring New York became the 16th state  to  adopt the Uniform Bar Examination (UBE), changing  its longstanding bar admission requirements.  Many voices opposed adoption including the New York State Bar Association (NYSBA) (see Committee on Legal Education and Admission to the Bar (CLEAB) report 10-29-2014  and vote of House of Delegates), the Clinical Legal Education Association (CLEA) and the Society for American Law Teachers (SALT).  Despite these and other  opposition voices, the proposal was adopted with the new changes going into effect for the July 2016 bar examination.

During discussion of the adoption of the UBE, the Court was encouraged  to include clinical or experiential  requirements for licensing so that lawyers admitted to the New York Bar would be ahead of the curve — a position I firmly support.   On the opposite coast, California had been engaged in a multi-year process examining licensure and profession readiness which resulted in a proposal requiring 15 credits of experiential learning before admission.  In response to the movement to incorporate experiential learning in bar admission,  the New York State Court of Appeals formed a Task Force on Experiential Learning and Admission to the Bar.  Just last month, that Taskforce requested comments on its proposal that

New York adopt a new mechanism for ensuring that all applicants for admission to the bar possess the requisite skills and are familiar with the professional values for effective, ethical and responsible practice. In light of New York’s diverse applicant pool, and in an effort to accommodate the varying educational backgrounds of applicants, the Task Force suggests five separate paths by which applicants for admission can demonstrate that they have satisfied the skills competency requirement.

The New York Law Journal examined the proposal in an article found here.   In addition, the Honorable Judge Jenny Rivera, chair of the Taskforce attended a meeting of NYSBA’s Committee on Legal Education and Admission to the Bar (CLEAB) to explain the proposal and answer questions.

It is heartening that the Court is concerned about and wants to  require the development of essential lawyering skills and professional values acquisition. However, without more, Pathway 1 of the current proposal will not actually ensure  that applicants to the bar experience the kind of skill development and value formation that the Taskforce desires.  Pathway 1, referencing new ABA standards,  requires schools to confirm that they have published  their “plan for incorporating into their curriculum the skills and professional values that,  in the school’s judgment,  are required for its graduates’ basic competence and ethical participation in the legal profession.” It also requires law schools to certify  that law graduate applicants for admission “have sufficient competency in those skills and sufficient familiarity with those values” which are publicly available on the law school’s website.  Although Judge Rivera believes that the certification process described in Pathway 1 can have some real bite, as pointed out in comments submitted by the Clinical Legal Education Association (11.9. 15 CLEA SUBMISSION ON EXPERIENTIAL REQUIREMENT ), Pathway 1 simply mirrors the experiential training requirements already mandated by the American Bar Association.     

New York’s  law school deans, not unexpectedly,  submitted comments supporting the “flexibility” of Pathway 1.  The  CLEAB report to the Experiential Taskforce expressed concern that without additional content to Pathway 1 “little will be accomplished” by the proposal.   And as one member of the NYS bar committee  argued, “what law school is going to admit that one of its graduates did not acquire the skills or  values promised on its website?”

In my opinion, the most important concern is whether applicants to the bar have ever represented or interacted with a client, or operated as a lawyer, in a live setting under guided, experienced supervision before admission.  In its comment to the Taskforce, CLEA urges that a “three- credit clinical training requirement” be added for all J. D. applicants to the New York Bar.  This makes sense.  Law school clinics and faculty-supervised externships are designed to create the very kind of skill development and value acquisition with which the Court is concerned.  And clinical faculty have developed the formative assessment tools to maximize skill and professional identity formation.

I am hopeful that, in its next iteration of the proposal, the Taskforce will heed CLEA and CLEAB’s comments and come back with recommendations that will ensure applicants for the bar are ready to engage in competent, ethical and professional representation of New York’s citizenry, corporations, and notforprofits.

 

 

 

 

Musings on the Bar Exam and Legal Education’s Attitude toward it

I have been studying and writing about the bar exam of late, so I appreciate the guest blogging opportunity, graciously offered by Mary Lynch, which I shall use to share some bar exam musings. Later this week, I hope to follow up with a bit more.

I noted with interest a recent New York Times feature, Is the Bar Too Low to Get into Law School? The feature offered perspectives from five legal professionals, four of whom are law professors, on how best to respond to declining bar exam passage rates. (Scores on the MBE, the anchor of the bar exam in almost every state, have declined again this year.) Two took issue with the bar exam itself, arguing for fundamental changes or its complete abolition. But Linda Sheryl Greene of the University of Wisconsin Law School argued that law schools simply need to do the work of preparing their students for the exam.

Law schools (or at least those not in the very top tier) indeed need to help their students prepare for the bar exam, but the bar exam also has to change in a way that allows law schools to do their part without the deleterious distraction of the exam’s heavy focus on recall of memorized law. Regrettably, bar exam reform efforts over the last 20 years have not focused on the one part of the exam that actually and exclusively tests lawyer competencies, requiring zero memorization of legal rules. That sadly neglected part of the exam is the performance test, which assigns a specific written lawyering task to be completed using a closed universe of factual materials and legal authorities. About one-fifth of the states do not even administer a performance test. Among states that do, the performance test remains the smallest part of the exam, accorded the least weight in scoring. It is in a very real sense the ugly step-child of the bar exam.

The behemoth of the bar exam, the MBE, compels examinees to study and memorize a copious number of legal rules. To be fair, the MBE does not test only for knowledge of law. But every skill set evaluated by the MBE—reading comprehension and legal analysis among them—is evaluated also by the performance test. The MBE’s primary value to the overall exam is psychometric—i.e., when scores on other parts of the exam are scaled to the MBE, the overall exam achieves testing reliability. A reasonable level of testing reliability can be achieved if the MBE is weighted at 40% of the overall score. (See page 13 of this article by the National Conference of Bar Examiners’ former Director of Research.) However, the NCBE recommends 50%, a recommendation that most states follow.

What of the rest of the exam? In every state, the remaining part of the score comes mostly from answers to essay questions, which, like the MBE, require memorization and recall of legal rules. If the MBE is testing knowledge of law (and creating more than enough focus on rote memorization), what reason other than inertia is there for essay questions to retain such a significant place on bar exams? Or to remain on bar exams at all? For years, essay questions were the venue for testing knowledge of state-specific law. However, most states now use the NCBE’s Multistate Essay Examination. And, as a growing number of states adopt the Uniform Bar Examination, several are employing other means outside of the bar exam, such as a required seminar, to ensure that new lawyers are familiar with unique attributes of local law.

And that takes me back to the performance test, the most valid of the testing instruments on the bar exam. The performance test was the answer from bar examiners 20 years ago to the recommendations of the MacCrate Report, which called on law schools and bar examiners to increase their attention to lawyering skills. Since then, while the MBE and essay examinations have been expanded, the performance test has remained stagnant. That needs to change. Through careful attention to the various skills today’s beginning lawyers have to perform, examiners should be able to reinvigorate the performance test and expand its skills coverage. They should also be able to increase the inadequate weight given to the performance test in scoring.

As for legal education’s attitude and approach toward the bar, I think an exam that focuses more heavily on skills through performance testing is one that would put law schools in a better position to help their students prepare. Because performance tests do not evaluate substantive knowledge of law, bar preparation specialists in law schools can easily administer performance tests from previous bar exams to students as both formative and evaluative assessments. Legal Writing professors have been using performance test-style problems for many years, especially with first-year students. Clinical professors use them, and, yes, even some doctrinal professors have too.  (I compiled a list of articles discussing the use of performance test-based problems by law professors in footnote 269 of my recent article.)

%d bloggers like this: