How LSSSE Informs Best Practices in Legal Education

Chad C. Christensen and Meera E. Deo

The Law School Survey of Student Engagement is based on decades of empirical research on effective educational practices showing that the more engaged students are, the better their academic and professional outcomes.[1] Since 2004, LSSSE has conducted an annual survey of law students in partnership with law schools across the country.[2] Survey results provide an opportunity for schools to better understand their student population and for LSSSE staff to document, reflect on, and influence trends in legal education.[3]

The LSSSE survey items were created out of best practices in teaching and learning; as such, they align well with Roy Stuckey’s Best Practices for Legal Education and the Carnegie Foundation’s Educating Lawyers – two publications that serve as foundational works of this blog.

For this post, we focus on best practice concepts described by Stuckey in his book and highlighted in Chapter 4, “Best Practices for Delivering Instruction, Generally”.  These recommendations come from Section C, which urges professors to “Create and Maintain Effective and Healthy Teaching and Learning Environments” by adherence to three principles:

  • Have High Expectations
  • Foster a Supportive Environment
  • Encourage Collaboration

Have High Expectations

Two questions on the LSSSE survey focus on high expectations and academic rigor.  The first asks students how often they worked harder than they thought they could to meet faculty members’ standards or expectations.  In 2019, 59% of law students frequently[4] worked harder than they thought they could to meet faculty standards or expectations, reflecting an increase since 2012 (52%). This positive trend indicates that students are being challenged in more meaningful ways than they were in the past and working hard to meet the high expectations of their professors.  

Another LSSSE question asks students to report the extent to which their exams have challenged them to do their best work.  For this question a score of five or higher on a seven-point Likert-scale indicates significant challenge.[5] In 2019, over 90% of LSSSE respondents indicated they were being challenged by exams in class. 

Taken together, this LSSSE data indicate that teachers are demanding a lot of their students, meeting Stuckey’s first suggestion to have high expectations. Students are also working hard and producing their best work to meet the challenges their professors put before them.

Foster a Supportive Environment

Creating a positive and supportive learning environment is critical to student success.[6]  A key component to this is student-faculty interaction – the ways and frequency with which faculty connect and interact with students in and out of the classroom.  Law students report overwhelmingly positive relationships with faculty.  In 2019, over three-fourths (76%) of students reported strong positive relationships with faculty.[7] Furthermore, 91% believed their instructors care about their learning and success in law school and 82% considered at least one instructor a mentor whom they could approach for advice or guidance.

Thus, faculty are creating supportive environments in class and effectively conveying their support to students.[8]

Encourage Collaboration

Teamwork and collaboration also are critical to student learning and the development of important professional skills for effective lawyering.[9]  It is important for students to engage with both faculty and classmates. Though students report positive relationships with faculty, LSSSE data reveal that law students are not collaborating with faculty as often as they could. A majority of students work with faculty on activities other than coursework, although a full 46% never do so.  Even more troubling, almost a quarter (23%) of law students report never having conversations with faculty outside of class. 

Surprisingly, students work with peers at even lower rates than they collaborate with faculty.  Only a quarter (24%) of law students report frequently[10] working with students on projects during class. One-third (33%) frequently work with classmates outside of class, again showing room for improvement.

When considering best practices in legal education, there is much to learn from Stuckey’s suggestions. And faculty have learned! LSSSE data reveal that students are working hard to meet their professors’ high expectations. Faculty also are succeeding in fostering a supportive classroom environment, as measured by overwhelmingly positive student-faculty interactions. However, professors can do more to promote teamwork and collaboration both inside and outside of class and both with students and amongst students themselves.


[1] More information on LSSSE is available at: https://lssse.indiana.edu/.

[2] To participate in the LSSSE survey, please contact the authors of this post or visit: https://lssse.indiana.edu/register.

[3] For instance, LSSSE Reports have shared trends regarding Diversity & Exclusion, The Cost of Women’s Success, and the ways in which Relationships Matter. For more information on LSSSE Reports, see https://lssse.indiana.edu/annual-results.

[4] This frequency includes respondents choosing “Very often” or “Often”.  

[5] Response options for this question range from 1 (“Very little”) to  7 (“Very much”).

[6] Stuckey, R. T. (2007). Best practices for legal education: A vision and a road map. Clinical Legal Education Association. P.87; Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE bulletin, 3, 7.; Wawrose, Susan, A More Human Place: Using Core Counseling Skills to Transform Faculty-Student Relationships (May 1, 2019). 55 Willamette L. Rev. 133 (2018), Available at SSRN: https://ssrn.com/abstract=3088008 or http://dx.doi.org/10.2139/ssrn.3088008

[7] These strong positive relationships are represented by a score of five or higher on a seven-point Likert scale.

[8] Women of color faculty, who typically carry more of the student services load than their colleagues, should be recognized for this work as it has clear implications for student outcomes and institutional success. Meera E. Deo, Unequal Profession: Race and Gender in Legal Academia (2019).

[9] Hamilton, N. W. (2014). Empirical research on the core competencies needed to practice law: WHAT do clients, new lawyers, and legal employers tell us?. The Bar Examiner, September, 14-34; Hamilton, N. W. (2019). Fostering and Assessing Law Student Teamwork and Team Leadership Skills. Hofstra Law Review, Forthcoming.

[10] This frequency includes respondents choosing “Very often” or “Often”.

AALS Panel Preview: Teaching Commercial Law in the 21

Kara Bruce, Professor of Law, University of Toledo College of Law

This year’s AALS Annual Meeting features a number of dynamic panels for the commercial law crowd.  Offerings include an update from the joint Uniform Law Commission and American Law Institute task force on potential UCC amendments to address emerging technologies,[1] a heavy-hitting panel considering “the next post-crisis financial reform,”[2] and a works-in-progress series featuring the work of junior consumer law scholars. The Financial Institutions Section will also host an offsite virtual scholarship workshop on the afternoon of January 6.[3]

I am looking forward to a panel co-sponsored by the Sections on Commercial and Consumer Law, Teaching Methods, and Technology, Law, and Legal Education, titled Teaching Commercial Law in the 21st Century.[4]  Perhaps because commercial law subjects are so difficult for students to access, I have found that my commercial law colleagues are particularly thoughtful teachers, and I invariably pick up good ideas from my discussions with them.

This panel, like many good pedagogical exercises, arose from the consideration of educational outcomes.  Two of our panelists, John McGarvey and Bill Henning, are members of the American Law Institute and Uniform Law Commission’s Permanent Editorial Board, which addresses interpretive problems with the UCC and recommends amendments to the Code.  In that capacity, they have observed several recent appellate court decisions that have flubbed application of core commercial law concepts.  These high-profile cases have led them to question how law schools are preparing graduates in the area of commercial law.  And thus, the panel was born.  Carliss Chatman and I were invited to join what we hope will be an engaging Q&A-style conversation.

Given that no professor can cover the entirety of their chosen field in a survey-style course, our panelists will share what skills and competencies they prioritize in their courses.  We will share techniques for integrating developing technologies, racial justice, and practice skills into the commercial law curriculum.  We will also consider how administrative-level decision-making can support our goal of producing graduates with a baseline competency in commercial law.

Teaching commercial law in the best of times often involves dragging students though a dense thicket of statutory text.  These challenges are magnified during a pandemic, when our students (and perhaps we) may face loss, grief, financial instability, tension, and other hardships.  The panel will conclude by discussing strategies for teaching effectively through the COVID-19 crisis.

We hope that readers of the Best Practices Blog will join us for this panel and contribute to the discussion.  If there are topics you’d like the panelists to address, please reach out to me at kara.bruce@utoledo.edu.


[1] Section on Commercial and Consumer Law, Co-Sponsored by Financial Institutions and Consumer Financial Services: Commercial Law in the 21st Century, Tuesday, January 5, at 11:00 a.m. EST.  See Program for more details. 

[2] Section on Financial Institutions and Consumer Financial Services, Co-Sponsored by Commercial and Consumer Law: The Next Post-Crisis Financial Reform, Tuesday, January 5, at 1:15 p.m. EST.  See Program for more details. 

[3] The panel will take place via zoom from 1-5 p.m. EST on January 6.  Please contact Patricia McCoy for log-in details and additional information.

[4] Section on Commercial and Consumer Law, Co-Sponsored by Teaching Methods and Technology, Law and Legal Education: Teaching Commercial Law in the 21st Century, Tuesday, January 5, at 4:15 p.m. EST.  See Program for more details. 

The Disparate Treatment of Clinical Law Faculty

By: Robert Kuehn, Washington University School of Law

In her recent presidential message, Abolish the Academic Caste System, the president of the American Association of Law Schools (AALS) called on law schools to address the caste system within law faculties by providing parity in security of positon and salary to non-tenure/tenure track faculty, such as the overwhelming majority of law clinic and externship instructors.[i] Data from the just completed Center for the Study of Applied Legal Education (CSALE) 2019-20 Survey of Applied Legal Education of  95% of law schools and 1,300 law clinic and externship instructors show widespread disparate treatment of clinical instructors (i.e., law clinic and externship instructors) and a lack of progress in providing parity between those who teach in law clinics and externships and those teaching doctrinal courses.[ii]

In 1998, 46% of clinical teachers were in tenure or tenure-track positions.[iii] Yet as the chart below indicates, the percentage of clinical faculty in tenure/tenure track positions, even when including lesser status clinical/programmatic tenure positions, has declined to just 29%, and decreased by more than 30% over just the last 12 years (temporary appointment clinical fellows excluded from all tables).


  Source: CSALE 2019-20 Survey of Applied Legal Education

Though there have been notable exceptions at a few schools, law clinic and externship hiring has disproportionately been for contract positions since the 2010 downturn in law school applications and accompanying financial challenges.

This increasing pattern of hiring non-tenure track clinical faculty can be seen below when comparing employment status to years of clinical teaching. Forty-six percent of clinical faculty teaching more than 12 years are in traditional or clinical/programmatic tenure or tenure-track positions. In contrast, only 23% of those hired within the last four-six years and just 16% of those hired in the last three years are in tenure/tenure-track positions. Although some clinical faculty hired into non-tenure-track positions may be permitted to move later into tenure-track positions, those limited instances cannot account for the increasingly lower status among more recently hired clinical instructors.


Source: CSALE 2019-20 Survey of Applied Legal Education

Non-tenure status has consequences for clinical faculty, beyond the limited participation in faculty governance and lower prestige that generally come with appointments other than traditional tenure. The table below compares the salaries the over 70% of law clinic and externship faculty not tenured/tenure track with the salaries reported by doctrinal faculty at the same schools. These clinical faculty are paid, on average, $30,000 per year less than their doctrinal colleagues at similar points in their careers. Even when salaries of clinical faculty with traditional or clinical tenure/tenure track are included in the calculations, clinical faculty on average make over $20,000 less than their doctrinal colleagues.

Sources: CSALE 2019-20 Survey of Applied Legal Education; 2018-19 SALT Salary Survey

The disparate treatment of clinical faculty in tenure appointments is most pronounced at schools ranked higher in the U.S. News annual law school rankings. Among schools with at least half of their clinical faculty in tenure/tenure-track positions, only one school ranked in the top 25 primarily appoints clinical faculty to traditional tenure-track positions, yet over 36% of the 50 lowest ranked schools provide this status to their clinical faculty.

Source: CSALE 2019-20 Survey of Applied Legal Education

Some law school clinical education programs even treat types of clinical instructors differently, providing less security of position and salary to those who teach in externships. CSALE survey data show that externship instructors are less likely to have traditional or clinical tenure/tenure track when compared to their law clinic peers (25% vs. 38%) and are almost 15 times more likely to be primarily in an administrative position with only occasional teaching responsibilities and sometimes little training in externship pedagogy.

Source: CSALE 2019-20 Survey of Applied Legal Education

Salaries of externship instructors also are considerably lower, with median annual salaries, on average, $20,000 less per year than those of law clinic instructors:

Source: CSALE 2019-20 Survey of Applied Legal Education

The latest CSALE survey shows that in spite of occasional stories about a school adopting tenure for its clinical faculty, the AALS president is right ─ the academy remains highly caste-like in its disparate treatment of clinical faculty, especially at higher ranked schools and even within clinical education programs at some schools. Indeed, if anything, progress toward parity appears to be slipping as an increasing percentage of new teaching positions in law clinics and externships are without the security of position and salary of doctrinal faculty.

The AALS has moved lately towards an Executive Committee comprised entirely of deans and former deans. If the members of the Executive Committee support their president’s call to end the caste system, they could act to do so at their own schools and call upon their fellow deans across the country to do the same.


[i] Darby Dickerson, Abolish the Academic Caste System, AALS News (Fall 2020), at https://www.aals.org/about/publications/newsletters/aals-news-fall-2020/presidents-message-abolish-the-academic-caste-system/.

[ii] Center for the Study of Applied Legal Education (CSALE), 2019-20 Survey of Applied Legal Education (2020), at https://www.csale.org/#results.

[iii] Richard K. Neumann Jr., Women in Legal Education: What the Statistics Show, 50 J. Legal Educ. 313, 328 (2000).

[iv] 2018-19 SALT Salary Survey, SALT EQUALIZER (Nov. 2019), at https://www.saltlaw.org/wp-content/uploads/2015/03/SALT-salary-survey-2019-final-draft.pdf.

Welcome, 2021! – and a Round-Up of Pedagogy Sessions at this week’s AALS Annual Meeting

Dear readers, authors, commenters, and friends far and wide:

Happy New Year!

We look forward to another year of exciting and thought-provoking discussion with you through the Best Practices for Legal Education blog. 

We begin 2021 with the AALS annual meeting, being held virtually, that you can access here

The conference will include some fantastic programs to help us share skills and techniques in this tumultuous teaching environment. We’ve compiled a round-up of the sessions best suited for those interested in deepening our grasp of pedagogy across a wide range of subject areas. Please feel free to comment below on what you’re learning as the conference progresses!

Tues. Jan. 5

4:15-5:30pm: Section on Civil Rights, Co-Sponsored by Criminal Justice: Teaching About Civil Rights During Incarceration

4:15-5:30pm: Section on Commercial and Consumer Law, Co-Sponsored by Teaching Methods and Technology, Law, and Legal Education: Teaching Commercial Law in the 21st Century

4:15-5:30pm: Section on Professional Responsibility: Bright Ideas and Best Practices for Online Teaching in Professional Responsibility Courses

Weds. Jan. 6

11am-12:15pm: Section on Pro Bono & Public Service Opportunities, Co-Sponsored by Clinical Legal Education, Leadership, and Poverty Law: Calling Out and Leaning In to Racial and Class Inequities in Experiential Learning Opportunities

2:45-4:00pm: Section on Global Engagement, Co-Sponsored by Teaching Methods, Technology and Law and Legal Education: Virtual Mobility: Innovating and Promoting Global Legal Education in Times of Crisis

4:15-5:30pm: Section on Criminal Justice: Beyond 2020: Decarceral, Anti-Racist and Non-Traditional Teaching

Thurs. Jan 7

2:45-4:00pm: AALS Discussion Group: How the Pandemic Made Me a Better Teacher – Lessons Learned and Plans for Change

4:15-5:30pm: Section of Family and Juvenile Law: Family Law – Creative and Experiential Teaching Tips

Fri. Jan. 8

2:45-4:00pm: Section on New Law Professors: Spreading the Word – Law Professors as Teachers, Scholars, and Legal Influencers

4:15-5:30pm: Section on Teaching Methods: Best Practices for Creating and Administering Mid-Term Exams

4:15-5:30pm: Section on Women in Legal Education, Co-sponsored by Clinical Legal Education, Legal Writing Reasoning, and Research and Teaching Methods: Gender, Power, and Pedagogy in the Pandemic

Friday, Jan. 8 2:45-4:00

Sat. Jan. 9

2:45-4pm: Section on Balance in Legal Education, Clinical Legal Education, and Leadership Joint Pedagogy: Teaching Leadership Skills in a Time of Crisis

As you consider your own teaching and writing, please consider posting your original content with us.  You can learn more about the purpose and history of the best practices blog here.

With best wishes for a great 2021,

Melanie and Davida

Looking At Ourselves–How Can Reduce Barriers to Entry in the Legal Profession?

Jennifer S. Bard, Visiting Professor of Law, University of Florida Levin College of Law

Over a series of past posts, I have looked at how law school could be adapted so that it does not disadvantage students who come not already knowing how to play the law school game. As we have known for a long time, some groups of students experience more initial success than others in law school–and these differences are magnified by the effect first semester grades can have on lifelong implications in terms of employment opportunities. Most law students catch on quickly after the first semester, but their self-esteem may have already been severely damaged in what Sara Berman has called “a zero-sum environment where initially-lower performers are not encouraged to improve in consistent and meaningful ways”.

But taking a step back from equalizing the experience of students already in law schools, it may be time to think about who isn’t there–and why. What aspects of legal education, such as the cost and program structure, create barriers to entry?  And how do these barriers to entry worsen an ever growing justice gap in the United States where only a small percentage of people who would benefit from legal representation have access to a lawyer?

The primary barriers are the cost of legal education and how it is structured.. The challenge we face is that there are barriers at every stage of the process, from high school graduation  to college entry and beyond.  Homelessness, substance use, mental health can all be factors in making  higher education inaccessible. Also, we know of the barriers students with disabilities face when they get to law school (or college), but we don’t know how many people who chose not to attend might have done so if they could do so from a more accessible location.  And of course, many scholars have pointed out that systemic racism is itself a formidable barrier in gaining admission to law school. 

We can’t as legal educators end the systemic racism and economic inequalities that block many people from even being eligible to attend law school.  Nor can we always reconfigure our aging infrastructure. But we can take responsibility for what we charge students to attend and how we structure the conditions for completion.

The first step to lowering the barriers within our own control is to recognize them. We need to re-evaluate the very structure of legal education–which can be most easily seen by reviewing the standards that both govern and reflect current practice. My intent is not to criticize the existing ABA standards or those who drafted and uphold them nor to suggest that they be rescinded, and legal education deregulated.   But rather to recognize the consequences and costs of these practices so we can better evaluate their value.  Below is a list of some obvious suspects–I’m sure everyone can generate more.

*Restrictions on Distance Education

With few exceptions, approved law schools cannot confer a degree on a student who does not earn two-thirds of their credits in face to face instruction. This continues to put law school out-of-step with nearly every other degree-granting program in the United States and to both the cost and physical demands of attending law school.

*Expensive Eligibility Requirements:

B.A. Required

Next on the list, we need to consider the cost in time and money of requiring that students earn a B.A. before enrolling in law school.  Lawyers in Europe, South America, Africa, Asia, Australia, New Zealand are all practicing laws at the highest possible levels without seven years of post-secondary specialty training.

 *Length of Degree and Time Limit on Completion

Not only must students complete a B.A., but they must also accumulate 87 credits within 84 months of enrollment.  This of course adds cost in the form of tuition, but it also requires an even greater expenditure of time away from family and limit on the ability to earn a living.

Each of these requirements, indeed each of the Standards which govern legal education, were developed with the best intentions, but if the pandemic has taught us anything, it’s that just because we haven’t done something before doesn’t mean we can’t or shouldn’t find a way to do it now.

Using What We’ve Learned About the Effect of Racism and Economic Disparities on Law Students During the Pandemic

Jennifer S. Bard, Visiting Professor of Law, Levin College of Law

In my last two blog posts, I wrote about how law school’s structure unfairly benefits students who come in knowing how the game is played and those with the resources to play it under the best possible conditions.  I’ve also linked legal education to the systemic bias of the legal profession.

Our current situation, a still spreading deadly pandemic that has so far claimed over 250,000 lives in the U.S., has intensified those inequalities for everyone.  The evidence is already coming in of how the Pandemic is harming first-generation students.  

But it has also provided a clearer window into what was always apparent to student services professionals but not so much to faculty–how much harder the law school experience is for students who come to it with fewer resources of every kind.   

For example, online learning is only as good as the environment in which students learn and we are already seeing effects on students with the least resources. In normal times, all of our students have near 24/7 access to quiet, safe, comfortable places to study, engage in co-curricular activities, meet with faculty, and even take exams. Places without pets, younger siblings, or household chores.  They have lightning fast internet, large monitors, bulk printers, and IT support for when things go wrong.

But of course a lot of what’s going isn’t visible in a 50 minute Zoom session.  On campus, students have access to food either directly, by attending events, or can sometimes be signed up for university meal plans.  But in a world where by some estimates, pre-pandemic as many as half of all U.S. college students experience food insecurity hunger is an increasing peril as is homelessness.   The end of the moratorium on evictions means that as many as 8 million people will lose their homes over the next four months. 

We  also know that during the pandemic college students are facing worse mental health and that for many of our students home is not a safe place as they face abuse from parents and domestic partners. Research is emerging that like other segments of the population, students are drinking more during the Pandemic and are likely part of the increase in overdose deaths.

And then there’s the virus itself. As we all know (and have known for a long time), it very much does infect young adults–and it can hit them hard.   All the factors that contribute to racial bias in health care are magnified by those that put Black, Indigenous, Latino communities at greater risk of infection and, once infected, at greater risk of dying.  In addition, the harm caused by the uncertainty, fear, and loss triggered by living in pandemic conditions.  These are only magnified by our law students who have faced trauma as bar examiners  are caught flat footed and many of the pathways to employment, such as in-person summer placements, were disrupted.

Layered on top of economic disparities issues of systemic racism, sexism, and homophobia, these economic disparities mean that students come into law school with very  different levels of debt. Which itself is affected by racial disparities.   These factors are magnified in law students who come to us after four years of borrowing money for college. (The best information on law school debt is at Accesslex).

The Pandemic will end, and law students will once again have full access to law school facilities. But this glimpse into the real differences in backgrounds and resources should be a starting place for us to look at the law school experience, the gateway to the legal profession.

If any good can come from the experience of being so much closer to our students’ day to day lives, it should be an increased urgency to think about how we can make law school more inclusive. 

In my next post, I will be more specific starting with a proposal reduce the cost of a law degree by moving a year of course work to the undergraduate level.  Doing that would reduce the barriers to entry in the legal profession that saddle lawyers with debt and deprive most individuals in need of legal help from those best trained to assist them. 

Best Practices in 2020

Sean M. Scott, President and Dean – California Western School of Law

As a dean, I have spent a tremendous amount of time over the past six months contemplating best practices as regards to students and how to manage the challenges wrought by COVID 19.  Should we move to pass/fail grading?  How can we incorporate best practices for online learning and teaching into our classrooms?  What are best practices as to the administration of exams? We have spent less time as a community discussing the impact of the virus on faculty, particularly untenured faculty.  The virus has intensified the existing tension between the demands of career and the demands of family.  For those faculty with school aged children, or elderly relatives, those competing demands are likely to negatively impact the faculty member’s ability to diligently pursue their research and scholarship.  What should the legal academy’s response be to those whose research and writing have been derailed because they are home schooling, or caring for elderly relatives whose isolation and physical vulnerabilities create increased demands on caregivers?  The faculty most likely to be facing this dilemma are women, as women regardless of sexual orientation, remain primarily responsible for childcare and elder care. 

Anecdotally, my conversations with my dean colleagues reveal that they are addressing these issues on a case by case basis.  Some faculty on the tenure track are approaching their deans seeking an extension of the tenure clock.  Other faculty are reluctant to seek such an extension, fearful that the request will be detrimental to their bids for tenure.  COVID has exacerbated this long-simmering tension between work and family demands.  What should a best practice be under these circumstances?  My recommendation is that we take the burden off of individual faculty members to make specific requests from their deans, and move towards a policy of granting a blanket one-year extension of the tenure clock for all pre-tenure faculty.  Faculty could opt out of the extension, rather than having to seek an individual accommodation.   As a best practice, it would reflect a profession wide recognition of the sacrifices that faculty who are caregivers must make, and takes a stand that they should not be penalized for meeting the immediate needs of family, the byproduct of which may be the delayed ability to focus on their research and writing. 

Race Ought to Be A Through-Line in Core Law School Curriculum

Darcy Meals, Assistant Director, Center for Access to Justice, Georgia State University College of Law

Long before law school we are taught that, as is engraved in the Supreme Court’s edifice, we are all entitled to “equal justice under law.” It is one of the fundamental ideals of the American legal system. And yet, it so often remains just that: an ideal to which we aspire but at which we have yet to arrive.

More than falling short of a collective goal, however, our nation’s history is replete with examples of racial injustice written into and undergirded by law: federally sanctioned redlining, internment of Japanese Americans, the failure to prosecute or convict police officers for killing Black people at rates three times their white counterparts. These more modern examples stem directly from the “manifest destiny” of our country’s founding and the early establishment of property law principles built on the commodification of Black bodies and seeking to justify taking land from indigenous peoples.

Despite the many overt examples, historical and current, of the ways in which race shapes our legal system, law faculty are often race-avoidant in teaching would-be lawyers. Race may be relegated to a “law and” discussion in upper-level seminars or covered only in reviewing seminal cases like Brown v. Board of Education. But its influence cannot be limited to one course or doctrinal area. Racial bias informs definitions of reasonableness and credible threat, shapes our views of what constitutes intentional infliction of emotional distress, and influences criminal sentencing and civil recovery. Stated or not, the influence of systemic racism pervades the law school curriculum because it permeates the entirety of the American legal system.

When race is absent from class discussions, that silence sends the message that the law is neutral and operates equally for all, when that is not the lived experience for so many. When we fail to incorporate issues of race and racism as foundational in core law school courses, we impede the professional development of future lawyers, who graduate without grappling with difficult but essential questions of how the law can operate to subordinate on the basis of race (and gender, class, age, sexual orientation, gender identification, religion, and ability – and the important intersections of those identities). Our silence about how race informs law and its application does real damage to students and can be particularly alienating – and intellectually violent – for students of color.

To encourage increased engagement with the ways in which race and racism undergird the American legal system, the Center for Access to Justice at Georgia State University College of Law compiled a Racial Justice Resource List. The non-exhaustive list, which will be updated as suggestions come in, is intended for law faculty teaching core (1L) courses who want to include assignments, readings, and discussion on issues of race. The list includes books, book chapters, law review articles, and multi-media for use in teaching how race influences law across the required curriculum. Where possible, the titles are linked to open-access sources. The resource list also provides suggested language regarding classroom expectations and learning objectives and considerations for how to amplify voices and stories that may not have been central in 1L syllabi.

Incorporating race into class assignments or discussions will likely lead to difficult, and even uncomfortable, conversations. Legal academia reflects the inequality otherwise manifest in the legal system: very few tenured law professors are Black. For white faculty, talking about race may run directly counter to the color blindness once expressly taught as virtuous. Leading a discussion, in a public setting, on a topic that has not been part of one’s scholarly expertise – and may not even feel a part of one’s personal experience – may lead to uncomfortable moments. But the work of antiracism requires that we give ourselves and our students the space to have brave and respectful discussions, to ask questions that will increase awareness of bias and how it manifests in the law.

Antiracism ought to inform every facet of legal education – hiring, promotion and tenure, admission, graduation – and it ought to be a through-line in the core law school curriculum. When it isn’t, we risk graduating lawyers who do not understand the origins of the law or its potential impact on clients, we perpetuate systems of inequality as if they were inevitable and deserving of maintenance, and we do a disservice to our students and to the profession, all the while undermining the commitment to equality we so proudly etched in stone.

Dean Darby Dickerson — on Equity, Security, and Status

Many of us were inspired to hear AALS President Darby Dickerson, Dean, UIC John Marshall Law School speak about “caste”, “candor”, and “change” during her address at the January 2020 Annual Meeting.

In this new article, first posted in the AALS Newsletter, she follows up with some worrisome data from the 2019-2020 CSALE study, works through potential harms to schools and students, and issues a call to action to address issues of pay equity, security, and status.

We value the conversations and discussions that often happen live and informally at in-person conferences as we share our progress (or lack thereof). So, dear readers, please let us know in the comments about any promising practices or initiatives at your schools. How are leaders addressing pay equity, security, and status in an era of hiring freezes and financial insecurity? What changes are you working toward?

Warmly,

Davida and Melanie

Could We Create a New Bar Exam?

Deborah Jones Merritt, Distinguished University Professor and John Deaver Drinko/Baker & Hostetler Chair in Law, The Ohio State University.

A few weeks ago, Logan Cornett and I published a major study of the work that new lawyers do—as well as the knowledge and skills they need for that work. John Lande has already offered two thoughtful posts about the study (thanks, John!). We think the research offers important information for legal educators, bar examiners, and workplace supervisors: we need to align education, licensing, and supervision to improve the service that new lawyers provide clients.

Will it be possible to achieve that alignment? In particular, can we make the bar exam a more valid assessment of the knowledge and skills that new attorneys need? I hope so. Neither states nor the National Conference of Bar Examiners (NCBE) have ever validated the current exams; that means we have no evidence that the skills and knowledge assessed on the exams match the ones that new lawyers use in practice. NCBE’s current attempt to validate the exam—through survey evidence—does not capture sufficient detail about the ways that new lawyers use their knowledge and skills in the workplace. Our study, based on 50 focus groups held in locations across the country, provides those more detailed insights. Combining our findings with those from NCBE’s study and other research could yield a valid licensing process.

An invalid exam would be embarrassing enough for a profession that prides itself on logic and reason, but our current bar exams have another flaw: they pass white candidates significantly more often than candidates of color. An exam that has never been validated, yet discriminates against candidates of color, is unthinkable in the modern age—and yet, we have tolerated these exams for decades.

It’s time to change, but will we have the courage to do so? Our profession has a deep attachment to closed-book exams, multiple choice questions, and time-pressured tests. The research that Logan Cornett and I did convincingly shows that none of these assessment methods are appropriate for the legal profession. Entry-level lawyers work from sources, not memory; they gather information to solve open-ended problems rather than choosing one canned answer from four; and, although they often practice under time pressure, the time constraints on the bar exam are dangerously unrealistic.

We need to challenge all three of these obstacles to a valid licensing process. Our report suggests many ways to accomplish that end. Here’s just one modest proposal that would significantly improve the validity of the bar exam:

  • Maintain the Multistate Professional Responsibility Exam (MPRE), which tests knowledge of basic principles of professional conduct, but make the exam open book. No one can wind their way through the dense rules of professional conduct and commentary without previous study, so an open-book exam won’t make the test “too easy.” On the contrary, an open-book exam would encourage new lawyers to check the rules and commentary whenever they face a conduct issue. That’s a habit we want to encourage, not discourage.
  • Maintain two performance tests like the ones currently prepared by NCBE, but allow 3 hours (rather than 90 minutes) for each test. Expanding the time frame would make these tests more realistic measures of minimum competence. It might also make grading more reliable because graders would be faced with real-world products produced under realistic time constraints.
  • Create a 3-hour research exam that consists of multiple-choice or fill-in-the-blank questions. This exam would test the kind of quick research that lawyers do routinely: What is the statute of limitations for medical malpractice in Ohio? Does a will need witnesses to be valid in Texas? Give candidates access to any online tools they desire to do this research.
  • Create a 3-hour, multiple-choice exam that tests (a) basic understanding of U.S. legal processes and sources of law OR (b) a single substantive subject (such as civil procedure, contracts, business law, or family law). If the latter, consider giving candidates a choice of the area in which they wish to test.

Our research suggests that the final doctrinal component of this exam should be open-book; new lawyers simply don’t work from memory. Instead, they internalize basic principles that allow them to identify issues in a client problem and find the specific rules they need to address that problem. But if bar examiners insist on a closed-book exam component, they could make this portion of the exam closed-book. Memorizing the rules related to a single subject—especially if the subject reflects an area in which the lawyer hopes to practice—more closely parallels the work that new lawyers do than memorizing the rules that govern ten or more fields.

A bar exam with the four components listed above could be administered according to our current schedule. I.e., candidates could take the MPRE on one occasion and the other three components over two days in late July or February. Alternatively, the proposed exam could be divided into five different components (the MPRE, each of two performance tests, the research test, and the doctrinal test) and states could allow candidates to take the components at different times and in any order. Breaking the exam into components would relieve some stress and give candidates more flexibility. It might also allow candidates to determine, while still in law school, that they lack skills needed for law practice—and either choose a different career or remedy those deficiencies through more coursework.

Our report offers other options for licensing, including a rigorously structured diploma privilege. We also recommend that states complement any written exam by requiring candidates to complete law school clinics and courses in client counseling and negotiation. Those experiences assure instruction, practice, and feedback on skills that are difficult to test through a written exam.

It’s time to get serious about aligning legal education and licensing with the work that new lawyers do. Only then will we fulfill our mission of protecting the public.


Virtual Practice/Virtual Externships: Past, Present and Future

By: Dena Bauman, UC Davis; Gillian Dutton, Seattle University; Kendall Kerew, Georgia State; Chipo C. Nyambuya, Loyola Chicago; and Amy Sankaran, University of Michigan

When we proposed a concurrent session focused on remote externships in November 2019 for the March 2020 Externships 10 Conference, we expected to be exploring a novel field placement arrangement, that of students working from a different location than their supervisors, and/or remote from the physical site.  A few months later, at the very time the conference was originally scheduled, we found ourselves in a global pandemic. We were living in the environment we planned to explore.

Overnight, our “novel” work arrangement became reality. Students were working remotely from both the law school and from field placements and supervisors. We planned to “model” a Zoom supervision meeting for panel attendees. Overnight, virtually all law school faculty, law students and field supervisors were relying almost exclusively on Zoom and other video conferencing platforms for classes, work at field placements and site visits.

Survey on Working Remotely and “Best Practices” Template:

Per our November 2019 proposal, we planned to survey the externship community about their current practices and rationales regarding remote placements. We also planned to create and distribute a template to use in evaluating and making decisions about virtual externship sites and advising students. We are linking to the template.

We distributed a revised survey through the national externship list serv in September 2020 that took a “snapshot” of practices before and during COVID. It also asked externship clinicians to consider what they might do post-COVID. [1] We used the 81 responses to create a template for assessing potential virtual placements and distributed the template at our rescheduled panel on October 23, 2020 along with a brief summary of the survey.  We used these three definitions.

  1. Traditional remote placement: Student and site supervisor physically present at the site, but remote from the faculty member.
  2. Remote supervisor placement: Student physically at the site but separate from the supervisor.
  3. Virtual remote placement: Student physically separate from both the site and supervisor.

We are including several tables summarizing the primary survey findings. The complete data report is linked here.

Pre-Pandemic: Few Schools Allowed Virtual Remote Placements but Many Allowed Traditional Remote Placements

Type of Placement AllowedPercentageNumber
Virtual remote10.78%11
Traditional Remote60.78%62
Remote supervisor11.76%12
None of the above16.67%17
Reason for not permitting remote placementsPercentage of SchoolsNumber of Schools
Students would not have the same immersive experience30.59%52
Students would not get the same or complete learning opportunities36.47%45
Supervisors would not give adequate feedback17.06%29
Reasons for allowing remote placementsPercentage of schoolsNumber of Schools
Benefit part-time/evening students19.44%7
Supervisors work in a different location16.67%6
Evolution of law practice16.67%6
Student want to work in virtual environments13.89%5

During the Pandemic (as of date of survey):  No Schools are Barring Remote and/or Virtual Placements

In response to the pandemic, we asked schools whether they were permitting a traditional remote placement and/or a virtual remote placement. 80 schools, out of the 81 respondents, permit virtual remote placements. 69 schools are permitting students to work apart from the faculty but are on site with the supervisor also present.

Another question asked why they were permitting these new arrangements. The three top reasons were:

ReasonPercentage of SchoolsNumber of Schools
Externship sites are closed or ask students to work remotely45.65%63
Students asking to work remotely for caregiving or other reasons31.16%43
Law school requirements10.14%14

Law school comments included: students need to complete graduation requirements, confidence that students are getting adequate supervision and experience in a new work environment, distance and traffic, and public health and safety concerns.

Nearly ¾ of the law schools said the changes have been both beneficial and problematic. As the pandemic continues through the fall, likely into the spring, and perhaps beyond, we will continue to learn much more about those changes.

Post-Pandemic: Training will be a Priority in Permitting Virtual Placements

Schools predict that training requirements for law students and field supervisors will be an important condition for considering virtual placements after the pandemic has passed.

RequirementPercentage of SchoolsNumber of Schools
Externship program will train students in remote work 22.73%50
Sites must demonstrate training for attorneys and students12.27%27
Supervisors must agree on training from externship programs11.36%25

Faculty Anticipate a Case-by-Case Approach

Factors that schools would consider in assessing the virtual and remote placement include family, health, financial and geographical circumstances, as well as the availability of the experience.  Schools also noted that it is too early to know if and how law practice will change, but that consideration is important in preparing our students for their careers. Additionally, we also realized that we need to scrutinize and analyze this nearly overnight reliance on technology in practice. Our presentation discussed these issues, such as access to technology and roles at home.  We will discuss those issues, and how they will affect our preparation of students for their post-graduate careers, in another posting.

Summary:  We believe our survey is a good starting point for future assessments, even as the pandemic continues to upend legal education and law practice.  Our template, which was favorably received at our panel, will help externship clinicians make decisions for placements and students, gather reasonably consistent information that they can use, and share with the community to build a record of useful information. Post-pandemic, we will depend on each other to share information about what we are contemplating and are implementing regarding virtual placements.


[1] Thanks to Inga Laurent, Gonzaga University School of Law, Theodora Pina, Santa Clara University School of Law, Sue Schechter, UC Berkeley School of Law and June Tai, Iowa College of law for their review and suggestions regarding the survey instrument.

Study Finds That Law Schools Fail to Prepare Students to Work with Clients and Negotiate

I previously posted an overview of the excellent report by Ohio State Professor Deborah Jones Merritt and Institute for the Advancement of the American Legal System Research Director Logan Cornett, Building a Better Bar: The Twelve Building Blocks of Minimum Competence.  This report is based on insights from 50 focus groups with 201 participants conducted in 2019-2020.  Forty-one focus groups involved new lawyers and nine focus groups involved lawyers who supervised new lawyers.  “New lawyers” were licensed mostly in 2016-2018.

This post provides detailed excerpts (without footnotes) from the report about new graduates’ unmet needs for instruction in interacting effectively with clients, negotiation, and understanding the “big picture” of client matters.

New Lawyers Were “Woefully Unprepared” to Work with Clients

More than half of the new lawyers in the study worked directly with clients.  And they were not ready for this.

“[They] described three clusters of abilities that they needed to work effectively with clients:


            “1. The ability to gain a client’s trust, gather relevant facts, and identify the client’s goals.


            “2. The ability to communicate regularly with clients, convey information and options in terms that a client can understand, and help the client choose a strategy.


            “3. The ability to manage client expectations, break bad news, and cope with difficult clients.  (p. 52)

“Gathering facts from clients was especially challenging for new lawyers.  In law school, one new lawyer explained, “they give you a set of facts” and “those are the only facts that exist in the world.”  In practice, she had to develop the skill of “getting more facts from the client and knowing which facts to ask for,” as well as the ability to “phrase questions to clients in a way that they understand what kind of information you’re trying to get, and they give you the information that’s actually useful to you.”  Supervisors agreed that new attorneys needed to do more “fact digging” with clients, “going back to ask some more questions to get really to the bottom of what’s happening.”

“Deciphering client goals was equally important. “Sometimes,” one supervisor reflected, “we don’t ask the client, ‘Well, what does victory look like?  What’s your goal here?”  Another supervisor agreed that new lawyers don’t pay enough attention to client goals:

“One thing that I noticed that a number of the young lawyers struggle with, which is helping a client get to yes.  Which is not, ‘well the law says this, so no, you can’t do that.’  [Instead, we need new lawyers to say]  ‘The law says this.  So if you want to accomplish your business goal, you will need to do these things.’  . . . Clients are not looking for us to tell them what they cannot do.  They’re looking for us to help them understand how to accomplish their business objectives.

“Several new lawyers elaborated on the same theme.  “I didn’t really understand,” a new in-house lawyer commented, the importance of “trying to understand the goals of what our business clients want to do.  Just because they have a certain idea of how to do it that may not be legal doesn’t mean we can’t find something legal to do, to try to get to the same result.”  Participants in another group nodded in agreement when a lawyer at a mid-sized firm observed: “we have to figure out this meandering way of getting to where they want to go that’s legal, so just asking the right questions so that you know what their actual final goal is” allows you to “get there in a way that actually makes sense.””  (pp. 53-54)

New Lawyers Had Difficulty in Communicating with Clients

After three years of law school, new lawyers had serious problems communicating with their clients.  We teach them to argue.  Listen and communicate effectively with clients, not so much.

“New lawyers in our focus groups described their need to communicate frequently with clients, especially when clients lacked experience with the legal system. “The number one complaint from clients of lawyers,” one declared, “is lack of communication, or poor communication, and not being told what the hell is going on in their case.”  “Especially in the discovery phase,” he continued, clients don’t understand the demands placed on them or the slow progress of the case. Taking time to “touch base” and offer “a lot of handholding” was essential for building client relationships.

“Equally important, new lawyers had to learn effective counseling skills.  Several contrasted their advocacy skills with advising ones.  They “felt very confident” with the former but not the latter.

 . . .

“Other new lawyers described learning how to “coach [clients] through a tough choice,” and helping them assess the costs and benefits of each course of action.  “I do that all the time with my clients,” a new lawyer from a small firm noted, “just like laying out like these are all the possibilities and their likelihood.  Do you really want to do this or do you want to walk away from it and just like call it a day?””  (p. 54)

Lawyers Had Problems Managing Expectations, Breaking Bad News, and Coping with Difficult Clients

In practice, lawyer serve clients who are real people, not the two-dimensional characters who populate law school textbooks.

“Some new lawyers in our focus groups grappled with clients living with mental illnesses, trauma, and other life challenges.  Counseling these clients was difficult, especially when delivering bad news.  “Somebody can know the black-letter law inside and out,” a bankruptcy lawyer observed, “and then their first day on the job they are sitting in front of somebody who is incredibly worried, incredibly anxious.”  There “hasn’t really been any formal training,” he continued, “on what do you do when this person’s on the brink of tears and you have to take him in front of the judge.”

“New lawyers in our groups had to overcome their initial desire to please clients, learning to deliver bad news candidly.  “It was a really hard skill for me to learn,” a new family lawyer admitted, “because I was kind of a pleaser at first. . . . But now I don’t really care if they don’t like what I have to say and advise them. I tell them they didn’t hire me to be a cheerleader.”  Another new lawyer learned to be “straightforward” about problems because his attempts to “tiptoe around” them led to misunderstandings.”  (p. 55)

New Lawyers Need Specific Communication Skills

The study found that “the current licensing scheme overlooks five key facets of [communication] competency.”   New lawyers must be able to:

  • “Communicate concisely;
    •          Communicate in language that clients understand;
    •          Choose communication methods that are effective for each audience and setting;
    •          Attend carefully to communications from others; and
    •          Negotiate effectively.”  (p. 65)

“New lawyers struggled to find the appropriate words for communicating with clients.  Even “practical” law school classes, one observed, “didn’t teach us how to talk to clients, how to get someone who’s charged with some heinous event to trust you well enough to tell you what’s happening.”  A prosecutor confessed: “One skill that actually I didn’t think I would need that I still don’t feel like I really have is talking with victims.  It’s definitely a skill and I’m not great at it.”” (p. 66)

“Effective communication requires reception as well as transmission.  Many focus group members faulted new lawyers for failing to attend carefully to messages sent from others.  They often referred to this problem as a failure to “listen,” but it was clear that the failure could occur either in written or oral communication.”


““Listening is vital,” one supervisor declared:

“”But a lot of new lawyers don’t seem to have that and maybe it’s maturity, but I think it’s something that can be practiced.  You need to listen to what your clients are saying.  You need to listen in our area to what members of the public are saying.  You need to listen to what the other lawyer at the other end of the phone is saying to read between the lines, ‘what does that lawyer really want?’” (p. 68)

New Lawyers Need Skills Negotiating with Counterparts, Clients, and Others

New lawyers are unprepared to negotiate because they are trained almost exclusively to argue.

“Focus group members identified negotiation as a distinctive communication style that was essential for their work.  Negotiation, they noted, is quite different from advocacy.  As new lawyers, they had to learn to “be collaborative,” “give a little to get a lot,” and “work together” with opponents.  The “litigious” argument styles they learned in law school did not work well during negotiations.

“In addition to negotiating with opposing counsel, new lawyers had to negotiate with union agents, pro se opponents, and their own clients.  One new lawyer working in-house even negotiated fee arrangements with outside law firms:

  •          “I do a lot of labor negotiation.  So it’s interesting because sometimes it is another attorney on the other side, but a lot of times it’s a business agent for a union who’s not an attorney.  So I guess approaching that in different ways has been an interesting thing to learn.”
  •          “Sometimes negotiating with your own clients on what we’d be willing to accept on a civil case, they think it’s worth a ton of money and you’re . . . trying to negotiate them to a reasonable place because ultimately going to trial wouldn’t really benefit them.”
  •          “Being in-house counsel, . . . the first negotiation is getting all the executives on board with the same deal.  Getting them all to agree that we should go after this deal on these terms is sometimes a bigger battle than negotiating it with opposing counsel afterwards.”
  •          “I remember one of my first discussions with my boss at the time and asking, he was like, ‘Reach out to outside counsel if you need to, talk about budget and things like that.’ . . . And so, that’s something that I had not done as far as negotiating price and what we can pay.”

“Several new lawyers wished they had taken negotiation classes in law school; a few suggested these classes should be required.  One tax attorney explained that, as someone who planned to do transactional work, he thought negotiation and mediation classes were only for people who planned to “do that for a living.”  Only after beginning his practice did he realize how much time he spent negotiating with clients and colleagues;  then he wished he had been encouraged or required to study negotiation in law school.

Supervisors agreed that “negotiation skills are huge,” and “absolutely important” in law practice.  One supervisor observed, “What I see lacking is the ability to negotiate provisions into a contract. . . .  [New lawyers] know the elements of different types of laws.  It’s just the question of negotiating contracts, or just negotiating in general, that seems to be lacking.  Which is what we spend a lot of time [doing].””  (pp. 69-70)

New Lawyers Need to Understand the “Big Picture” of Client Matters

Law students mostly learn little disconnected pieces of the law but not how to understand the “big picture” of their cases and act accordingly.

“Focus group members urged that lawyers must see the “big picture” in client matters to represent clients competently.  New lawyers, they suggested, often lack that ability.  One supervisor summarized this perspective by observing that new lawyers need “to think more at the forest level and less at the tree level.”  A new lawyer offered a similarly graphic explanation. “It took a few cases, seeing the full life cycle of the case, to really understand strategy,”

“A lack of experience with “forests” or “maps,” according to our focus group members, caused at least two problems.  First, without the ability to see the big picture, new lawyers could not effectively manage projects.  When given responsibility for their own cases, which was common in many organizations, they struggled to manage those cases.  Even when working as part of a larger team, they sometimes missed critical deadlines because they did not understand the full project’s timeline.

. . .

“Failure to understand the big picture caused a second failing among new lawyers: they had difficulty developing strategies to guide client matters.  These new lawyers knew the rules, but they did not know how to combine the rules into a successful strategy.”  (pp. 72-73)

Recommendations for Required Courses in Working with Clients and Negotiation

The report recommends that all law students take three credits to develop working with clients and three credits of negotiation.

Recommendation Six:  Require candidates to successfully complete three academic credits of coursework that develop their ability to interact effectively with clients.  These credits should focus specifically on client interaction and should include opportunities for students to practice that interaction and receive feedback.  Instructors should understand that successful completion of the course signifies that the student possesses the ability to interact effectively with clients as an entry-level lawyer.

Recommendation Seven:  Require candidates to successfully complete three academic credits of coursework that develop their ability to negotiate. These credits should focus specifically on negotiation and should include opportunities for students to practice that interaction and receive feedback.  Instructors should understand that successful completion of the course signifies that the student possesses the ability to negotiate effectively as an entry-level lawyer.”  (p. 98)

This post includes a list of ABA books that address these needs.  Marjorie Corman Aaron’s excellent book, Client Science, provides empirically-based advice for lawyers about counseling clients when giving bad news.  In this piece, I argue that law schools should teach students to think strategically, which is really what it means to think like a lawyer.  And my co-authored book, Litigation Interest and Risk Assessment:  Help Clients Make Good Litigation Decisions, helps lawyers work with clients to get the big picture of their cases, and to develop good litigation, negotiation, and mediation strategies.

Messages for Students

The report finds that pursuing continuous, self-directed learning is an “essential component of minimum competence.”  (pp. 80-82)

In my view, law schools should incorporate this from the outset.  Unfortunately, law schools emphasize external motivation so much that it can overwhelm students’ internal motivation to learn.  The hidden curriculum in legal education teaches that the most important criteria are curved grades, competitions, certificates, law review, clerkships, prestigious jobs, etc. etc.  This not only sends the implicit message to the “bottom” 90% of the class that they aren’t very good, but it also distracts students from taking the initiative to develop skills they will need in practice.

Faculty should counteract the hidden curriculum by repeatedly emphasizing that simply getting a law degree isn’t sufficient to be a competent new lawyer, and that students should continuously assess and address their learning needs, starting in law school.  You might require students to read this post and encourage them to read the full report.

Our Students are Stressed; Exercise Compassion

At the best of times, the life of a law student is stressful. Law students, like lawyers, are over-represented in reported statistics of depression and anxiety. Because law schools know the pressure their students are under, it has become common practice to provide stress-reducing interventions – mindfulness training and yoga classes come to mind.

This November, the political climate and an enduring pandemic will add to the high baseline of anxiety and depression that law students experience.

The 2020 election, at the very least, will feel like the most consequential election our students have experienced. Students will, of course, be aware of the political discord that is dividing our country. The discord may also be dividing their friendships and families. For young people finding their way in life, this is unsettling.

Law students may engage in behaviors that give them a sense of control over the election outcome. This is a good thing and should be encouraged. They can vote (assuming they are not casualties of some vote suppression strategies), support the candidate of their choice, or work at the polls. But even doing these basic things, the outcome of the election is likely to feel uncertain, uncontrollable and yet highly significant for students’ future. The combination of these factors is recipe for stress.

Compounding the electoral stress is the global pandemic. The coronavirus has disrupted students’ education, turned the typical law school experience on its head, and ripped away the one thing we all need when feeling anxious and depressed – a social network we can talk to, gain support from, and re-center our perspective of the future. And remember those mindfulness and yoga classes designed to help students’ well-being? Without being on campus or in the classroom, those have been relegated to afterthoughts for law school.

These are the reasons why now more than ever we need to exercise compassion. Compassion is more than empathy, an ability to take the perspective of others, to understand what law students are feeling. That’s a start. But compassion is when our feelings motive us to help alleviate at least some of their suffering.

Here are four things I’ve identified that I can do to support my students. Please feel free to share your own ideas by commenting on this post.

  • Laughter: Organize a lighthearted online pop quiz with your students. Inject humor into some of the material they should review. Avoid political humor, of course. Don’t worry if you’re not a trained comedian. Laughing at yourself for creating such bad jokes is also stress reducing.  As you can probably guess, there are many online resources to consult for anything from the best legal puns to the worst dad jokes.
  • Exercise: The thing we least want to do is often what we should do. As the weather turns colder – at least in my State – exercise becomes less appealing. But the science has become undeniable – exercise reduces stress. Assign a podcast and encourage your students to listen to it while exercising. Have students share their methods for working exercise into their day. Some of my students shared their plans to “commute” to class everyday – walking from and to their apartments before and after class.
  • Support the Right to Vote: If you haven’t already, give your students the Day off on Nov. 3rd, with no make-up class required.
  • Support each other: When we are kind, generous, and supportive of our friends and colleagues, we can make a difference in their well-being. I usually have my clinical students review their peers at the end of the semester. Each student provides me with their feedback for each of their peers. They must answer two questions. First, what do you admire most about Student A. Second, what is one thing Student A could do to improve. Each year I give this assignment, I’m amazed by how thoughtful my students are. And after I compile the feedback and communicate it to my students in an end of semester meeting, my students seem overwhelmed by the admiration and positive feelings their peers have for them. Of course, if you are teaching a larger class where students don’t get to know each other well, you could be the one to deliver a positive message about something you admire about them.   

Looking Ahead: The Performance Test on the Bar Exam Post Covid-19

Even amidst great uncertainty as to how the Covid-19 pandemic will impact future bar exams, we have already seen a few “when the dust settles” articles addressing how best to move forward with bar exams after the effects of the pandemic have waned.[i] While there are justifiable calls to rethink the exam entirely, especially in light of disparate outcomes based on race,[ii] multiple commentators have highlighted the need to enhance the exam’s focus on the performance test.[iii]

A telling indicator of the importance of the performance test appears in the Phase One Report of the National Conference of Bar Examiners’ (NCBE’s) Testing Task Force, which is conducting “a future-focused study to ensure that the bar examination continues to test the knowledge, skills, and abilities required for competent entry-level legal practice in a changing legal profession.”[iv] Among the points made most frequently by various bar exam stakeholders during listening sessions were the following: “The MPT [Multistate Performance Test] is the strength of the current exam” and “Lawyering Skills should be emphasized over subject matter knowledge.”[v] The performance test remains the only vehicle on the bar exam that directly evaluates lawyering skills and not subject matter knowledge of—some would say rote memorization of—law, which is tested by both the Multistate Bar Exam (MBE) and essay questions. As such, it ranks high as to testing validity and has great potential as an evaluator of competence. 

The NCBE, which currently provides the testing instruments used on the vast majority of state bar exams,[vi] is an influential force in the bar exam world, and it approaches change at a glacial pace.[vii] Hence, there is good reason to think that, while the pandemic might lead to changes in the manner of exam administration (e.g., remote administration in lieu of in-person), the heavy majority of states will continue to use the NCBE’s three tests: the MBE, the Multistate Essay Exam (MEE), and the MPT. Hence, a closer look at the most important of those three, the MPT, and ways in which performance testing can be improved is in order. What follows is a review of the current status of the performance test on bar exams in the United States and some thoughts on enhancing performance testing moving forward:

What is the performance test, and what is its purpose?

A performance test assigns a specific written lawyering task (e.g. memo, brief, or a letter), to be completed by the examinee relying on a closed universe of provided factual and legal materials. Performance tests, unlike essay questions and MBE questions, require no advance memorization of doctrine, as they provide a Library, which includes all the needed legal authorities in the form of cases, statutes, rules, regulations, or some combination thereof. The facts are presented in the File, usually through assorted documents such as transcripts of witness interviews, correspondence, etc. The allotted time for completion of a single performance test is 90 minutes.

The performance test is a creature of the ABA’s landmark 1992 MacCrate Report, which called on law schools to enhance training in assorted fundamental lawyering skills.[viii] The NCBE responded to the MacCrate Report by crafting the MPT as an instrument to evaluate six of the fundamental lawyering skills listed in the report: (1) Problem Solving, (2) Legal Analysis and Reasoning, (3) Factual Analysis, (4) Communication, (5) Organization and Management of a Legal Task, and (6) Recognizing and Resolving Ethical Dilemmas.[ix] The NCBE began offering MPT items to jurisdictions in 1997.[x]

How is the performance test currently being used on the bar exam?

The NCBE continues to produce the MPT, and 44 states administer it on their bar exam.[xi] Thirty-six of those states administer two MPT items, pursuant to their administration of the Uniform Bar Exam (UBE).[xii] Some of the remaining eight administer one MPT, and some administer two. In addition, Pennsylvania creates and administers its own performance test,[xiii] as does California.[xiv] In total then, 46 states administer at least one performance test on their bar exam. Only Florida, Louisiana, Michigan, and Virginia eschew performance testing.[xv]

Is the performance test evaluating all of the MacCrate skill sets adequately?

As I discussed in detail in a 2015 article, the performance test is not living up to its potential, as typical test items implicate mostly just the core skill set of legal analysis and reasoning.[xvi] It remains not at all clear that the performance test is encompassing all six of the MacCrate skill sets on a consistent basis. Most notable among the skill sets that remain under addressed are (1) Problem Solving and (3) Factual Analysis. 

Problem Solving

The MPT occasionally implicates problem solving through a test item requiring an evaluation of multiple competing approaches to a given client’s problem—for example, MPT2 on the July 2019 exam, in which examinees had to evaluate the merits of two different estate planning approaches for a mock client.[xvii] In the last several years, neither California’s nor Pennsylvania’s performance tests has presented comparable test items. 

Factual Analysis

As to factual analysis, because the source materials in the File include both relevant and irrelevant facts, the performance test does a good job of testing an examinee’s ability to identify relevant facts within a given set of materials. However, among the core competencies encompassed by the skill set of factual analysis are determining the need for further factual investigation and planning a factual investigation.[xviii] Performance tests have not encompassed these competencies. Indeed, in the past several years, no performance test has tasked examinees with identifying what additional facts would be useful and drafting a discovery plan for obtaining those facts. 

To be fair, given that the performance text currently plays a limited role on the exam, it is perhaps not possible to consistently encompass all of the MacCrate-inspired skill sets. The addition of more performance tests, or multiple-choice questions within performance tests, discussed below, could help in this regard. 

Does the performance test make adequate use of statutes and regulations as legal sources?

Of course, statutes and regulations are central to much of law practice today—so much so that numerous law schools are requiring first-year students to take a course on legislation and regulation. It would seem beyond dispute that all examinees on every bar exam should be required to display competence in reading and applying a statute or regulation without the aid of a case interpreting the statute or regulation at issue.  Neither MBE questions nor MEE questions (nor state-specific essay questions) directly require examinees to do this. In contrast, the performance test easily can. Unfortunately, however, statutes or regulations (or comparable types of non-case authorities) do not consistently appear on recent performance tests—to say nothing of the four states that do not administer a performance test at all. 

To the NCBE’s credit, at least one of the two MPT test items in each exam administration since 2010 has included at least one statute, rule, or regulation.[xix] It is not clear the extent to which examines on these various MPTs needed to reason directly from the statute, rule, or regulation (as opposed to reasoning from a provided case that interpreted the statute, rule, or regulation) to resolve an issue, but a cursory review suggests in several cases that, to a goodly extent, they did. By way of example, one of the MPT items on the February 2015 exam included only a statute and regulations in the Library, and examinees were tasked with “parsing . . . HIPAA regulations” in order to answer effectively.[xx]

 In contrast, however, since shifting to the 90-minute performance test format in July 2017, California has included only cases as authority on its performance tests through the February 2020 exam.[xxi] This reflects a major failing of the California bar exam. 

The Pennsylvania examiners have done a better job, having deployed statutes as part of the law library on several different performance tests,[xxii] the first of which (February 2017) I highlighted in an earlier post. Still, though, given that there is only one performance test on each administration of the Pennsylvania exam, each Pennsylvania performance test with only cases (and there continue to be several of those[xxiii]) represents an exam that does not evaluate the fundamental competency of reading and reasoning from a statute, rule, or regulation. 

How can bar examinations make greater use of performance tests?

Ideally, all 50 states would administer at least a bare minimum of two performance test items on every exam. Though progress is being made toward that goal, we are not there yet. 

First, as noted earlier, four states do not administer a performance test at all. One can only speculate as to what reason, aside from inertia, leads the bar examiners in Florida, Louisiana, Michigan, and Virginia to forego a testing instrument of such value.

Second, as also noted earlier, several states that use the MPT outside the construct of the UBE, as well as Pennsylvania and California, administer only one performance test, not two. The use of two test items, as required for the UBE, affords an opportunity to evaluate a greater array of lawyering skills via the performance test. The use of only one relegates the performance test to playing a minimal role on the exam overall. By way of illustration, while the two MPT items in UBE jurisdictions count for 20% of the exam score, the one MPT item in Pennsylvania counts for only 11%, and the one MPT item in Nevada counts for only 10.5%.[xxiv]

Third is the broader question of expanding the use of the performance test beyond just two test items and the current 20% allocation on the UBE. As I noted in my 2015 article, this is a tougher challenge.[xxv] The MBE counts for such a large chunk of the score on the exam—50% in UBE and most other jurisdictions—because it is a psychometric anchor for the exam. In other words, the MBE has very strong testing reliability. As a multiple-choice exam that tests knowledge of various areas of substantive law, thereby calling for rote memorization, the MBE has weaker testing validity. In contrast, the performance test has high testing validity, but, because of its size and the subjectivity inherent in grading, has lower testing reliability. The NCBE prioritizes strong testing reliability for its products and hence allots greater percentages to the MBE and essay questions, 50% and 30% respectively.[xxvi]

There are many possible approaches that could allow for increased use of, and a greater scoring weight allotted to, performance tests, but one suggested by Jason Solomon in his recent article bears serious consideration. Solomon suggests the use of multiple-choice questions, which inherently afford greater testing reliability, within a performance test format.[xxvii] Instead of, or in addition to, writing an answer, as required on the traditional performance test, examinees would answer a series of multiple-choice questions on the materials provided in the test and on the most effective ways to resolve the issues presented. Unlike MBE questions, these questions would not require examinees to recall memorized legal doctrine, but rather to carefully review the provided factual or legal materials. Multiple-choice questions within a performance test format could also be an effective vehicle for adding legal research to the bar exam, as even within a closed-book format, examines could be asked questions about the most effective research strategy to build on the provided materials.[xxviii]

Conclusion

In sum, to improve the bar exam going forward still requires a focus on the performance test. The following goals are worth pursuing and achieving:

  • Performance testing that consistently encompasses more of the fundamental lawyering skills that the test was originally designed to encompass, including problem solving and factual analysis.
  • More and consistent use of statutory and regulatory authorities in the law library of performance tests
  • Use of performance testing in all 50 states, not just 46
  • At least two performance test items in all jurisdictions
  • Research into the use of multiple-choice questions within a performance test framework, including as a vehicle for testing legal research

[i] See, e.g., Vikram David Amar, What About the Bar Exam After the 2020 Dust Settles?, Verdict, Legal Analysis and Commentary from Justia (Sept 17, 2020), https://verdict.justia.com/2020/09/17/what-about-the-bar-exam-after-the-2020-dust-settles; Jason Solomon, INSIGHT: Saving the Bar Exam By Focusing on Performance, United States Law Week (July 16, 2020), https://news.bloomberglaw.com/us-law-week/insight-saving-the-bar-exam-by-focusing-on-performance.

[ii] Several thoughtful pieces touching on racial inequities caused by the bar exam appear in volume 3, Issue 4 of AccessLex’s publication, Raising the Bar, 3-12 (Fall 2020), https://www.accesslex.org/resources/raising-the-bar-fall-2020.

[iii] Amar, supra Note i; Solomon, supra Note i. 

[iv] Nat’l Conf. of Bar Exam’rs, Your Voice: Stakeholder Thoughts About the Bar Exam, Phase One Report of the Testing Task Force, 1 (Aug. 2019), https://testingtaskforce.org/wp-content/uploads/2020/03/FINAL-Listening-Session-Executive-Summary-with-Appendices-2.pdf.

[v] Id. at 3. 

[vi] See Adoption of the Uniform Bar Examination, with NCBE Tests Administered by Non-UBE Jurisdictions, Nat’l Conf. of Bar Exam’rs (Aug 18, 2020), https://www.ncbex.org/pdfviewer/?file=%2Fdmsdocument%2F196.

[vii] The NCBE is notorious for approaching change very cautiously and incrementally. In 2012, the then President of the NCBE invoked geological metaphors to describe how bar exams evolve, writing that any evolution of the exam will be “more glacial than volcanic.” Erica Moeser, President’s Page, B. Examiner, Dec. 2012 at 4, 4.

[viii] MPT Skills Tested, Nat’l Conf. of Bar Exam’rs https://www.ncbex.org/pdfviewer/?file=%2Fdmsdocument%2F54 (last visited Oct. 20, 2020) (citing Am. Bar Ass’n Section of Legal Educ. and Admissions to the Bar, Legal Educ. and Prof. Dev. – An Educational Continuum, Report of the Task Force on Law Schools and the Profession: Narrowing the Gap 138-41 (1992) [MacCrate Report]).

[ix] Judith Gunderson, Happy Birthday, MPT!, B. Examiner, Nov. 2007, at 18. See also MPT Skills Testedsupra Note viii.

[x] Id. at 20. 

[xi] Multistate Performance Test: Jurisdictions Administering the MPT, Nat’l Conf. of Bar Exam’rs, https://www.ncbex.org/exams/mpt/ (last visited Oct. 20, 2020).

[xii] Adoption of the Uniform Bar Examination, with NCBE Tests Administered by Non-UBE Jurisdictions, supra Note vi.

[xiii] Pa. Bd. of Law Exam’rs, Bar Examinationhttps://www.pabarexam.org/bar_exam_information/bebasics.htm (last visited Oct. 20, 2020).

[xiv] The State Bar of Cal., Examinations, California Bar Examinationhttp://www.calbar.ca.gov/Admissions/Examinations (last visited Oct. 20, 2020).

[xv] Multistate Performance Test: Jurisdictions Administering the MPTsupra Note xi.

[xvi] Ben Bratman, Improving the Performance of the Performance Test: The Key to Meaningful Bar Exam Reform, 83 UMKC L. Rev. 565, 584-97 (2015).

[xvii] 2019 MPT Summaries, Nat’l Conf. of Bar Exam’rs, https://www.ncbex.org/pdfviewer/?file=%2Fdmsdocument%2F233 (last visited Oct. 20, 2020).

[xviii] MPT Skills Testedsupra Note viii.

[xix] See Free Summaries of MPTs from Recent Administrations Free MPTs and Point Sheets from Older Administrations, Nat’l Conf. of Bar Exam’rs, https://www.ncbex.org/exams/mpt/preparing/ (last visited Oct. 20, 2020).

[xx] 2015 MPT Summaries, Nat’l Conf. of Bar Exam’rs, https://www.ncbex.org/pdfviewer/?file=%2Fdmsdocument%2F175 (last visited Oct. 20, 2020).

[xxi] The State Bar of Cal., Past Exams, Performance Tests and Selected Answershttps://www.calbar.ca.gov/Admissions/Examinations/California-Bar-Examination/Past-Exams (last visited Oct. 20, 2020).

[xxii] See, e.g., Pa. Bd. of Law Exam’rs, July 2019 Pennsylvania Bar Examination, Essay Questions and Examiners’ Analyses and Performance Test, 59-61, https://www.pabarexam.org/pdf/qa/qa719.pdf (last visited Oct. 20, 2020).

[xxiii] See, e.g., Pa. Bd. of Law Exam’rs, February 2020 Pennsylvania Bar Examination, Essay Questions and Examiners’ Analyses and Performance Test, 62-70, https://www.pabarexam.org/pdf/qa/qa220.pdf (last visited Oct. 20, 2020).

[xxiv] Nat’l Conf. of Bar Exam’rs, Comprehensive Guide to Bar Admission Requirements 36-37 (2020), available at https://www.ncbex.org/assets/BarAdmissionGuide/CompGuide2020_021820_Online_Final.pdf.

[xxv] Bratman, supra Note xvi, at 605-10.

[xxvi] Id. at 608.

[xxvii] Solomon, supra Note i.

[xxviii] If and how to test legal research on the bar exam has been on the NCBE’s radar screen since at least 2006. See Erica Moeser, President’s Page, B. Examiner, May 2006, at 4, 5.

SALT Social Justice in Action series: Anti-Racist Hiring Practices

by Professor Brooks Holland, Gonzaga Law School

The Society of American Law Teachers has been proud to share its four-part Fall 2020 Social Justice in Action webinar series. This webinar program already has presented leaders in the field to address anti-racism values and practices in three specific contexts, including anti-racism in the core law school curriculum, equity and inclusion in online legal education, and anti-racist work for the long-haul. Approximately 500 people attended these first three webinars, and many more have since viewed them on SALT’s website.

The fourth installment of SALT’s webinar series will take place on October 16, 2020 at 12:00pm PT and 3:00pm ET, and it will address one of the most important influences we as educators have on the future of higher education and the legal profession: our hiring practices. We all know that law school and higher education hiring practices is one of the areas where we most struggle to fulfill our commitment to diversity and antiracist values. To explore how we better can fulfill this commitment, SALT is honored to present three highly respected leaders in legal education to share their perspectives on anti-racist values and practices in the recruitment and retention of faculty, staff, and administrators:

Tamara F. Lawson, Dean, St. Thomas University School of Law

Angela Onwuachi-Willig, Dean, Boston University School of Law

Sean M. Scott, President and Dean, California Western University School of Law

Please join SALT for this important event! You can register for this program here.

%d bloggers like this: