Let’s Take this Unprecedented period of Rapid Change to Consider What Can We Learn from the Delay of Fall On-Campus Interviewing: Further Thoughts on Equity and Inclusion

Posted on June 11, 2020 by jennifersbard

 

Jennifer S. Bard, Visiting Professor, University of Florida, Levin College of Law

In Tuesday’s post I suggested that we take the opportunity of dramatic, unexpected, and unwanted change delivered to legal education by the arrival of the Covid-19 virus and the need to rapidly revise decades, if not centuries, of conventions regarding grading and ranking that are tailored to the needs of a majority culture representative of the Harvard Law School Class of 1880 for whom it was created.

I also suggested that these historic grading conventions encouraged the persistent lack of diversity in our profession.

Today, I look at the opportunity that the likely delay of Fall (late summer) On-Campus Recruiting provides to evaluate one of the justifications for these grading practices which is the need to support our students’ ability to compete for the most high paying post-graduate jobs: employment at a “Big Law” firm (usually defined as one of about 200 firms employing more than 200 lawyers).  I take as a starting point the foundational point of this post: we don’t have the information we need to make good decisions about the consequences of the curricular conventions common to almost every U.S. law school.

We do know, however, that despite efforts (at varying levels of success) among law schools  to diversify their student bodies, this has had little effect on the diversification of the legal profession.  There is, moreover,  persistent evidence,  that discrimination plays a role in the interviewing practices of Big Law Firms and impacts the careers of those who are hired.  This could be intensified by the Covid-19 related slowdown of the economy.

What if our current  student grading and ranking practices are both 1) not in the best interests of the education of most of our students and 2) are playing an unwitting role in the lack of diversity in the legal profession by over-emphasizing early success?

The  first hint that a nearly universally adopted grading system based on sorting students for the benefit of Big Law firms may not be in anyone’s best interests is the absolute lack of uniformity in the role that Big Law plays in the employment of law students.  Consider, for a moment,  what we would think of a medical treatment that was administered to all patients but developed to help only 20% of patients or a restaurant that served food that would be appealing to only 20% of its customers–not much probably.  Yet despite serious efforts by many smart and caring people, the basic structure of legal education is exactly that.  81% of the roughly 190 (ties play a role in the count) law schools ranked by the otherwise defunct magazine, U.S. News & World Reports, send less than 20% of their graduates to Big Law firms. Only 9 law schools (all within the top 15) had 70% or more of their graduates employed at graduation with big law firms. Stick with this list of numbers, because it may surprise you.

The next 5 law schools had 60% of its students working in Big Law, 7 had 40%, 7 had 30%,  9 had 20%, 2 had 19 %, 4 had 18%, 6 had 17%, 6 had 16%, 5 had 15%, 2 had 14%, 4 had 13%, 2 had 12%, 11 had 11%. 10  had 10% and the rest-80 more law schools,-had less than 10% of their students employed by a Big Law firm at graduation.  Of that 80, thirty had less than 5% and 14 had zero.  Yet despite the vast differences in the likelihood that any student at any class rank will be hired by a Big Law Firm, almost every one of these schools has some form of grading curve and comparative ranking.

What would happen if most law schools took a step back-and just stopped?  What if they developed a grading system best suited to their educational goals of having the most students reach the highest possible level of legal competency?

The cynical answer is that Big Law would simply by-pass them in favor of the few that continued ranking.  But not so fast.  While there’s no basis to say that Big Law is unhappy with the pool they get from this practice, they certainly are aware that their hiring practices are very inefficient, and are giving increasing thought to how they might do better. Perhaps the pyramidal business model of today’s Big Law firm is an  historic accommodation to their hiring methods,  not a desirable outcome.  Also, current hiring methods are not resulting in the kind of diversity that their clients are asking them to achieve.  Indeed, many law firms, notably Holland & Knight, are working hard to increase diversity.

They also probably know how atypical their reliance on grades is among comparable organizations hiring graduate students. Kellogg Business School Professor and Sociologist Lauren Rivera’s book Pedigree recounts her research based on “embedding” herself in the hiring practices of law firms, banks, and consulting firms.  What she finds is not surprising—all three industries are more interested in the prestige of the graduate school than in the actual ability of any individual student.  But only law firms fail to incorporate any kind of competency based evaluation in the admissions process.   At least in part, this is because prestigious business school have long refused to even release grades to employers.  Thus, employers have had to develop an interview process that involves analyzing case studies, behavior based interviewing, and answering technical questions. We see similar retreats from grade based hiring in medical residency programs.

Would the tests that law firms themselves develop be any more equitable than the ones that, cumulatively, make up a GPA?  Maybe not.  But they could be more targeted toward what students learned in law school, rather than what they brought in with them.

So, given this opportunity for a pause in the hiring cycle and a freeze, for many schools, in the first year ranking process we could partner with our university collogues who conduct research in higher education, such as that on the curricular barriers to the success of underrepresented populations in STEM education, to see if what we are doing achieves the results we want.  And if not, to start the process of working with the legal profession to achieve something we both want: a diverse and equitably recruited cohort of lawyers who provide the highest possible quality of representation for their clients.

Filed under: Diversity & Social Justice, First Year Law School Innovations, Outcomes & Assessment Techniques | Tagged: , , , , | Comments Off on Let’s Take this Unprecedented period of Rapid Change to Consider What Can We Learn from the Delay of Fall On-Campus Interviewing: Further Thoughts on Equity and Inclusion

The role of law school internships and supervisors

Posted on August 9, 2019 by Mary A. Lynch

Today’s ABA Journal contains an op ed by a law student complaining that “law school  biases”  infringe on his right to free speech. Part of his critique involved a change in clinical policies after he wore a #BuildTheWall T-shirt to his internship.

“It had been expressed that we could wear T-shirts, and that has been the norm for my one year at this internship. I took extra precaution by bringing a light jacket to cover it up if a client came to meet with me unannounced.”

Others are better prepared than I to debate the issue generally of whether his claims demonstrate bias in higher education or bias on the part of the student. Others can ponder whether as educators, we are more apt to be triggered by exclusive versus inclusive messages since we value designing welcoming learning environments  and growth mindset .  However, I am not interested in this school’s particular behaviour or this student’s startling apparent nonchalance about how his clothing affected his colleagues, peers and the workplace.  Rather, I am more interested in developing a better understanding of the difference between an academic discussion about self-expression, and the responsibilities and possible repression of some self-expression that most lawyers and law students undergo when donning their professional role as legal interns do.

In my 30 years in clinical education, I have witnessed multiple instances of clinical faculty navigating the tricky balance in communicating professional norms, protecting clients and academic programs, and  respecting a student’s rights. Here are just a few issues we have addressed:

helping students without wealth obtain professional clothing

multicultural insensitivity to clients by both majority and minority students

student difficulty interacting with racist, homophobic and/or sexist, clients, judges, witnesses or opposing attorneys

Unlaundered clothes, smelly students

tight clothes (in men and women)

Clacking heels, scuffed shoes, or wearing clogs all day, every day, one’s whole life

Hair over eyes

dirty fingernails

evolving norms around piercing, black women’s hair, women wearing pants, more casual clothing, hair with color not found in nature

evolving norms around cell phones in local courts, e-mail

learning to use an ancient device called a telephone, to actually initiate a call or listen to voicemail

navigating support for transgender students in unwelcoming situations

drooping pants, belly showing, off the shoulder outfits, cleavage

loud talking, gum chewing,

informality in general which can appear as rudeness to supervisors

“distracting” jewelry

women students raising their voices in a question at the end of a sentence

…and I am sure you teachers can add many more. Feel free.

As a law professor steeped in clinical legal pedagogy and theory, I start the conversation with a few  questions:

  • what is the student’s “educational goal” for her academic/professional journey or experience
  • what is the student’s “lawyer goal” in the context of this internship, case or professional experience
  • what are the client’s/workplace’s needs and goals
  • what are the needs and goals of the community that supports you having this experience — the support staff, the court officers, your sister and fellow students, the local legal community (in this area I first must acknowledge my priorities and how current student behavior may close off opportunities for future students)

Then I discuss with the student how the student’s desired self-expression fits within those questions and priorities, and the possible disconnect from her goals and the programs.

This is my approach.  What do you do?

 

 

Filed under: academic freedom, Best Practices & Curriculum, Best Practices and Clinics, Best Practices, Outcomes & Assessment Techniques, Catalysts For Change, Diversity & Social Justice, Outcomes & Assessment Techniques, Teaching Methodology | Tagged: , , | 2 Comments »

A Fresh Look at the Uniform Bar Examination

Posted on January 22, 2019 by ottostockmeyer

The bar exam is back in the news. Later this month the ABA House of Delegates will consider a proposal to raise the bar-passage standard. The proposal would require that 75% of an accredited law school’s graduates pass a bar exam within two years of graduation.

In most states, bar exam means the National Conference of Bar Examiners’ Uniform Bar Exam. The UBE has been adopted by 35 jurisdictions and is under consideration in others. It carries the endorsement of many ABA-related entities. They include the Conference of Chief Justices, the Law Student and Law Practice Divisions, the Section of Legal Education and Admissions to the Bar, and the House of Delegates.

As a reminder of the features of the UBE, and an update on the pace of its adoption, I have accepted Mary Lynch’s gracious invitation to reblog my recent post “Will the Uniform Bar Exam Come to Michigan?” from the Western Michigan University-Cooley Law School blog. (Spoiler alert: no time soon.)

Filed under: Bar Exam, Best Practices, Outcomes & Assessment Techniques, Outcomes & Assessment Techniques, Uncategorized | Tagged: , , , | Comments Off on A Fresh Look at the Uniform Bar Examination

Leadership Education in Law School: You’re Already Providing It

Posted on January 16, 2019 by Paula Schaefer

Regardless of whether they think of themselves as leaders, our law students will play a leadership role for the rest of their lives. Certainly many will be leaders in their local legal community, in their law offices, and in various bar associations. But beyond that, all lawyers will be expected to lead outside of their law practices. As a lawyer (and sometimes the only lawyer) in their community group, family, or organization, they will be looked to for leadership.

Just as our students may not recognize themselves as leaders, we may not recognize ourselves as teachers of leadership. But we are. Most of our classes provide excellent opportunities to talk about leadership, even if “leadership” is not in the title. And we model leadership in how we treat our students and other members of the law school, how we contribute to and connect with our communities, and how we help move our law schools forward to address the changing profession.

Recognizing the growing interest in leadership education for lawyers, the AALS Section on Leadership was chartered in November 2017. The section describes its purpose as promoting “scholarship, teaching, and related activities that will help prepare lawyers and law students to serve in leadership roles.” This section is a great place to start for a law professor who wants to learn more about leadership education.

Law professors interested in getting some innovative ideas for integrating leadership-related topics into their classes should consider attending a workshop and roundtable at the University of Tennessee College of Law on April 4-5, 2019. The program is titled Leadership Development for Lawyers. The “workshop” day of the program will give attendees the chance to choose two of four interactive sessions: collaborating with career services; integrating well-being into leadership curricula; assessing leadership development efforts; and effective leadership skill development exercises. Then, the “roundtable” day of the program will provide an opportunity for conference attendees and panelists to share ideas and experiences in leadership education.

The goal of the Tennessee workshop and roundtable is to bring together a large group of legal educators who are working in the area of lawyer leadership education–including professors who don’t (currently) think of themselves as “leadership” teachers.

 

 

Filed under: Best Practices, Outcomes & Assessment Techniques, Disruptive Leadership, Outcomes & Assessment Techniques, Teaching Methodology | Tagged: , | Comments Off on Leadership Education in Law School: You’re Already Providing It

What can Law Schools Learn about Bar Passage from Medical Schools’ Approach to Studying Students Who Struggle with Licensing Exams?

Posted on November 28, 2018 by jennifersbard

It’s not unusual for a provost or a colleague or a relative at Thanksgiving to ask a legal academic why law students have so much trouble passing the bar exam when the pass rates for medical students are usually in the high 90th percent.  The short answer to that question is that the two processes are completely different—and there’s no obvious trick, technique, or intervention that could convert our bar passage rates into their licensure passage rates.   For one thing, it’s the wrong question.  “Passing” the medical licensing exams is certainly important, but unlike the “all or nothing” process of passing the bar exam, the score achieved on Step 1 affects medical students’ entire career path.  But there is a lot to learn about the methods that medical schools use in studying the very few students who have trouble as well as how they evaluate the effect of changes to their curriculums on scores on the licensing exams.

Quick recap on professional licensing—future doctors take a series of three exams over the first six years of their undergraduate medical education and the start of their residency.  (more links in a post I wrote earlier this year here).  The exams are almost entirely national although the actual process of being licensed is conducted on a state by state basis.   Law students take a licensing exam in the state where they intend to practice upon graduation.  For purposes of this post, the closest analogy to the bar exam is the more academic Step One students take during their second year of medical school.  Like  our NCBE, the National Board of Medical Examiners which produces United States Medical Licensing Examination works with medical licensing boards and depends on their confidence.  It issues annual reports.

The focus of this post is on the methods that medical schools use to study the small number of their students who do have trouble passing the licensing the exams as well as the factors that can affect the scores students achieve.  I’ve tried to focus on articles outside of paywalls, and would certainly encourage you to conduct your own searches in the various data bases to which you have access.  There are several journals devoted directly to studying medical education—although these articles can pop up anywhere.

Medical educators use a wide range of research techniques to learn more about students who struggle with licensure exams.  Like us, medical schools would prefer students pass the first time and many articles like this one look for characteristics who fail the first time but eventually pass.  Others look for characteristics of students at risk for failure here and here  or even  what students think of the exam.    Another area for inquiry involves the role stress plays in the score students achieve.   In partnership with social scientists at our schools or in our communities, we too could be conducting studies to help us learn more about students who face difficulty passing the bar exam.  These studies can be part of graduate student work or may even be funded by groups like Access which is making money available to study bar passage.

 

The actual reason the medical school pass rates are so high, though, may not be all that helpful.

It’s not just because they are able to limit admission to students who have already demonstrated an ability to score very highly on the MCAT.  A test that is much more similar to step 1 than the bar exam is to the LSAT.  Indeed, medical schools have direct input in both the MCAT and the Licensing Exams—so when one changes, the other can too. And it’s not clear that anything in the curriculum makes a difference at all—the industry offering study aids and licensure prep courses dwarfs the bar prep and study aid market to a point where students often start studying for the licensing exams before the first day of medical school.

But if it is the curriculum, it’s important to remember the vast difference in time scale between medical and legal education.  We have students for three years post B.A. Medical schools in the U.S. plan their curriculum based on  8 plus years of increasingly specialized medical education.  They are therefore comfortable holding off on the direct teaching of practice skills for the first two years while they are aligning their curriculum with the content of the Step 1 exam.

Even Step 1, though, is far more focused on practice than on knowledge accumulation or deliberately confusing question formulations that characterize the bar exam. Step 2,  the second round of licensing exams prior to graduation medical school,  go past paper and pencil in that they actually test students’ ability to conduct exams and exercise medical judgement.  Another reason for the high pass rate is that most medical schools have stopped developing their own tests and instead use assessment instruments (shelf exams) provided by the same company that produces the exam.   Sure, there is grumbling and criticism about content & timing of the licensing exams, but medical schools work hard to make sure that their curriculums are aligned with the content of the exams.  Finally, medical education is extremely self-reflecting–they are constantly aware of the risks that come from confusing correlation and causation.  How do you know that a change in one part of the curriculum is the cause of a change in test scores?  You run Pearson correlations followed by stepwise linear regressions.  Seeing is not believing when comes to identifying factors that affect performance on licensure exams.   Look here, here, here, and here for studies evaluating curriculum changes.  They take nothing for granted—does attendance make a difference, does flipping classrooms really work? Does reducing the number of hours spend in the anatomy lab reduce USMLE scores?

Another standard practice in medical schools is curriculum mapping— an essential first step for any school that wants to understand what they are teaching—let alone make changes.   Like all maps, curriculum maps are DESCRIPTIVE, not PROSCRIPTIVE.  Here is   Harvard’s curriculum map, but you can find examples on the home page of just about every U.S. Medical School.This is a an article walking through how to map a curriculum.

So what’s helpful to us isn’t so much what medical schools are doing, but how they are evaluating themselves. 

In recap, neither I nor anyone else who has ever practiced law thinks it would be a good idea to emulate medical schools by fully aligning our curriculum with the bar exam so as to turn the three years of law school into one extended bar prep course.  Among other reasons, the material tested on the bar is quite static and doesn’t reflect the realities of today’s law practice.   It also wouldn’t make much sense for schools whose students take the bar exam in many different jurisdictions.   Also, the bar exam is just not equivalent to the three rounds of USMLE exams in actually testing both the knowledge and application of knowledge needed to be a successful lawyer.  If it was, we wouldn’t hear so many complaints about how students who have passed bar are never-the-less not “practice ready.”

Tomorrow—where can we get the help we need to find out this information, and who is going to pay for it?  Spoiler--Access Lex has a program.

Filed under: Bar Exam, Best Practices, Outcomes & Assessment Techniques, First Year Law School Innovations, Outcomes & Assessment Techniques | Tagged: , , , | 7 Comments »

On the Value of Gap Years and Non-Legal Experience to Legal Employers (and Law Schools)

Posted on September 21, 2018 by Ben Bratman

Reviewing the results of the Foundations for Practice survey conducted by the Institute for the Advancement of the American Legal System (IAALS), I cannot help but note how the 24,000 responding attorneys ranked the helpfulness of various criteria for hiring beginning lawyers. (See here.) Just under 80 percent (78.3%, to be exact) identified “life experience between college and law school” as either a very helpful or somewhat helpful hiring criterion. Moreover, while “legal employment” (88.4% ranking as very or somewhat helpful) and “legal externship” (81.5%) unsurprisingly sat near the top of the list, “other experiential education” — meaning non-legal — was very close behind at 79.4%.

The responding attorneys, from a wide variety of practice areas and from throughout the country, ranked these two non-legal experience criteria — “life experience between college and law school” and “other experiential education” — as slightly more helpful than certain types of legal experience, including federal court clerkships, state court clerkships, and participation in law school clinics. The starker divide, however, came when considering traditional hiring criteria related to law school performance. While well over three quarters of respondents classified both personal and professional experience of a non-legal nature as helpful hiring criteria, only 62.5% said the same about law school class rank. Similarly, only 61.1% said so about law school attended, and merely 51.2% said so about law review experience.

One narrative coming out of the survey could indeed be that practical experience matters more than academic experience, and that seems to be what IAALS is highlighting. But, consistent with the results discussed above, I would suggest another as well: Non-legal experience — both personal and professional — matters almost as much, if not just as much, as legal experience.

The survey was just the first phrase of IAALS’s broader project, entitled Foundations for Practice, and the second phrase, which is being implemented now, directly implicates law schools. IAALS is working with four law schools to “translate the survey results into actionable learning outcomes and hiring rubrics.”

The current phrase focusing on law school outcomes turns my mind to the “incoming” side of law school admissions. The results of the survey suggest to me that law schools should more explicitly prioritize admission of students with meaningful life experience or non-legal professional experience. In addition, if it is not already, LSAC ought to be gathering and reporting to law schools pertinent data as to what percentage of law school applicants are undergraduate students who would be going directly to law school. And, as to those who are not, what are the percentages one year out from the undergraduate degree, two years, three or more, etc.? Just as law schools view national statistics on other important admissions criteria (GPA, LSAT, ethnic diversity, to mention a few) as important benchmarks, they ought to be in a position to do the same for number of years since undergraduate degree.

Having a significant percentage of students with meaningful life experience outside of the law is indisputably of great benefit to the law school learning environment. I see it every year in my classroom. More to the point of the IAALS survey results, by bringing in a significant number of students with such experience, law schools will be contributing to better outcomes — learning outcomes and employment outcomes. In a typical incoming J.D. class at my home school, the University of Pittsburgh School of Law, 20% to 30% of the students are three or more years removed from their undergraduate degree. We do not have a part-time or night program, and certainly those schools that do will have higher percentages of that demographic.

At least one-third and in some years close to one-half of the students in our typical incoming class are coming straight out of their undergraduate studies without even one gap year. This demographic exists at nearly every law school in the country (in varying percentages). Given what we know about the next generation of law students, and given the importance of life experience and non-legal experience as hiring criteria to today’s legal employers, these students would seem to face a more challenging path. What do law schools need to do, if anything? Offer or enhance existing professional development programming or curricula? Offer or enhance existing experiential opportunities that are not exclusively legal in nature and that expose students to non-lawyers and other disciplines and experiences? (Just two examples: teaching or working with high school students, or working with entrepreneurs at a tech startup. Law schools affiliated with a university can offer assorted interdisciplinary educational opportunities as well.)  I will be interested to see if the second phrase of the IAALS project emphasizes ideas like these or others that respond to the demonstrated need for lawyers with life experience and non-legal professional experience.

Filed under: Best Practices & Externships, Best Practices & Setting Goals, Best Practices, Outcomes & Assessment Techniques, Outcomes & Assessment Techniques, Uncategorized | Comments Off on On the Value of Gap Years and Non-Legal Experience to Legal Employers (and Law Schools)

What inspires the scenarios and characters in your final exam questions?

Posted on December 26, 2017 by Paula Schaefer

As we wrap up another season of grading, I return to the thought that grading finals can feel like reading the same story again and again. This task is slightly more entertaining for me if the story involves some interesting characters or scenarios. Here are a few places I look for inspiration when I write final exams.

  1. Real Cases. Sometimes, a case in the news serves as inspiration for a final exam. That happened this fall when my PR final posed a question involving a lawyer who solicited clients in a funeral home in a state where he was not licensed. Other times, I work backwards and pick an issue I want to address in my final (like Rule 19 in civil procedure) and then find a case involving that issue. (For the Rule 19 case, I once used a scenario based on Diaz v. Glen Plaid in which the defendant asserted that the University of Alabama was an indispensable party in a case involving the trademark-protected image of a houndstooth elephant).
  2. TV Lawyers. The set-up for my essay question is often a memo from a lawyer asking a junior lawyer to help with a client’s problem. I often base that senior lawyer’s name on a tv lawyer. Through the years, those attorneys have included Alicia Florrick, Ally McBeal, Jimmy McGill, Kim Wexler, and many others. The facts have nothing to do with these lawyers or their tv shows. The names are really just for my personal amusement.
  3. Other Characters from TV and the Movies. Beyond tv lawyers, I sometimes look to other tv shows and movies for inspiration for scenarios and character names. My civil procedure exam once described a federal lawsuit arising from a bowling accident involving characters from The Big Lebowski. Knowledge of the movie does not help exam performance, but often inspires a joke (perhaps something about a rug that really tied the room together) that makes exam grading easier for a moment. I have learned not to make the scenarios sound too much like something that might be happening on the actual show. (During the show’s heyday, a student complained I had included “spoilers” in an exam question involving Nashville. I assured her that the scenario was just my imagination and that I had not spoiled anything she was planning to watch on DVR once finals were over).
  4. People I Know.  Even if I have the scenario, it is hard to come up with the multitude of character names needed for a three-hour exam. I tend to return again and again to the names of people I know. Most of my exams include character names inspired by my childhood neighbors, elementary school classmates, and law school friends. (I finally admitted this to my law school friends and the conversation quickly turned to how much worse it is to take a law school exam than to write or grade it. I did not try to win that fight).  My civil procedure exam typically includes a character named after my own civ pro professor.
  5. People My Students Know. Finally, another source of character names is people that my students know: their own law professors. I would never use my colleagues’ names in a scandalous scenario, but rather in a (mildly) funny scenario that the students will appreciate. For example, a multiple choice question on my civil procedure exam described my students’ contracts professor suing me for breach of contract.

In truth, reading dozens of exams involving these characters does not make the month of December “fun” (or make it feel like the “vacation” that my mom thinks I get at this time each year).  But it helps a little.

Filed under: Outcomes & Assessment Techniques, Teaching Methodology, Uncategorized | 1 Comment »

Experience with Peer Support, Peer Review and Feedback on Teaching?  

Posted on September 25, 2017 by Mary A. Lynch

We are all familiar with engagement in peer review of scholarship. Law faculty culture prioritizes peer input and review of scholarly ideas and articles. Sending drafts of articles to colleagues for feedback, “workshopping” preliminary ideas, and vetting scholarship is part and parcel of the work we do. We visit other schools, make presentations and attend conferences because we value peer discussion and  input. It is the basis by which we create and communicate knowledge.

I don’t believe, however, we have a similarly pervasive culture for formative peer review when it comes to teaching in law schools, although such culture exists at other higher education institutions. According to The University of Texas Faculty Innovation Center, an academic culture which prioritizes informed peer collaboration, review and input on teaching benefits everyone,

Good teachers continually learn and develop. Peer Review, which combines the examination of course materials with in-class observations and collegial discussion, helps prompt this learning among faculty. Ideally, these interactions and conversations can create opportunities for us as colleagues to reflect on and adapt our teaching practices in order to become better teachers and increase student learning.

Northeastern University Center for Advancing Teaching and Learning through Research recommends a four step process:

  • Initial conversation between the observer and the observed
  • The observation itself as an informal data collection and distillation process
  • Follow-up conversation in which the observer shares the observations and collaborates with the observed teacher in any kind of brainstorming or troubleshooting that the observations invite.
  • Reflective summary written by the observed instructor, integrating what was learned from the process and how this will influence future teaching.

Vanderbilt University’s Center for Teaching includes the goal of “enabling more intentional and mutually supportive communities of scholar teachers.”

It is true that we have made some progress in elevating the role of teaching in law schools in the past decade. Legal Education certainly woke up to the need for a culture change around curriculum and teaching following the publication of Best Practices for Legal Education  and Educating Lawyers.  The economic downturn heavily affected the admission process and the need to focus on student learning. ABA requirements regarding student learning outcomes also redirected attention and resources towards what students actually learn while in law school. Moreover, organized efforts such as the Institute for Law Teaching and Learning  and the AALS Section on Teaching Methods  have converted many to the idea that teaching and learning are matters worthy of scholarship, innovation and peer discussion.  Places like this blog and others support exchange of ideas, methods and innovations.

It is also true that as far back as 2008, pioneering legal scholars Gerry Hess and Sophie Sparrow studied factors which encourage or assist the professional development of law teachers including peer observation. So there are many resources available to improve teaching in law schools. Yet, across the academy, are we truly immersed in a continual process of formative feedback for law teachers? If so, the web shows little evidence of it.

I think some of the culture gap is explained by the fact that historically peer review of teaching only happened during a promotion and tenure process that resulted in an up or out decision by the faculty — hardly a formative approach. A voluntary formative program of peer support and review – not used for personnel decisions – should allay those fears.  Appropriate concerns about interference with academic freedom in the classroom might explain some of the culture gap. Except that, even more concerns about academic freedom arise with respect to peer input into “controversial scholarship,” since draft writings can be more easily captured and reproduced than can observations of a single class session. What I think explains the gap, instead, is that we have not properly trained or equipped law faculty with the tools and methods for conducting and receiving helpful peer observations.

At Albany Law, we have promoted a culture of inquiry around teaching and learning for many years now — colleagues sit in each others classrooms from time to time, our Academic Dean prioritizes teaching support, our Center for Excellence in Law Teaching showcases teaching ideas and invites collegial discussion through teaching workshops, and our Director of Online Learning and Instructional Technology facilitates flipped classrooms and other innovations. What we haven’t done is formalize a voluntary peer support and review program. This year, we are planning to revisit our very loose approach and learn from the ever evolving resources and experimentation of others.

So readers, contributors and chance internet searchers, please post here what if any processes have you implemented to support peer observation of law teaching? Is it a voluntary program as we envision at Albany? How has it worked? Or, if you have an opinion about faculty peer review programs, let us know what you think!

I hope to compile the results and report back later in the year!

P.S. If you are more comfortable with e-mail than a blog comment, feel free to contact me at mlync@albanylaw.edu. 

Filed under: academic freedom, Best Practices, Best Practices for Institutional Effectiveness, Best Practices, Outcomes & Assessment Techniques, Catalysts For Change, Outcomes & Assessment Techniques, Teaching Methodology | Tagged: , , , , , , , , | 6 Comments »

Law Schools Going Beyond Learning Outcomes Mandated by ABA

Posted on March 30, 2017 by benjmad

Having taken part in two recent symposia on learning outcomes (PLOs) in legal education, I was encouraged to see the number of law schools that are taking advantage of the recognized pedagogical benefits of adopting and assessing learning outcomes. As most law professors now know, ABA Standards require the adoption of learning outcomes. These standards also mandate programmatic assessment of whether students are attaining these outcomes. ABA Standard 302 dictates certain PLOs that all schools must adopt (e.g., knowledge of substantive and procedural law, legal analysis, research, and writing skills.) However, I saw evidence at each symposium that schools are going beyond the mandatory PLOs and are shaping their learning outcomes for knowledge, skills, and values beyond the minimum. That phenomenon suggest schools recognize the pedagogical value of outcome-based education and are seeking to provide more than the minimum.

The first symposium was entitled “The Next Steps of a Professional Formation Social Movement,” at St. Thomas School of Law on February 16-18–https://www.stthomas.edu/law/events/ symposium-21717.html.  One of the primary themes of the conference was that between thirty and forty law schools had adopted learning outcome that incorporated professional formation, consistent with the third apprenticeship advocated by the Carnegie Institute’s Educating Lawyers. Because ABA Standard 302 does not require such learning outcomes, the efforts of a growing number of schools to include them show a recognition of the significance of Carnegie’s emphasis on the need to do a better job of helping law student to develop a professional identity as they learn doctrine and lawyering skills. The conference explored professional formation learning outcomes in medical and military education and suggested potential points of comparison to law teaching, the conference further reported new data suggesting that the growing professional formation movement is consistent with the goals of law students. Finally, participants formed working groups to continue with the work necessary to keep the momentum going for the role of professional identity formation in legal education. In short, the symposium demonstrated the steady increase of faculty and schools advocating for integration of professional identity formation into the legal curriculum. See http://beyondtherule.blogspot.com/2017/ 02/cefler-cosponsors-symposium-on.html. The results of the symposium will appear in St. Thomas Law Journal’s upcoming symposium issue.

The University of Detroit-Mercy Law Review also hosted a symposium, on March 2, 2017, which reviewed the impact of learning outcomes and assessment—both institutional assessment of the degree to which students attain the outcomes law schools state as objectives, and more creative assessment in law school classes in the form of both formative and multiple summative assessments — http://www.udetmercylrev.com/symposium/outcome-measure-legal-education-symposium. The symposium highlighted again PLOs being adopted by a wide range of schools that exceed the minimum of ABA Standard 302.   The message of such a response to the advent of learning outcomes in legal education seems to be clear: law schools are willing to use this proven method of ensuring educational quality to improve their programs, not just in the least possible way but in a manner that will help law students achieve the most from their time in school.

These are but some examples of a broader movement in legal education improve pedagogy not only in the classroom (e.g., more formative assessments) but throughout the program (institutional reforms). Despite fears of widespread recalcitrance, a substantial number of law schools appear to be making a genuine effort to improve their programs.

Filed under: Best Practices & Curriculum, Best Practices, Outcomes & Assessment Techniques, Outcomes & Assessment Techniques, Teaching Methodology, Uncategorized | 1 Comment »

Competencies-Based Legal Education

Posted on March 29, 2016 by Michele Pistone

[This was originally posted by the Clayton Christensen Institute on Disruptive Innovation]

 Last week, I discussed why law schools need to respond to the changing marketplace for legal services and legal education.  In thinking about how best to prepare for that changing world, law schools need to consider how competency-based educational models can be employed to advance educational objectives for students seeking to enter the market for legal services.  As Michael Horn and I explain in our new whitepaper, Disrupting Law School, regulatory protections that have sheltered law schools from competition will continue to subside.  In this new environment, law schools need to reimagine themselves as educators for students interested in learning about the legal services sector, not simply those seeking a JD.

One way to do this is to think about legal education from a blank slate.  Rather that try to retrofit our current pedagogy to address 21st century needs, instead we need to think about it from its inception — if one were to start a school today to educate those who want a career in the legal services field, what would that school look like?

Upstart competency-based education programs have done just that in other parts of higher education.  They provide at least three new considerations for traditional law school as they begin to think about and prepare for the future.

1. Time is no longer the measure of accomplishment

Online competency-based learning reverses the traditional relationship in education between time and student learning. In the traditional educational model, time is fixed while each student’s learning is variable. With online competency-based learning, the relationship between time and learning is reversed — time becomes the variable and each student’s learning becomes essentially fixed. Students process at their own pace, moving from topic to topic upon mastery of each. Those who need more time to master a concept before moving on to the next take the time they need, while others move ahead to the next set of material and learning objectives.

2. Centrality of competencies, learning outcomes, and assessments

Online competency-based programs shift the teaching pedagogy toward student-centered learning. In an online, competency-based program, faculty and instructional designers start by identifying the competencies students must master to achieve the desired learning outcomes and then work through each to understand how a student would demonstrate mastery of those objectives. Through constant feedback, students know how they are doing and what they need to do next and teachers can determine when students have mastered competencies and are ready to move forward. The assessments in other words are both forward looking—assessments that help determine what a student studies next—and backward looking —assessments that indicate whether a student has mastered the material.

3.  Modularization of course material provides more flexibility and different business models

Online competency-based learning is also changing key elements of the traditional higher education business model. Online technologies make it possible to modularize the learning process—that is, to break usual semester-long courses into shorter learning units or modules, which can be studied in sequence or separately. When material is packaged in online modules, it is easier to use for multiple educational purposes and multiple audiences in different combinations.

Stackable modules allow students to create individualized curricula based on their own learning goals and objectives. For students who attend law school knowing the area of law in which they want to practice—a segment of the student body currently underserved due to limited course offerings in any one topic at any one law school—modules open up opportunities to stack credentials from multiple sources. The long tail of the Internet opens up these opportunities; there may be sufficient student demand if online courses can aggregate demand and serve students from around the country or even the world.

Modules also eliminate duplication and optimize teaching resources. This flexible architecture can create an entirely new business model for law-related education. When learning is broken down into competencies—rather than semester-long courses—modules of learning can be packaged into different scalable programs for very different audiences—for example, paralegals, legal technicians, law students, lawyers (CLE), judges, administrative agencies, non-JDs working in law-related fields, foreign students, high school/college moot court teams, undergraduate students, journalists, clients, life-long learners, and so forth.  The possibilities abound.

This exercise can take us in a lot of different directions.  Every direction, though, will ask us to change and move beyond the status quo.  While change is hard, it is also necessary.  I hope our whitepaper provides sufficient impetus to get started.

Filed under: Best Practices & Curriculum, Catalysts For Change, Outcomes & Assessment Techniques, Teaching Methodology, Technology, Uncategorized | Tagged: , , , , , , , , | Comments Off on Competencies-Based Legal Education

The 25 Most Important Lawyering Skills?

Posted on October 8, 2015 by Ben Bratman

In discussing bar exam reform in my earlier post, I referenced the results of this job analysis survey of newly licensed attorneys. The attorneys, all in practice for three years or less, were asked to rate the significance to their jobs of various skills or abilities (e.g., legal reasoning, organizational skills, written communication) and various knowledge domains (e.g., Rules of Evidence, Contract Law, Rules of Civil Procedure). Ever since I first saw the results, I have been taken with one particular statistic: The respondents rated 25 different skills or abilities as more significant to their jobs than the highest rated knowledge domain.

After the results came out, I looked more closely at these 25 skills and organized them into five broader skill categories. (My chart, which includes all 25 skills and each one’s average rating on a scale of 1 to 4, is below.) I then led a discussion on the importance of all of this to legal education at a legal writing conference last spring. Some of the colleagues in attendance offered insightful and practical comments that I’d like to share here.

One suggested that the 25 skills are a good starting point for formulating a new course to satisfy the ABA’s expanded practical skills requirement in the new Standard 303(a)(3). Others suggested that my chart, or something akin to it, could be a means for identifying and measuring learning outcomes for “other professional skills needed for competent and ethical participation as a member of the legal profession” under Standard 302(d), or additional learning outcomes under Interpretation 302-2.

I hope that many in legal education will find this chart, my colleagues’ ideas, and the overall survey results to be valuable tools. And, if anyone has feedback on how to revise the chart to make it a more useful tool, please get in touch.

Communication Analysis Research Project Management Professionalism
Written communication 3.77 Critical reading & comprehension 3.55 Computer skills 3.28 Paying attention to details 3.67 Professionalism 3.58
Listening 3.60 Synthesizing facts & law 3.55 Electronic researching 3.26 Using office technologies 3.56 Judgment 3.29
Oral communication 3.58 Legal reasoning 3.54 Fact gathering & evaluation 3.22 Knowing when to go back & ask ?s 3.46 Diligence 3.26
Interpersonal skills 3.44 Issue spotting 3.43 Organizational skills
3.46
Answering questions succinctly 3.30 Information integrating 3.10 Working within established time constraints 3.44  
Advocacy 3.24 Decisiveness 3.31
Consciousness of limitations 3.15
Planning & strategizing 3.13

 

Filed under: Best Practices, Outcomes & Assessment Techniques | 4 Comments »

Building on Best Practices now available as eBook

Posted on July 9, 2015 by dmaranville

Are you trying to:

  • Develop a meaningful law school mission statement?
  • Understand new accreditation requirements, learning goals, and outcomes assessment?
  •  Expand your experiential offerings?  Decide whether to use modules or courses?  An on-site clinic, an externship, or community partnership?
  •  Teach ALL of your students in the most effective ways, using a full range of teaching methods?
  • Add to your curriculum more of the professional identity, leadership, intercultural, inter-professional and other knowledge, skills, and values sought by 21st century legal employers?
  • Lead thoughtfully in the face of the challenges facing legal education today?

These and other topics are addressed in Building on Best Practices:  Transforming Legal Education in a Changing World,  now available in ebook format from LexisNexis at no charge.

The print version is not yet out.  LEXIS-NEXIS is taking advance orders for $50, plus shipping.  BUT we understand that they will make one copy available to every US legal educator for free upon on request.  Details on this and international availability still to come.

Thanks, and congratulations, to book project sponsor Clinical Legal Education Association (CLEA), the more than fifty legal educators who participated as authors, and the countless others who assisted as readers and in numerous other ways.

And, a huge shout-out to my wonderful and talented co-editors, Lisa Radke Bliss, Carrie Wilkes Kaas, and Antoinette Sedillo Lopez.

Filed under: Best Practices, Best Practices & Curriculum, Best Practices & Externships, Best Practices & Setting Goals, Best Practices and Clinics, Best Practices for Institutional Effectiveness, Best Practices, Diversity & Social Justice, Best Practices, Outcomes & Assessment Techniques, Catalysts For Change, Diversity & Social Justice, First Year Law School Innovations, International Initiatives and Models, Outcomes & Assessment Techniques | Tagged: , , , , , , , , , , , , , , , , | 2 Comments »

A Survey Instrument for Cultural Sensibility Learning Outcomes

Posted on May 4, 2015 by andicurcio

As law schools begin to grapple with identifying and measuring law student learning outcomes, cultural sensibility [a.k.a. cultural competence] should be on the learning outcomes list. A validated survey instrument has been developed to help measure some aspects of cultural sensibility learning: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2451300.  The instrument helps measure students’ understanding that we all have multifaceted cultural backgrounds, experiences, and biases that affect how we perceive and analyze legal problems and how we interact with clients and colleagues.

As lawyers, we must recognize and grapple with our own biases and stereotypes, as well as the influence cultural factors and systemic racism have had, and continue to have, upon the US legal system. As I note in a forthcoming Nevada Law Journal article: “While racial categories are artificial constructs, there is a long and ongoing history of real differences in the treatment and, therefore, collective experiences of “racial” groups. Those experiences influence how we perceive and assess facts, attitudes, legal problems and legal processes.”

An integral part of legal education involves developing law students’ abilities to identify their own cultural biases and helping future lawyers understand how those cultural perspectives and biases impact their legal analyses and interactions. There are many learning outcomes that contribute to law students’ cultural sensibility knowledge, attitudes and skills, many of which may be measured in various experiential learning and doctrinal courses.

The survey instrument measures some over-arching cultural sensibility learning outcomes, such as recognizing that: 1. one’s own cultural experiences affect how one views the legal system; 2. legal training in “rational thinking” does not insulate lawyers and judges from our own cultural biases; 3. subconscious cognitive processes hinder our ability to identify when we are acting based upon biases and stereotypes, and 4. we need to withhold judgment about others’ behaviors.

The survey instrument may be administered to students as they enter law school and shortly before they graduate. While we did not administer the survey to the same cohort of law students as they entered and then graduated, we did administer it to 309 entering law students and 281 upper level students. Amongst those students, we found that upper level students had a better understanding that one’s own cultural experiences affect how one views legal problems and interacts with clients. To the extent that cultural sensibility education requires that baseline understanding, the survey instrument is one way to measure some aspects of cultural sensibility learning.

At this June’s AALS Workshop on Measuring Learning Gains, Professor Raquel Aldana and I will continue the dialogue on how else one might measure cultural sensibility learning outcomes across the curriculum.

Filed under: Best Practices, Diversity & Social Justice, Outcomes & Assessment Techniques | 1 Comment »