Unmasking Assumptions about Employment Outcomes and Legal Education

In an upcoming Wisconsin Law Review article, Robert Kuehn, Associate Dean for Clinical Education and Professor of Law at the Washington University Law School, presents a cogent, well-supported and thoughtful article describing the limitations of and lessons we can learn from the existing empirical analysis correlating student enrollment in clinical education and employment outcomes.  Kuehn’s article, entitled Measuring Legal Education’s Employment Outcomes is particularly powerful because it provides a thorough empirical rejection of the claim that clinical coursework might actually harm employment outcomes, as asserted by Professor Jason Yackee and which attracted some sound-bite attention earlier this year. In what is, perhaps,  an unexpected twist, Kuehn demonstrates that using Yackee’s statistical assumptions and methodology also would produce negative correlations for those students who participate on law journals or in moot court competitions.  Kuehn argues that one can’t draw any reliable conclusion from Yackee’s 2013 model, and perhaps not from any nationwide statistical model – as opposed to a particularized analysis of one school –  on the likely effect of clinical courses (or other activities like law journal or moot court) on employment, and surely not the negative effect Yackee posits. Kuehn points out that as to clinical coursework, the available evidence (through surveys) indicates that such experiences do aid some students in securing employment.

If you, like me, still become a bit nervous about how much you actually remember from undergraduate statistics courses, do not be alarmed by this post!  You will find Kuehn’s article accessible and a quick good read, even when he is using words like “regression analysis,” “granular data” and “variable choices.”   Here are the points made in Measuring Legal Education’s Employment Outcomes which I found most helpful:

  1. Kuehn’s reminder that when one confuses correlationwith causation one is bound to come up with a “misdiagnosis.” One problem with Yackee’s analysis is the lack of granular data to calculate the true employment rate for those who took a clinic (or who did not).  In fact, the data is so poor that “the results never account for more than half of the variability in employment across schools.”
  2. Kuehn’s explanation of the “confounding effect of prestige” and bar passage on employment outcomes.
  3. The problems of validity and reliability raised by analyses which employ information from ABA questionnaires, particularly those self-reports submitted prior to 2014.
  4. The fact that “13% of law schools” provide 80% of the school-funded jobs to law graduates. Not surprisingly, Kuehn found this factor biases many results if you examine nationwide statistics. And when Kuehn removes those jobs from the statistical analysis, Yackee’s correlation with clinical education falls apart even using his own assumptions and methodology.
  5. Yackee’s model yields completely different results if one uses the US News Lawyers/judges data versus academic peer data to control for the possible influence of perceived prestige.
  6. Application of Yackee’s model to “Law Journals” and “Skills Competition” and S. Newssub-groups also show no relationship to employment outcomes!
  7. In Yackee’s model, a better ranking is “strongly associated with improved employment outcomes.” However, Kuehn points out that a “closer examination of the relationship between rank and employment indicates that this positive association, although statistically significant when applied across the entire range of top 100 schools, does not hold true for schools ranked 51 through 100 (emphasis added).” 
  8. Kuehn’s documentation of employers who require, “strongly prefer” or identify law clinic experience as a positive factor in hiring such as The U.S. Department of Homeland, legal services and  legal aid offices, district attorney, public defender, fellowships and private law firms.
  9. Kuehn’s description of National Association of Law Placement (NALP) existing information: such as the  2011 survey of lawyers with non-profit and government offices;  the NALP survey of lawyers in firms of predominantly more than 100 attorneys; the NALP survey of public interest legal employers;  and the NALP 2013 presentation on the employment market reporting that ” law firms say they want new graduates to have ‘more experiential learning, client-based and simulation.”
  10. Kuehn provision of good information on other employer information such as the Lexis-Nexis WHITE PAPER: HIRING PARTNERS REVEAL NEW ATTORNEY READINESS FOR REAL WORLD PRACTICEProfessor Neil Hamilton’s employer survey to determine the relative importance of twenty-one different competencies in employer hiring decisions, and Professor Susan Wawrose’s legal employer focus groups which found employers prefer new hires with ” well developed professional or ‘soft skills” along with “strong fundamental practice skills.”

Professor Kuehn concludes by recommending that studies could best be done on a school-by-school basis by “surveying likely employers to find out what educational experiences of students are most valued.”  Professor Kuehn also recommends that schools could also “retrospectively look at various employment outcomes for graduates and any relationship” to students’ experiences while in school.

I agree with Professor Kuehn and am happy to report that  Albany Law School,  through its faculty Assessment committee and Admissions office,  is currently engaged in conducting employer focus groups and analyzing what best helps our students obtain employment in their desired career paths.  Until good data and information suggests otherwise, Professor Neil  Hamilton’s advice to law students,which Professor Kuehn quotes in his “must read” article, bears repeating:

In this challenging market for employment, a law student can differentiate herself from other graduates by demonstrating to legal employers that the student both understands the core competencies that legal employers and clients want and is implementing a plan to develop these competencies, including an ability to demonstrate that the student has experience with these competencies.

Experiential Learning: ABA Standards 303 and 304

Although “experiential learning” has been a term of interest to many legal educators for years, the ABA’s new standards have brought it front and center by mandating that schools “require[] each student to satisfactorily complete at least . . . one or more experiential course(s) totaling at least six credit hours.”

Two well-known ways to meet the requirement are law clinics and field placements. Law schools across the country already widely offer both. But the third path to satisfying the requirement, “simulation courses,” is less familiar to many professors. So what do “simulation courses” require?

Standards 303 and 304 together define what a course must include to be a “simulation course.” First, and likely most importantly, the course must be “primarily experiential in nature.” According to a March 2015 ABA Guidance Memo on these standards, that means the course is essentially, mostly, or chiefly experiential and “the substantive law or doctrinal material . . . [is] incidental to [the course].”  The latter element is an important limitation: it could mean, depending on how the ABA interprets the standard, that even courses rich with simulation exercises are not a “simulation course” if the course is using simulations primarily to teach doctrinal law. In fact, the same guidance memo notes that “[i]nserting skills components in otherwise doctrinal courses” is great, but it does not necessarily mean the course falls under the “simulation course” umbrella. It would be unfortunate if highly experiential doctrinal courses along the lines of “The Practice of [Specific Doctrinal Area]” did not qualify under these standards simply because they were teaching students skills relevant to a particular area of the law.

Next, the course must provide “substantial experience not involving an actual client that is reasonably similar to the experience of a lawyer advising or representing a client or engaging in other lawyering tasks.” While many skills-based courses will meet this element—for example, courses focusing on interviewing clients, drafting legal documents, and mediation—other courses are more debatable. Is a course teaching students how to run a practice, manage cases, track time and bill, and related pragmatic skills “reasonably similar to the experience of a lawyer” under the standard? It is unclear, but a good argument can be made that it is—those skills fall under Standard 302‘s explanation of what skills a law school curriculum should foster, e.g. “organization and management of legal work” (interpretation 302-1).

The remaining requirements are more straightforward. The substantial experience must occur within “a set of facts and circumstances devised or adopted by a faculty member,” and there must be multiple opportunities for performance, feedback from a faculty member, and self-evaluation. Most courses that clear the above hurdles will naturally meet these. Similarly, the requirement that a faculty member directly supervise each student’s performance will also be met in most cases.

Finally, there must be “a classroom instructional component,” the class must integrate doctrine, theory, skills, and legal ethics, and the class must “engage students in performance of one or more of the professional skills identified in Standard 302” (e.g.  counseling, negotiation, trial practice, and many other professional skills). These requirements are relatively straight-forward but will pose a problem for some for-credit co-curriculars like moot court and trial team, which often do not require a classroom component beyond the team practices (which the guidance memo indicates will generally not be enough).  Interested parties can learn more in the guidance memo about how such programs can qualify for the experiential credits.

In sum, the revised ABA standards are breathing new life into experiential learning across the nation. Combined with the prospect of state-specific experiential learning requirements, mandatory experiential learning is here to stay. However, with attention to Standards 303 and 304 and study of the guidance memo, simulation courses are a third option for the required six credits.

Annual Leadership in Legal Education Issue of Univ. of Toledo Law Review Filled with Best Practices Nuggets

The new issue of the University Toledo Law Review is out, featuring its annual “virtual symposium” on legal education by law school deans. These annual issues should be read not just be deans and people who are thinking about pursuing a law school deanship, but they should be read by college and university presidents and provosts, members of law school boards of trustees and advisory boards, senior administrative staff, and most important, by law school faculty. The articles in each volume, taken together, offer terrific insights into current challenges facing legal education, interesting historical background on various aspects of legal education, and innovative ideas to shape the future of law schools and legal education. The winter 2015 volume is no exception.

While I will not address all twelve of the articles/essays in this brief review, I do want to highlight several important themes in four pieces. Beginning with the opening contribution by two-time former dean Peter C. Alexander (Indiana Tech and Southern Illinois), more than mere references to “best practices” principles abound. One of Alexander’s assertions is that law schools, in “the new normal” must do more to create “practice ready” graduates as part of the ongoing curricular reform taking place. He also suggests, “Faculty members have to design new methods of instruction and create new pathways for students to learn….Deans must make funds available for faculty members to learn how people learn and how to teach the current generation of students.” (p. 263) This is an astute observation and one not lost on many in the academy. Most of us on the law faculty did not receive any formal education or degree in pedagogy. While those who work with students from pre-K through 12th grade must be certified as teachers after formal baccalaureate and post-baccalaureate training, there are no such requirements in higher education. Few, if any, dispute that in law school the learning styles of our students has changed over time, and this challenges law faculty to more attune to the need to change our teaching methodologies.

Another piece written by Professor George Critchlow, former interim dean and former director of the clinical programs at Gonzaga University School of Law, focuses on ensuring that legal education in a broad sense is accessible to those who wish to serve the public good – including non-lawyers (a good and controversial read). In his discussion on affordability, Critchlow reviews a number of ideas that have been circulating for years including, but not limited to: law schools partnering with legal services organizations and firms (resembling aspects of the medical school model); a discretionary third year program that consists entirely of a practice-oriented experience; participation by law schools with apprenticeship programs that allow or encourage students to engage in actual work outside of the law school in addition to classes (this goes well beyond the current law school supervised externship and clinic experiences); and cost savings to clinical programs by entering into “hybrid” arrangements with community based legal service providers.

A theme in Critchlow’s article is picked up in greater detail in an article by IIT Chicago-Kent College of Law dean Harold J. Krent and director of clinical legal education Gary S. Laser. Krent and Laser focus on meeting the experiential challenge through the operation of a fee-generating law clinic. By highlighting the example of the IIT Chicago-Kent model which in essence is organized as an in-house law office, the authors point out that students are exposed not just to the traditional live client experience of a clinic, but they develop an appreciation for the economics of law practice. This is important given the increasing attention that many law schools are giving to the business aspects of running law offices, whether it be through the incubator movement, the addition of courses on law office management, and the introduction of business skills to the curriculum.

The symposium ends with an essay by UC Hastings College of Law Dean Frank Wu which I highly recommend everyone read. Dean Wu offers his prescription for reforming law schools, much of which I will not address here due to space and my focus on best practice. Wu states, “A lawyer should be like a doctor. There is no medical school graduate who altogether lacks clinical experience. Every licensed physician has seen a live patient presenting actual symptoms before charging anyone for a diagnosis. Yet some law school graduates manage to do quite well by book learning alone. They need not interview, counsel, or draft, to earn honors, if their exams and seminar papers are good enough.” (p. 420) He discusses the increasing importance of the need for the academy and the profession to understand and appreciate the impact that technology is having and will have on the future of the practice of law and lawmaking. Wu addresses the ongoing and long-time debate over the profile of law professors as practitioners or intellectuals. (p. 440) In addressing the costs of change, Dean Wu asserts that the most expensive and most worthwhile change we have “recently” made in legal education is clinical legal education.

Every year I find fascinating the articles and essays published by the Toledo Law Review in their special “deans” issue. I am surprised that many people do not know that this annual symposium exists. It is a good read that should not be missed.

What’s going on in California? “TFARR- recommended” 15 credits of competency training

For those who did not closely follow the California State Bar debate on the requirement of 15 credits of competency training for bar admission (the work of the Task Force on Admissions Regulation Reform, or “TFARR”), I summarize the current status.  (Although I am currently co-prez of the Clinical Legal Education Association, known as CLEA, this post is not written with that hat on.)  This is my own thinking, albeit, informed by the excellent work of the CLEA Advocacy committee.

The TFARR process was two-staged, over a three year period, with opportunities for public comment throughout. CLEA  participated in that process and submitted five separate comments on the proposals that are available at http://www.cleaweb.org/advocacy under “Briefs and Other Advocacy” (documents 4-8).

In the end, TFARR recommended 15 credits of competency training which can be achieved in a variety of ways (in addition to how experiential credits can be earned under the new ABA regulations), and which include six credits of summer work. You can read the TFARR Phase II Final Report  at: http://www.calbar.ca.gov/AboutUs/PublicComment/Archives/2014PublicComment/201411.aspx

The process was complete in November, 2014, with final TFARR recommendations to the State Bar Board of Trustees (that responded to public comments) and unanimous adoption by the Board: http://board.calbar.ca.gov/Agenda.aspx?id=10891&tid=0&show=100008800&s=true#10013881 (agenda item 113). The TFARR Phase II FInal Report represents a compromise based on extensive input.

Lately, some confusion has arisen because of a letter posted to the AALS website authored by a non-standing committee of Deans.  The confusion arises because:

  1. Neither AALS nor this special Dean’s committee ever participated in the two stage TFARR process and so appear to be sort of “johnny come latelys, ” and
  2. The letter mistakenly focuses on an earlier draft of the final proposal failing to recognize the compromises already reached in the final proposal.

I understand that there are efforts underway to correct the confusion which makes me happy since the Deans’ letter is signed by two people whom I have long admired in a variety of contexts.

Other blogs are already exploring the 15 credit  proposal and its interesting and creative approach. For example,   “Kudos to California”  What do our readers think?

Gender and Lawyers’ Worklives

As we think about how to improve legal education, it’s always helpful to understand  our students, their careers, and what they value.   UW Law reference librarian Mary Whisner shared this item, that I missed when it initially came out:

Harvard Study: Women Lawyers Work More Than Men,

Bloomberg BNA Big Law Business, May 12, 2015

Harvard Law School’s Center on the Legal Profession released the results of a widespread survey of its graduates which suggests women work more hours on average than men, among other potentially myth-busting findings.

Through a survey of HLS graduates from the classes of 1975, 1985, 1995 and 2000 and other research, it provides a detailed portrait of the gender gap within the legal profession, including all the ways women have advanced or failed to advance.

. . .

https://bol.bna.com/harvard-study-women-lawyers-work-more-than-men/

The study also finds the women graduates satisfied  with the substance of their work, but dissatisfied with their compensation, while the reverse is true for men.

The full study (86 pp.) is David B. Wilkins et al., The Women and Men of Harvard Law School: Preliminary Results from the HLS Career Study (2015).

Would these findings about Harvard Law grads would hold true for lawyers generally.  If so, are there any implications for legal education?

Here’s one speculation:  Perhaps men experience more cultural push towards financial security and success in the form of work in Big Law. They might also experience less cultural encouragement toward emotional self-awareness, introspection about purpose in life,  and a service orientation. If so, the result might be that more men focus on external motivations and pursue the Big Law path,  even when it’s a bad fit with their interests, skills and values. They then find themselves less satisfied with the substance of their work. (And, given gender myths about women’s lesser commitment to the workforce, the men might be able to meet expectations with fewer hours.)  If so, legal education would be well advised to improve efforts to help students develop their professional identify, focusing both  on developing students understanding of lawyers work in different settings, and on students’ own talents, interests and values.

Another speculation:  Perhaps women tend to be less confident about the quality of their work and log more hours as a result.  Legal education could help them appreciate their own talents and skill level.

Other thoughts?

What Makes Your Subject Distinctive?

As law schools continue to develop their learning outcomes, an important question we all should consider is, “what makes my course distinctive?”  For example, in my research on assessment in legal research courses, I was struck by how much the analytical and problem solving skills developed by legal research instruction are the same as those developed by many other courses in the law school curriculum.  That led me to ask, “what makes legal research instruction distinctive?”  The answer was not simply, as an outsider might suggest, that legal research classes teach tools for finding law (digests, Westlaw, etc.).  Rather, I was struck that legal research instruction is distinctive in the extent to which an effective legal researcher must have an appreciation for the power of taxonomies, must exercise imagination in the context of realistic boundaries of time, cost, and purpose, must be able to ask for help, and must develop strong metacognitive practices (to continually question “is this process working?”).  The difference is of degree rather than kind of course, but it is a distinctive difference nonetheless.

Given the narrow focus of legal education, it seems that this question of distinctiveness or “value added” is the most critical question I can ask in planning my courses.  Not that the distinctive outcomes of my courses should be the sole, or even dominant outcomes.  Legal education outcomes require an iterative process and cross-curricular experiences for students to become competent and to enable transfer of learning to new settings.  Yet, understanding what makes my outcomes distinctive forces me to justify my outcomes and consider their connections with other law school outcomes.

So what makes my outcomes in Professional Responsibility distinctive?  Certainly the identity of the anticipated uses of the doctrine we are learning leads me to choose to emphasize professional identity formation outcomes as important if not distinctive.  In most law school courses, students are learning the law to serve others and are encouraged to use, interpret, and advocate about the law to achieve a client’s objectives.  In Professional Responsibility, the students will be using the law to advise themselves.  My outcomes include expecting that students will be able to clarify their observational standpoint when considering issues of professional ethics; recognize that self interest clouds judgment and ways to gain more objectivity; and differentiate the approaches to interpretation of law that one might use to advocate for a client regarding past conduct from approaches that are wise, ethical, and effective when interpreting the law to guide our own future conduct.  Finding effective methods to assess students development of these perspective is a challenge but I have found that simply asking students to read cases of attorney discipline and ask, “what went wrong with the attorney’s thinking?” is a good place to start.

What makes your course outcomes distinctive?  How has that led to distinctive assessment practices?

Building on Best Practices now available as eBook

Are you trying to:

  • Develop a meaningful law school mission statement?
  • Understand new accreditation requirements, learning goals, and outcomes assessment?
  •  Expand your experiential offerings?  Decide whether to use modules or courses?  An on-site clinic, an externship, or community partnership?
  •  Teach ALL of your students in the most effective ways, using a full range of teaching methods?
  • Add to your curriculum more of the professional identity, leadership, intercultural, inter-professional and other knowledge, skills, and values sought by 21st century legal employers?
  • Lead thoughtfully in the face of the challenges facing legal education today?

These and other topics are addressed in Building on Best Practices:  Transforming Legal Education in a Changing World,  now available in ebook format from LexisNexis at no charge.

The print version is not yet out.  LEXIS-NEXIS is taking advance orders for $50, plus shipping.  BUT we understand that they will make one copy available to every US legal educator for free upon on request.  Details on this and international availability still to come.

Thanks, and congratulations, to book project sponsor Clinical Legal Education Association (CLEA), the more than fifty legal educators who participated as authors, and the countless others who assisted as readers and in numerous other ways.

And, a huge shout-out to my wonderful and talented co-editors, Lisa Radke Bliss, Carrie Wilkes Kaas, and Antoinette Sedillo Lopez.

Law School Curriculum Review & Reform: Lessons Learned

In 2012, my dean asked me to chair a review our curriculum at the University of Tennessee College of Law. He asked our committee to consider the current three-year curriculum in light of our learning outcomes. It sounded like an overwhelming job.

During the first year of our curriculum review, I remember reading the book Reforming Legal Education: Law Schools at the Crossroads. Michael Hunter Schwartz and Jeremiah Ho wrote a great chapter titled Curriculum Reforms at Washburn University School of Law.

I would describe Schwartz and Ho’s chapter in two ways: (1) full of practical suggestions about the process for reviewing curriculum and considering reform; and (2) terrifying.

I stole many of the practical, process-related suggestions from their chapter. We had a committee retreat where we spent an uninterrupted day discussing the curriculum. Committee members went door-to-door and talked to each faculty member about the curriculum and possible changes. The committee developed two proposals for curriculum reform and discussed these proposals with the full faculty. I am sure there are other ideas we borrowed from Schwartz and Ho.

The terrifying part of Schwartz and Ho’s chapter was this line: “[O]ne might conclude that, after nearly three years of work, Washburn’s curriculum reform efforts have been unsuccessful.”

Three years? We may do this for three years and feel it wasn’t a success?

Of course, Schwartz and Ho go on to explain that there were successes in the three-year process. (The Washburn faculty reached a consensus on key issues and made progress toward some important goals detailed in the chapter). But it was daunting for me to think that the process would be difficult and might take three years.

In 2015, the University of Tennessee College of Law faculty adopted a package of significant changes to the 1L curriculum. While the substance of those changes is important, I think it is also important to contribute to Schwartz and Ho’s discussion about the process. So here are a few of the lessons I learned about the process of curriculum review and reform over the past three years.

1) Three Years is a Good Start. When we started, three years sounded like a long time to work on a curriculum review. I now know that three years of curriculum review passes in the blink of an eye. We needed that much time to understand our curriculum, talk to faculty, alumni and students, research what was happening elsewhere, create proposals for change, seek more input, and generate new proposals.

2) Less is More. Our committee accomplished something in three years because we narrowed the focus. Even though our original committee charge was to review the entire curriculum, we ended up focusing on the first year curriculum. That was a more manageable project. Also related to “less is more,” after two years we realized the committee was spread too thin. Our dean originally gave the curriculum review charge to the Academic Standards & Curriculum Committee. For two years that committee juggled the curriculum review and the regular business of Academic Standards. In the third year, our dean created a separate task force to focus solely on the curriculum review. That change made us much more efficient in year three and allowed us to reach a faculty vote on a package of proposals.

3) Seek Input from Faculty, Alumni, and Students Multiple Times, in Multiple Settings. Throughout the three years of our curriculum review, we talked to faculty, alumni, and students. When we met with alumni and students, we gave them the chance to address the room, answer questions anonymously (with clickers), and respond in writing to questions. We often continued these discussions on the phone, by email, and in person. We were able to compile all of this input and share it with the faculty. The committee spent even more time gathering ideas from the faculty in one-on-one meetings, in multiple forums, in small group sessions, and in many informal conversations over the course of three years. Seeking input in all of these settings helped us learn from all of our stakeholders and resulted in a variety of suggestions.

4) Compromise Can Lead to Something Better. Near the end of our second year of the curriculum review, the committee presented the faculty with two packages of possible reforms to the 1L curriculum. Discussing and debating the merits of these proposals helped the committee see potential problems we had missed and opportunities for meaningful change. With that information, we met with small groups of faculty to generate ideas about new classes and other innovations. In these meetings, members of the faculty often suggested they wanted to take the lead in making a change or teaching something new. As the third year came to a close, the faculty approved a package of 1L curriculum changes that was substantially better than what the committee had suggested at the end of year two.

5) Curriculum Review “Success.” Three years ago, it was unnerving to read that Schwartz and Ho thought we might not find curriculum reform “success” in three years. But I now know that is a good thing. Curriculum review and reform does not have to be perfect, because we are never done. Curriculum review should be an ongoing process. This allows us to identify what is working and determine what we will do next as we prepare students for practice.

A 21st Century “generalist legal education”? Skills & professional identity focused.

More musings on generalist v. specialist education, and how much doctrine law schools need to teach.

A conversation with one of our University of Washington alums — Leo Flor, Westpoint grad, Gates Public Service Law Scholar, Equal Justice Works Fellow at Northwest Justice Project, and spark plug for the new resource Representing Washington Veterans  — has me chewing on whether we need a new understanding of what a “generalist legal education” means.

Leo noted that the JD is often viewed as a relevant generalist credential, even though most law school grads move into traditional bar-passage-required “law practice” jobs.  And he observed that many job postings for alternative positions list an MBA or MPA as a relevant qualification, but not the JD.

The traditional generalist education of my era, and to a significant extent still, was intended to teach a set of analytical skills and and expose students to a broad range of legal doctrine potentially relevant to a general practitioner and to passing the bar exam. Though passing the bar remains important and is a significant factor in designing the educational program for lower tier schools, few 21st century lawyers are truly general practitioners.

Perhaps the generalist foundation needed in this era is built on skills, more than doctrinal knowledge.  And for Leo’s purpose not only skills in a technician sense.  Skills also in a “professional identity” sense.  Self-awareness & understanding of ones’ own gifts.  Leadership and interpersonal skills. Such an understanding of generalist could make the JD an appropriate credential for the types of job Leo described.

In a previous post, I suggested that that, at least for those students who come to law school with significant self-knowledge and experience, a substantively specialized curriculum could make sense, if combined with the general analytical and research skills to learn new areas.  This is not a new  idea.  Back in 2002 then-law-student Kevin E. Houchen self-published a detailed review of the trend toward certificate programs and concentrations, arguing that for a subset of students such specialization makes sense.

A decade later in 2012 the New York Times touted  NYU’s  limited moves toward greater specialization not just once, but again in an article  promoting specialization for law schools focused on Biglaw.

And in early May of this year 2015 at the National Summit on Innovation in Legal Services sponsored by the ABA and Stanford Law School, speakers reiterated these themes.   Richard Susskind (13:08) argued that legal education needs to train graduates for 21sth century jobs like legal project managers, legal process analysts, legal knowledge engineers, and legal risk management.  Prof. Deborah Rhode(13.29) observed that it “makes no sense to train Wall Street M &A lawyers the same way we train someone who’s going to be doing routine real estate and divorce work in a small town.”

It is not so very difficult to understand what acting on Prof. Rhode’s observation might mean.  As a practical matter, some curricular differentiation based on where graduates will practice already takes place, linked primarily to  different levels in the law school hierarchy.  Beyond that, many schools offer an extensive enough curriculum – both in doctrine and skills — to permit considerable specialization aimed at traditional law practice niches, even beyond formal concentration tracks and certificate programs.

For more specialized training law schools that offer extensive LL.M. programs routinely allow students to complete an accelerated JD/LL.M in three calendar years.  In the tax field, where the LL.M. has long been de rigueur, many schools provide such opportunities, including NYU, the long-time leader in tax LL.M’s.  And increasingly, schools educate students not just in substantive tax specialties, but also — using my own school, the University of Washington as an example — with  tax-focused skills and clinical training for both JD. and LL.M. students.

My hunch is that increasing numbers of students already opt to specialize, sometimes with a substantive law focus, often combined with a skills focus, e.g.  corporate deals with drafting or criminal & tort law with trial advocacy.

Richard Susskind’s challenge is a bigger stretch for legal education, though, again, some initiatives are visible, such as Michigan State‘s Reinvent Law Laboratory.

A key challenge for law schools is to learn how to identify prospective students or develop admitted ones who understand their life goals and values, and their intellectual and personal gifts well enough to make intelligent decisions around specialization.  To meet that  challenge, a holistic approach to education is needed –whether understood in the MacCrate framework of knowledge,  skills and values, or the Carnegie framework of cognitive, professional skills and ethical professional identity apprenticeships.

Inner Development, Community, Social Justice (Concurrent Session, AALS Conference on Clinical Legal Education)

Last, but not least, in this series highlighting lessons from experts in other disciplines relevant to how to navigate the chaotic “new normal”  in legal education: Thursday’s concurrent session organized by Tennessee’s Paulette Williams:  “A Commitment to Inner Development: Connecting the “New Normal” with Clinics’ Social Justice Mission”.

The session brought  Edward Groody and Timothy Dempsey from the Community Building Institute in Tennessee.  The Institute helps social service and criminal justice organizations become more effective by training participants in community building practices.  Taking an evidence-based approach built on motivational interviewing, trauma-informed care, and pro-social supports, community building is a “highly experiential process that helps participants remove barriers to communication and unlearn unproductive attitudes and behaviors.”

Groody began the session with a detailed overview of a four-stage process for building community:

  • Pseudo-community
  • Chaos
  • Emptying/Letting Go
  • Community

That process adds an important step — emptying/letting go — to Bruce Tuckman’s familiar “forming, storming, norming, performing” model of group formation.  My own interpretation of this additional,  third step is that it provides space for  participants to recognize,  and learn skills to address, the emotional issues that so often get in the way of honest connection with others.

Dempsey then shared powerful stories of how that process helps ex-offenders with post-prison re-entry,  allowing them to move past behavioral responses that may have seemed — and perhaps were — functional in their previous lives, but would block their efforts to move forward.   Or, to put it another way, this step acknowledges that in order to take advantage of education or employment opportunities, people need to let go of fears, resentments or trauma.  This is challenging work that is the foundation of many spiritual traditions, but can help build strong connections with others.

Time constraints prevented Paulette Williams from speaking in detail about how she makes use of this process in her clinical teaching work.  I hope she finds other forums for sharing those experiences and insights.

The insights of this community building process struck me as relevant not only to social justice and clinical legal education work, but also to faculty interactions within our law schools.  From another time and place, I well remember a year when every faculty meeting resulted in controversy, usually about something relatively minor that seemed to be a proxy for other, larger, but unacknowledged issues festering beneath the surface.    I suspect that many faculties are experiencing something similar as they operate  in the  current climate of uncertainty and change, too often getting stuck in the fear those conditions foster.  It’s  difficult for me to imagine applying this model in the typical law school environment.  But successfully moving through the “emptying/letting go” phase, as individuals and a group,  could be oh, so helpful!

Lessons from “Counseling Our Students” (Mini-Plenary at AALS Conference on Clinical Education)

At the recent AALS Conference on Clinical Education two additional sessions provided important insights from experts iin other disciplines on how to operate effectively in the midst of the current period of change in legal education.

Wednesday;s Mini-Plenary on Counseling Our Students In the New Normal included an inspiring guest speaker who was even more impressive as a listener.

Moderated by Mercer’s Tim Floyd, the session began with a helpful overview of the current state of the job market (bottom line:  recovering, slowly) by Abraham Pollack, GW’s  Professional Development dean. But the centerpiece of the session was Carolyn McKanders, Co-Director and Director or Organizational Culture, Thinking Collaborative and, not incidentally, mother of Tennessee’s Karla McKanders,

Carolyn brilliantly demonstrated “cognitive coaching” (check out the app!) in an unscripted coaching session that allowed Mary Lynch (yes, that Mary Lynch,  Editor of this blog) to expand  her acting career into improv. The session was designed to help Mary think through her goals and approaches in counseling students on career development in an environment where predictable and linear career tracks are no longer the norm.

After the role play Carolyn summarized three keys to cognitive coaching:  pausing, paraphrasing and posing questions (with a rising inflection that communicates curiosity and openness, not control or credibility).  The beauty of this approach is that it helps the individual “self-monitor, self-analyze, and self-evaluate“.

The session certainly reinforced three lessons that clinicians should know; after all, a foundational goal of clinical legal education is fostering reflection, and most of us teach interviewing and counseling, at least to some extent.

  • First, the power of listening.  In a world of fast talking, sometimes monologue-happy, often living-in-our-heads law professors, so easy for this lesson to “go missing”  if we ruminate worriedly, trying to cope with the new normal in faculty and committee meetings and informal conversations.
  • Second, the value of paraphrasing for understanding to ensure accurate communication.
  • And finally, the importance of  founding our questions on authentic curiosity — listening in order to understand, not to counter an argument.

In a constantly changing world, where so many verities are in play, it’s too easy for us to get stuck in fear and suspicion.  Though the stated rationale for the mini-plenary was to help us counsel students, for me it spoke at least as powerfully to how we can most effectively interact  with our colleagues.  And, perhaps, “counsel” ourselves.

In the next, and final post of this series, I’ll discuss a Thursday concurrent that linked “inner development” with community building and social justice.

Birth, Maturity, Creative Destruction & Renewal At AALS Clinical Conference

As someone who collaborated on a concurrent session titled “Facing Our Fears in Changing Times” at the AALS Conference on Clinical Legal Education, it’s probably not surprising that I was especially drawn to sessions that brought in models or speakers from other disciplines to provide insight on how to operate effectively in the midst of the current period of change in legal education.

In addition to my last post on Michele Weise’s Closing Plenary, in this and my next two posts, I’ll discuss three other provocative sessions that addressed different aspects of this theme.

On Tuesday morning my University of Washington colleagues Jennifer Fan and Lisa Kelly, worked with Rutgers-Newark’s Randi Mandelbaum and Syracuse’s Mary Helen McNeal to introduce the “liberating structures eco-systems model” of leadership.  That model views organizational change as an  infinity loop in which organizations move through four cycles that call for different styles of leadership:

Stage                                                   Leadership Style

Birth                                                     Entrepreneur

Maturity                                                Manager

Creative Destruction                           Heretic

Renewal                                               Networker

The model suggests that embedded in the cycle are two “traps“:

1. Between the Maturity and Creative Destruction stages lies the Rigidity Trap of “not letting go” of what the organization has birthed and brought to maturity.  Staying stuck in the past and wedded to the old ways of doing things.

2. Between Creative Destruction and Renewal lies the Poverty Trap of “not investing enough to accomplish renewal”.

Sound familiar? The session included an exercise where attendees decided which stage  they perceived their individual clinic, program, institution, or the clinical legal education movement to be in.  Participants  then added on the infinity loop diagram post-its with their results.  Although responses were spread around the loop, most clustered  among Maturity — Creative Destruction — and Renewal.  Most responses addressed clinical programs and law schools.

I find this framework a helpful reminder that our current struggles are “normal” and that they won’t last forever.  And inspiration to let go of fears and rigidity.

I’m grateful to my former colleague Tim Jaasko-Fisher for his work with liberating structures in the Court Improvement Academy of UW Law’s Children and Youth Advocacy Clinic.

Disruptive Innovation & the AALS Clinical Conference

One of the highlights of last week’s AALS Conference for Clinical Law Teachers was the closing talk by Michele Weise, Senior Fellow, Education at the Clay Christensen Institute for Disruptive Innovation. (A big shout out to Michele Pistone for her role in making that talk happen!) I was superficially familiar with the  disruptive innovation thesis, but Weise’s half-hour talk brought to life its relevance to the current moment in legal education in a way that previous exposure had not. Disruptive innovations that shake up a market or industry often follow a predictable pattern, it is argued. The established players in the market target a higher end client base and compete on quality, improving the product and selling it at a high margin.  This leaves a significant, low-end segment of the market unserved. New entrants provide an inferior product to these unserved consumers, and gradually improve the product and expand their market.  Poof go the established players. Think personal computers, print media, digital cameras, mobile phones . . . . Traditional higher education has long failed to reach a significant segment of potential consumers and the federal government’s shift from financial aid grants to student loans has greatly exacerbated that problem. Arguably, the stage is set for disruptive innovation and on-line technology may be the means to that disruption. The next step of Weise’s analysis was what really captured my attention. She noted that higher education currently serves many functions – transmission of content and certification of knowledge or skills; providing a safe space for young adults to mature socially; networking opportunities, mentoring and tutoring; research & dissemination of scholarship. These functions can be – and are being – disaggregated and provided more cheaply on line. Even the Harvards of the world are potentially at risk, according to Weise. Law schools have traditionally provided a generalist education.  As legal practice becomes more specialized, that educational model arguably serves to mask more specialized functions that could be disaggregated.  This is already being tried in my home state of Washington with our new Limited Licensed Legal Technician (aka/ Triple LT) program.  But lawyers also wouldn’t have to be trained as generalists.  As course offerings expand, the potential for moving away from the traditional generalist education does also.  Already,  this shows up in the transcripts of some of my students who are not necessarily taking the doctrinal courses that were considered foundational in my day.  Does this matter? Before hearing Weise’s talk, during the Law Clinic Directors Workshop, I raised the question “how much doctrine do we need to teach?” Good lawyers, I observed,  have extensive doctrinal knowledge.  (Of course, law schools historically haven’t taught doctrine in connection with the experiential anchor points that many of us need in order to retrieve that knowledge for practice.)  Elliott Milstein later challenged the importance of doctrinal knowledge,  observing that his clinic students handle their cases well regardless of whether they have taken relevant doctrinal courses.  Often true.  And yet . . .  The counter-example that I didn’t have a chance to share:  one of my  students  recognized that we could challenge a new unemployment compensation statute on the ground that the subject was not properly included in the title of the legislation.  A classic case of issue spotting that came about solely because he was taking a Washington State Constitutional Law course.  (I didn’t recognize the issue.) A reminder that the ability to issue spot is valuable.  But  . . . state constitutional law isn’t a classic “foundational” “bar course”. This issue spotting was strictly serendipity – a traditional doctrinally-focused course load would not have accomplished this result. I’m still struggling with the generalist/specialist question.  But it leaves me thinking about the potential for niche curricular innovation aimed at students – often older ones who understand their talents, passions and life goals – who come to law school with a commitment to a practice area like criminal law, immigration law, or business law.

  • Are there enough of those students to justify a legal education targeted at those niches?
  • If so, can we focus their education in a way that really prepares them for their specialty?
  • And, can we at the same time identify a “sweet spot” of “just enough” generalist knowledge to accompany that specialization?  One that provides a foundation for passing the bar exam and the analytical and research skills to master new areas of the law, but does not take up the bulk of a three year curriculum?

I don’t know the answer to these questions.  But they strike me as worth investigating.

Building on Best Practices for Legal Education Manuscript Submitted to Publisher

Four editors,  59 authors, 92 readers, three copy editors, librarians from two schools, a secretary, miscellaneous consultants, three student assistants for bluebooking, and one for setting up perrmacc links.*

Many people, occasionally in multiple roles, were needed to produce the manuscript sent to Lexis last Monday for the forthcoming book Deborah Maranville, Lisa Radtke Bliss, Carolyn Wilkes Kaas, and Antoinette Sedillo López (eds.),  Building on Best Practices:  Transforming Legal Education in a Changing World. (Lexis 2015).  A monster project — but, as I assured a friend, no, not a manuscript about monsters and not monstrously unpleasant to produce – just big, ambitious, and sometimes exhausting for the editors and authors.  A big thank you to all who participated!

The book is a follow up to CLEA’s Best Practices for Legal Education, the 2007 volume by Roy Stuckey and others that inspired this blog.  Like Best Practices, this book will be distributed for free to legal educators.  Lexis has promised to make it available in electronic format through their e-book library and to provide print copies on request.  Look for it in four to six months — if all goes smoothly perhaps in time for the AALS Clinical Legal Education Conference in early May.

The coverage of Building on Best Practices is wide-ranging.  To quote from the Introduction, “[t]his volume builds on the call to link mission and outcomes; emphasizing the themes of integrating theory, doctrine and practice, developing the broader spectrum of skills needed by lawyers in the twenty-first century, and taking up the question how best to shift law school cultures to facilitate change.”

Advance praise for the book has included:

  • “[M]ilestone in legal education . . . that legal educators will rely on as much as . . . on the first Best Practices book.”  (Patty Roberts, William & Mary)
  • “Educational for folks who don’t know much about experiential education and insightful for those who do. . . .Really something to be proud of . . . an invaluable resource to schools as they go to work on implementing the ABA’s new requirements for learning outcomes and assessment. . .The perfect product coming out at the perfect time.” (Kate Kruse, Hamline)

Once again, CLEA deserves kudos for its support of an important scholarly project on legal education.  And the Georgia State University, University of New Mexico, Quinnipiac University, and University of Washington Law Schools deserve a big round of thanks for supporting the co-editors in this project.

https://perma.cc/ provides an archive for those annoying website links that quickly become outdated.

Five Problems to Avoid in Writing Student Learning Outcomes

As law faculty across the country strive to improve student learning and meet ABA standards of accreditation through the assessment process, it is perhaps appropriate to stop and assess our efforts in that regard.  Here are five common problems that occur when first writing learning outcomes for a course:

1. Don’t focus on you – focus on the students
Student learning outcomes are designed to give students an idea of what they will be learning.  Avoid learning outcomes that describe what or how your will teach and instead focus on what the students will be able to know, do, or believe.

NOT: UMKC457  Trees as Thought
Student learning outcome:  In this course, I will be exploring the philosophical thought experiment “If a tree falls in a forest and no one is around to hear it, does it make a sound?”  I will explain my book “Trees as Focal Points for Reality” and refute critics of the proposals presented therein.

BETTER: UMKC457  Thought Experiments
Student learning outcome:  At the end of this course, students will be able to think critically and communicate effectively the metaphysical theories regarding the existence of that which cannot be perceived. Students will be able to describe how the theory of subjective idealism has impacted religious and scientific philosophy.  Through discussion and written reflection, students will demonstrate clarification of their individual values.

2. Avoid Vague Verbs
Probably one of the most common verbs found in student learning outcomes is “understand,” as in “students will understand [course content].” The problem with this as a learning outcome is that it is difficult to know what evidence would demonstrate that understanding.  A student learning outcome that uses more active and concrete verbs can unpack the type and degree of “understanding” that a professor expects.

NOT:   LAW8000  Family Law
Student learning outcome:  Students will understand the law regarding marriage regulation and the constitutional constraints on that regulation and the law of divorce, including child custody.

BETTER:  LAW 8000 Family Law
Student learning outcome: At the end of this course, students will be able to:
• identify the legal issues raised by a fact pattern involving a marriage regulation, make critical and effective arguments regarding the meaning of that regulation and its constitutional validity, and confidently predict the outcome of a challenge to that regulation
• identify relevant facts necessary to gather from a client seeking a divorce and child custody with property including real estate and pensions; draft a complete and legally effective petition for that divorce and custody action, including a parenting plan; and identify legal issues and make critical and effective arguments, applying the statutory and case law, to determine the divorce, property division, child custody and economic support in the case.
To read more about it, see Chapter Two. Understanding Understanding, of GRANT WIGGINS & JAY MCTIGHE, UNDERSTANDING BY DESIGN (2nd Ed. 2005).

3. Avoid “elementitis”
A student learning outcome should not merely summarize the syllabus or be a list of topics the course will cover.  Rather, the student learning outcomes should focus on thematic elements that tie these topics together or ways in which the students will be able to use this knowledge.  As David Perkins of the Harvard Graduate School of Education notes:
We educators always face the challenge of helping our students approach complex skills and ideas. So what to do? The two most familiar strategies are learning by elements and learning about. In the elements approach, we break down the topic or skill into elements and teach them separately, putting off the whole game until later — often much later….to have a little fun I call it ‘elementitis.’
DAVID PERKINS, MAKING LEARNING WHOLE: HOW SEVEN PRINCIPLES OF TEACHING CAN TRANSFORM EDUCATION (2010).  Avoid student learning outcomes that are plagues by “elementitis” and describe instead what it is students will be able to do with course coverage.

4. Don’t Always Expect Mastery
Student learning outcomes should indicate not only the content the students will learn but how well they will learn it.  We cannot aim for mastery of all aspects of the course.  Rather, learning outcomes in some courses are necessarily going to be at an introductory level (students will “recognize” or “describe” or “identify”) while other outcomes may be aimed at higher levels of mastery.  An effective tool to determine the proficiency level of your learning outcomes is Bloom’s Taxonomy of Educational Objectives, which provides a hierarchy of increasingly sophisticated learning outcomes.  To read more about it and see a list of verbs associated with differing levels of learning, see Rex Heer, A Model of Learning Objectives from Iowa State University Center for Excellence in Learning & Teaching (2012). To read an application of this model to law school, see Paul Callister, Time to Blossom: An Inquiry into Bloom’s Taxonomy as a Hierarchy and Means for Teaching Legal Research Skills 202:2 LAW LIBRARY JOURNAL 191 (2010-12).

5. Don’t Avoid Outcomes that May be Difficult to Measure
Student learning outcomes for a classroom rarely will focus entirely on the acquisition of knowledge.  At a minimum, most classes expect students to develop their cognitive and communication skills in using the knowledge base of the course.  Courses may also help students to clarify values, reconsider beliefs, appreciate new perspectives, or develop greater self-awareness.  Some faculty recognize that these skills and values are some of the most important benefits that students take away from the courses, but are reluctant to state these as learning outcomes because they are unable to “test” these outcomes.  However, any important skill or value can be assessed – even if there is a good deal of subjectivity involved in that assessment.  By stating these objectives as learning outcomes, faculty members can challenge themselves and their students to more clearly describe the dimensions of this learning.  Measurements of this learning may be through written reflections, observations of performance, or surveys of opinions.  These are perfectly valid assessment tools.

%d bloggers like this: