Best Practice Contributors Highlighted in Best Articles of 2019

A big congratulations to our very own bloggers, Jennifer Bard and Benjamin Madison, for being featured on the TaxProf Blog!

Jennifer Bard’s article, “Are the Students Failing the Bar Exam Today Canaries in the Coal Mine warning us of a More General Need to Change Legal Education?” and Benjamin Madison’s article, “New Rubrics Available to Help Law Schools that Have Adopted Learning Outcomes Related to Professional Identity Formation” were both listed as TaxProf Blog’s “Best Legal Education Articles of 2019.”

Refuting the False Trope on Clinical Courses and Bar Passage

Robert Kuehn, Washington University School of Law

It has been observed that “the fewer the facts, the stronger the opinion.” Until recently, this could be said about the possible influence of enrollment in clinical courses on a student’s likelihood of passing the bar examination. While there was a shortage of empirical studies on any possible relationship, there have been plenty of opinions on how taking those courses might be harmful, opinions often reflected in graduation restrictions on clinical courses and requirements for bar subject-matter courses.

But, there are now significantly more facts to refute those opinions. Two recent, large-scale studies have both found no relationship between the number of law clinic or externship courses or credits a law graduate took and her likelihood of passing the bar exam.

In a forthcoming article in the Journal of Legal Education, academic records of ten years of law school graduates of Washington University in St. Louis and Wayne State University were reviewed for any relationship between the number of law clinic, externship, or, more generally, experiential courses or credits and bar passage. After first accounting for the possible influence of law school grades on bar passage (the most significant predictor of bar success), the study found no correlation at either school between law clinic or externship enrollment and bar passage — no relationship between participation in a law clinic or externship and passage, none between the number of clinical courses and passage, none between the number of clinical credits and passage, and no evidence that students graduating with lower GPAs were disproportionately enrolling in those courses as a way to avoid doctrinal courses (another not uncommon trope). This lack of any relationship was in spite of increased enrollment in experiential courses at both schools over the ten-year period and decreased enrollment in courses teaching material tested on the bar (referred to as bar subject-matter courses).

The article notes that nationwide data on experiential course enrollment and bar passage also belie any claim the two are related. That data indicate that as enrollment in experiential courses was increasing from 2006-2016, bar passage percentages were fairly steady and that the recent decline in passage coincided with decreased, not increased, enrollment in those courses.

A recent study commissioned by the California State Bar found a similar lack of relationship between law clinic and externship courses and bar exam performance. The study reviewed law school coursework and performance on three July exams for over 7,500 bar applicants from eleven California schools. It found no relationship between the number of academic credits from law clinic courses and exam performance, either across all schools or even when reviewing schools separately. Similarly, there was no relationship between the number of externship or internship credits and performance, again when examined across all schools or within schools. The broad range of entering credentials at the eleven schools, and lack of a relationship even within those schools, indicates that the results should be applicable to most law schools, including those with lower LSATs and undergraduate GPAs for entering students.

The study results from Washington University/Wayne State and the California State Bar are similar to smaller studies at Texas Tech and the University of Denver that also reported no statistically significant relationship between enrollment in a law clinic or externship course and bar passage.

The Washington University/Wayne State and California State Bar studies further revealed that opinions about the value of bar subject-matter courses should be moderated. There were small correlations at both schools between the number of bar subject courses and bar passage. But this result (explaining less than 5% of the variability in bar outcomes) was only for low performing students and additional courses showed no marginal benefit once students took the school’s average number of bar courses.

The California State Bar study focused on whether taking a specific course was related to performance on the bar exam topic taught in those courses. It found that neither attendance nor performance in courses covering any of the 13 bar-related topics was related to performance on the corresponding California bar exam or Multistate Bar Exam content covering that subject.

Studies at other schools also indicate that enrollment in bar subject-related courses do not support broad claims about the benefit of taking those courses.

It is time to put away the misinformed trope of participation in law clinic and externship courses harming a student’s chances of passing the bar exam and let the facts do the talking. Law schools should recognize and students should be told they can obtain valuable preparation for the practice of law by enrolling in clinical courses without affecting their likelihood of passing the bar exam.

Active Retrieval Practice: Known to Improve Learning but Underappreciated?

Exam time has arrived in law schools.  Students who want to excel on exams (and later, as attorneys) would do well to try out active retrieval practice.  To understand the value of retrieval practice, some brief discussion of well-established cognitive science is necessary.  Learning involves (1) taking knowledge into short-term working memory, and then (2) moving it from working memory to long-term memory by actively using the knowledge.[1]  In their excellent book, Teaching Law by Design, Dean Michael Hunter Schwartz, Professor Sophie Sparrow, and Professor Gerald Hess explain this process of storing learning in cognitive schema.[2]  They liken schemata to a “folder system[] provided for users of computer operating systems.”[3]  As they observe, however, storing knowledge “isn’t enough.  To analyze a problem, students must recall (“retrieve”) what they have learned and use that learning . . . .”[4] 

Research on cognition demonstrates that meaningful learning in any discipline requires the learner to perform some form of active retrieval exercises to be able to use the knowledge in analyzing and solving problems.   Active retrieval methods are ways in which the learner recalls knowledge and uses the recalled knowledge to solve problems or answer questions.[5]  Recalling for mere “knowledge checks,” sometimes called rote learning, is not effective.[6]   In the law school arena, a student can recite a memorized rule but not be able to apply it to fact patterns in a way that shows understanding.   Effective retrieval-based learning activities require the student to solve problems or to answer questions.  By doing so, the learner strengthens her understanding  of, and ability to recall, the knowledge.[7]  In law school, mid-term exams require students to recall information at least in mid-semester.  The problem there is that neuroscience shows a marked forgetting curve: if learning is not retrieved within a few days of its being stored, the knowledge is lost and must be relearned.[8]  Indeed, retrieving and using the knowledge are the critical parts of developing meaningful learning. 

               Spurred by the adoption of ABA Standard 314, my colleagues and I have been giving mid-term exams and using a variety of interim assessments designed to have students to recall information from previous classes. I regularly have a mid-term that includes multiple-choice, essays, or both.  The exams are graded and students receive a model answer. I discuss with students the answers to the assessments and common mistakes (e.g., failing to state rules accurately, insufficient application of facts in supporting one’s analysis).   The exam and follow-up discussions achieve the goal of providing the “meaningful feedback” that ABA Standard 314 seeks.   The mid-term is also summative. My experience is that many students do not take a practice, ungraded mid-term seriously. Having the exam count, but not so much as to prevent a student from recovering from a poor exam, helps to ensure that students prepare for and spend time on the mid-term.

               Another way that I have incorporated active recall practice is by using multiple-choice polling questions.  In the first class after we finish a course segment, we begin the class with multiple-choice questions that students answer by polling.    I ensure participation by recording the polling, by student, and including their responses (or lack thereof) as a class-participation part of the grade.  In answering the questions through polling, students must recall knowledge to analyze the question and reach a conclusion.   For example, after completing the study of removal jurisdiction and procedures, I use a series of multiple-choice polling questions that explore the many nuances of removal.   These sessions provide “meaningful feedback” to both students and to me.   If everyone misses a question, you can be sure I go back to discuss the area.  I also encourage anyone who missed a question in these polling sessions to meet with me after reviewing the topic addressed by the question.

               I urge students not to rely solely on the mid-term and the polling sessions as a means of ensuring they have learned material well.  Instead, I highly recommend preparing answers to essays under timed conditions.  At times I provide a model answer after they practice an essay. I also invite students to meet with me to go over their answers.   In these discussions, I almost always find some area in which a student has a mistaken understanding of a rule or concept.  If we did not uncover that misunderstanding, a student could repeatedly recall a flawed rule or approach.  Hence, I appreciate more than ever the wisdom of Standard 314’s emphasis on formative assessment.   After resolving any misunderstandings, I encourage a student to rewrite an answer.  That allows the student to revisit the topic and solidify her understanding.  Indeed, the act of writing itself helps students to embed the rules and concepts more firmly in their memory.[9] 

               What is true for law school is also true for the Bar Exam.   Last summer Sara Berman and I created a podcast for the ABA on practicing tests (essays, Multistate Performance Tests [MPTs], etc.) as some of the most effective ways to prepare for a state’s Bar Exam.[10]  Ideally, a student learns everything she needs to know in law school Bar review is just that–review.  More often, Bar applicants have a vague recollection of rules and concepts from their time in law school.  In other words, Bar applicants often find themselves relearning rules and concepts.  In so doing, they will learn more effectively by practicing an essay or MPT answer and by submitting these answers for grading to their Bar Preparation company and the faculty at their law school who help Bar applicants.  In doing these exercises, they benefit for at least two reasons.  First, it will identify areas in which the applicant’s knowledge of rules and concepts is so weak that she cannot answer a question.   Knowing that, the applicant can review that area and know that she needs to do so.  Second, exercises such as practice essay answers will require applicants to recall the rules they do know and apply them.  The more they do so, the more likely they are to remember them on the Bar. 

               Repeated, active retrieval practice is one of the best ways to learn to perform on exams or in law practice.  Yet, despite the data showing its effectiveness, such practices are not the norm in higher education.[11]  The practice is likely not the norm in law schools.   Standard 314 ought to help to some extent increase active retrieval before the end of the law school semester.  Yet, at present students are not spending their time as wisely as they could. Instead of preparing detailed outlines, and memorizing rules or flash cards, they would learn more from methods that require them to recall and apply legal rules and analysis.   Indeed, one might say this fact is one of the best kept secrets in law school. Perhaps it is time to let this secret out.


[1] Michael Hunter Schwartz, Sophie Sparrow & Gerald Hess, Teaching Law by Design 4­–7 (Carolina Academic Press 2009).

[2] Id.

[3] Id. at 5–6.

[4] Id. at 6.

[5] Jeffrey D. Karpicke, A Powerful Way to Improve Learning and Memory: Practicing Retrieval Enhances Long-Term, Meaningful Learning, American Psychological Association (2016), available at https://www.apa.org/science/about/psa/2016/06/learning-memory (last checked Dec. 6, 2019).

[6] Id.

[7] Id.

[8] Louis N. Schulze, Jr., Using Science to Build Better Learners: One School’s Successful Efforts to Raise its Bar Passage Rates in an Era of Decline, 68 J. Legal Educ. 230, 245 (Winter 2019).

[9] Bryan Goodwyn, The Magic of Writing Stuff Down, 75 Educational Leadership 78–79 (April 2019).

[10] Sara Berman and Ben Madison, Practice Makes Passing, Episode 6 of American Bar Associations Path to Law Student Well-Being Podcast series, available at  https://www.spreaker.com/show/path-to-law-student-well-being  (June 22, 2019).

[11] According to Dr. Karpicke, college students likewise use rote learning methods more than they use active retrieval exercises.  See Karpicke, supra note 5.

Guiding Students From Law School Into the World

It seems that one of the things we law professors can do to help our students develop their identities as professionals and their obligations to the greater society is to incorporate into the law school events that plug students into what’s going on in the “real world.“ I did just this in a small way this week by offering all the students the opportunity to attend and participate in a talk/discussion about the Supreme Court arguments that were heard last week in the DACA case.

The students were invited to attend a portion of my immigration clinic class. Food and pro bono credit hours helped, I’m sure, but the event brought a plentiful group of students I had not interacted with before, who were both knowledgeable about and interested in the issue of the day.  The event lasted only about 45 minutes, but that was long enough to produce a lively and I think informative conversation about oral arguments, professionalism, case theory, the role of policy, administrative law, and of course the specific legal issues raised by the case.

With so much of the law school endeavor focused on exam taking and other tasks that force students into a single-focused, competitive role, bringing them into a discussion about key issues at stake in our country in the moment could likely enhance their connections to their future and help them envision some individual goals they can aim for once out in that “real world.”

Best Practices on Halloween Must Include the Notorious RBG

By Jessica N. Haller, Blog Assistant of “Best Practices for Legal Education” Blog

Last week, Albany Law Students gave back to our host community, enjoyed the mental health break of seeing children in costumes enjoy Halloween, and collaborated with each other in meaningful ways.

Every year, Albany Law School opens wide its doors and creates a safe place for local children to trick or treat on Halloween right in the school’s gymnasium. There are families who now come year after year and who now see the law school as a place their children might one day attend. Most of the student organizations participate, setting up tables, handing out candy to the children or face painting. Many groups often have an activity or game or other treat for the costumed kids. For example, the Student Bar Association had a “Connect Four” game and the Black Law Student Association made “worms in dirt” dessert (chocolate ice cream, Oreos, and gummy worms, yum!).

I am currently on the e-board for the Albany Law School Women’s Law Caucus and this year, we decided to combine fun with a chance to teach children about our favorite U.S. Supreme Court Justice: Ruth Bader Ginsburg (RBG). We created a game called, “Pin the Dissent Collar on RBG.” We knew that many of the children coming to this event might not know who RBG is and we hoepd it might spark some conversation. We printed a large poster of RBG’s iconic Supreme Court photo and purchased three different kinds of black ribbon. The first little girl to participate was very enthusiastic about playing the game and asked about this mysterious “RBG.” You can see her playing the game in the photo below:

Ruth Bader Ginsburg is a feminine icon for women, especially for those of us in law school and in the legal academy. Despite the unending obstacles, she persevered. She was turned away from jobs despite being at the top of her class, and when she was able to get a job, she was paid considerably less than her male counterparts. She argued several cases, making it clear in each that gender equality is a constitutional right. For more information on RBG’s life and career, see her bio on Oyez (I also highly recommend a great documentary called “RBG” found here).

(Author and Blog Assistant, Jessica N. Haller, dressed as Agent “Peggy” Carter)

“Being Human To My Students And Letting Them Know I Care”

This is a wonderful blog post I found on the Institute for Law Learning and Teaching by Jane Korn, Professor of Law at Gonzaga University School of Law. As a current law student, I think that this practice should be implemented in all law schools for first year law students. I had a professor during 1L who did something similar. He would start the class every week with, “so how is 1L going?” and we could spend 10 minutes discussing general concerns about 1L and papers or exams we had coming up. Not only did it ease some of the anxiety, it also showed that the professor really cared about the students. It was like they were saying, “I’ve been there too and I’m here to support you.” Kudos to Professor Korn for setting aside some time in her class to do this!

“I have taught first year law students for a long time.  Please do not ask how long!  But years ago, I became worried about the mental health and stress levels of my first semester, first year students. I teach a four credit, one semester course in Civil Procedure during the first semester of law school.   On the last day of the week that I teach in Civ Pro, I take a few minutes out of class time and ask my students to tell me how they are doing.

The first time I do this, usually at the end of the first week of law school,  I tell my students that it is my custom, from time to time, to take time out from Civ Pro, and talk about anything they would like (with some limits).  In some years, it takes weeks for them to take me up on this offer.  Other years, they start right in.  They ask questions like the following:

  1. When should I start outlining?
  2. How much time should I spend studying every night?
  3. How important is getting involved in extracurricular activities?
  4. What if I don’t know what kind of law I want to practice?
  5. Do professors care about grammar and organization on a final exam? (I only answer what I expect and do not answer for other faculty)

I think that much of the time, they do not get a chance to ask a law professor these kinds of questions, and can usually only ask upper class students.  While we have faculty advisors, students may or may not feel comfortable asking them questions like the above.  They eventually do (and sometimes quickly) feel comfortable asking me a wide variety of questions.  They sometimes ask personal questions and, within reason, I answer them because it makes them feel more comfortable with me.  Questions on gossipy matters about other faculty are off limits. If for example, they complain about another professor,  I handle this question with a smile and say something like – you should ask that professor about this issue.

I set aside class time for several reasons. First, while I do worry about giving up valuable teaching time, lessening the stress of my students may make them more able to learn.  Second, students often feel like they are the only one with a particular concern during this first semester, and they often do not have the ability to know that others have the same concerns or questions.  In the first year, many of our students are not from this area and are far away from support systems, at least at first until they can make friends at law school.  The ability to know that other students have the same problems they do can lessen the feeling of isolation.  Using class time to answer questions to the entire group may help them with this sense of isolation and being the only one who doesn’t know something.  It also lets them see that their concerns are important and credible.

Every year my teaching evaluations reflect this process positively.  Students feel like I care (which I do).  However, the reason I do it is to increase their comfort during those first few exciting, confusing, and terrifying months of law school.”

 

Building an Ethos of Self-Directedness Among First-Year Law Students

By: Mary Walsh Fitzpatrick, Assistant Dean for the Career and Professional Development Center at Albany Law School

Background

I attended a workshop on professional identity formation sponsored by the Holloran Center for Ethical Leadership in the Professions last June. In preparation for the workshop, I read a number of articles on professional identity formation, including “Self-Directedness and Professional Formation: Connecting Two Critical Concepts in Legal Education.” The article posits “[f]or law students to move towards real professional identity formation in their career, they must be self-directed.” Self-directedness, some of the hallmarks of which are self-reflection, goal setting, seeking and receiving feedback, and using sound judgment, is integral to finding meaningful employment and career satisfaction. I know from first-hand experience working within law school career development for the past 13 years, students who are most successful in gaining meaningful employment take ownership of their experiences and make intentional choices early in law school. I believe self-directedness more than any other factor, including grades, leads to meaningful careers for law school graduates. The challenge is to cultivate self-directedness in all law students by creating an ethos of self-directedness with regard to career development beginning in the first year.     

While at the Holloran Center workshop, I devised a career development program to introduce first-year students to professional identity formation with emphasis on self-directedness. I subsequently shared the proposed program with my team at the Albany Law School Career and Professional Development Center and we collaborated on the program presentation and exercises. At Albany Law School, students are assigned an individual career counselor with whom they work one-on-one over the entire course of law school. Thus, in planning and implementing the program my colleagues and I chose to each lead the program for our sections separately, beginning individual relationships with our students and setting expectations.     

We decided upon a method of teaching that would allow students to practice self-reflection, seeking and receiving feedback, and using good judgment in the context of career development. The overarching goal of the program was to help students recognize self-directedness as a key component for successful professional identity formation leading to meaningful careers.

The Program – Setting the Stage

We communicated the program as a mandatory one-hour program and emailed to first-year students several weeks before the program the Individual Career Plan (ICP), a self-assessment tool we created several years ago, and our handbook for developing a legal resume. The students were asked to complete their ICPs and draft their legal resumes in preparation for the program. 

  • Reflection

We began the program by introducing professional identity formation and self-directed learning, emphasizing curiosity, initiative, feedback, self-reflection, resilience, judgment and ethics. We provided students with the Holloran Competency Milestones Assessment of Student’s Ownership of Continuous Professional Development (Self-Directedness) and asked them to take a moment to reflect upon and identify their current stage of development on the continuum. Recognizing each student comes to law school at a different stage of self-directedness we did not ask students to share their findings with the group, rather we called attention to law school providing students with the opportunity to move along the continuum with the goal of graduating competent learners who take full ownership over their careers by setting goals and seeking resources to meet those goals.

Next, we asked students to form small groups and to reflect upon and share with each other why they chose to attend law school, skills they hope to build, and experiences they hope to gain during law school. After this breakout session we asked one student from each group to report back some of the group’s findings. Two distinct motivations for attending law school emerged from this exercise, students: wanting to utilize existing strengths they identify as befitting a legal career; and wanting to acquire the skills necessary to be catalysts for change. Notably, both motivations evidence students’ strong desire to align their skills and values with meaningful employment.

  • Seeking and Receiving Feedback

In the second portion of the program we focused on seeking and receiving feedback in the context of career development. We began by educating students on critical thinking skills sought by legal employers, such as analyzing, evaluating, reasoning, and problem solving. We then asked students to provide peer-to-peer feedback on their resumes utilizing the resume handbook we provided before the program and tools we discussed during the program. Students worked in couples or groups of three to seek and provide each other with constructive feedback on how to better formulate existing resume descriptions for a legal audience. After the exercise we asked students to contribute one piece of valuable feedback they received.    

  • Judgment

In the final portion of the program students were divided into groups and provided three different hypotheticals related to career development decisions. Each group was asked to analyze the issues and report back how they would address the situation presented. The hypotheticals included issues of reneging on a job offer, misrepresenting grade point average on a resume, and failing to follow up with a professional connection. Through dialogue following the exercise we emphasized the importance of reputation, impact of reputational damage, building professional relationships, and the imperative of follow-through.  Many students acknowledged although no single hypothetical scenario would necessarily determine success in finding meaningful employment, the decisions made with regard to these issues could impact one’s professional reputation and future opportunities.   

Conclusion

We hope to have initiated student appreciation for the impact of self-directedness on professional identity formation that is integral to beginning meaningful careers after law school. The next step is for each student to take the initiative to complete an online strengths assessment, the VIA Character Strengths Survey, make a first career counseling appointment where they will receive individualized feedback on the ICP and legal resume and identify next steps in planning their careers.  

%d bloggers like this: