Active Retrieval Practice: Known to Improve Learning but Underappreciated?

Exam time has arrived in law schools.  Students who want to excel on exams (and later, as attorneys) would do well to try out active retrieval practice.  To understand the value of retrieval practice, some brief discussion of well-established cognitive science is necessary.  Learning involves (1) taking knowledge into short-term working memory, and then (2) moving it from working memory to long-term memory by actively using the knowledge.[1]  In their excellent book, Teaching Law by Design, Dean Michael Hunter Schwartz, Professor Sophie Sparrow, and Professor Gerald Hess explain this process of storing learning in cognitive schema.[2]  They liken schemata to a “folder system[] provided for users of computer operating systems.”[3]  As they observe, however, storing knowledge “isn’t enough.  To analyze a problem, students must recall (“retrieve”) what they have learned and use that learning . . . .”[4] 

Research on cognition demonstrates that meaningful learning in any discipline requires the learner to perform some form of active retrieval exercises to be able to use the knowledge in analyzing and solving problems.   Active retrieval methods are ways in which the learner recalls knowledge and uses the recalled knowledge to solve problems or answer questions.[5]  Recalling for mere “knowledge checks,” sometimes called rote learning, is not effective.[6]   In the law school arena, a student can recite a memorized rule but not be able to apply it to fact patterns in a way that shows understanding.   Effective retrieval-based learning activities require the student to solve problems or to answer questions.  By doing so, the learner strengthens her understanding  of, and ability to recall, the knowledge.[7]  In law school, mid-term exams require students to recall information at least in mid-semester.  The problem there is that neuroscience shows a marked forgetting curve: if learning is not retrieved within a few days of its being stored, the knowledge is lost and must be relearned.[8]  Indeed, retrieving and using the knowledge are the critical parts of developing meaningful learning. 

               Spurred by the adoption of ABA Standard 314, my colleagues and I have been giving mid-term exams and using a variety of interim assessments designed to have students to recall information from previous classes. I regularly have a mid-term that includes multiple-choice, essays, or both.  The exams are graded and students receive a model answer. I discuss with students the answers to the assessments and common mistakes (e.g., failing to state rules accurately, insufficient application of facts in supporting one’s analysis).   The exam and follow-up discussions achieve the goal of providing the “meaningful feedback” that ABA Standard 314 seeks.   The mid-term is also summative. My experience is that many students do not take a practice, ungraded mid-term seriously. Having the exam count, but not so much as to prevent a student from recovering from a poor exam, helps to ensure that students prepare for and spend time on the mid-term.

               Another way that I have incorporated active recall practice is by using multiple-choice polling questions.  In the first class after we finish a course segment, we begin the class with multiple-choice questions that students answer by polling.    I ensure participation by recording the polling, by student, and including their responses (or lack thereof) as a class-participation part of the grade.  In answering the questions through polling, students must recall knowledge to analyze the question and reach a conclusion.   For example, after completing the study of removal jurisdiction and procedures, I use a series of multiple-choice polling questions that explore the many nuances of removal.   These sessions provide “meaningful feedback” to both students and to me.   If everyone misses a question, you can be sure I go back to discuss the area.  I also encourage anyone who missed a question in these polling sessions to meet with me after reviewing the topic addressed by the question.

               I urge students not to rely solely on the mid-term and the polling sessions as a means of ensuring they have learned material well.  Instead, I highly recommend preparing answers to essays under timed conditions.  At times I provide a model answer after they practice an essay. I also invite students to meet with me to go over their answers.   In these discussions, I almost always find some area in which a student has a mistaken understanding of a rule or concept.  If we did not uncover that misunderstanding, a student could repeatedly recall a flawed rule or approach.  Hence, I appreciate more than ever the wisdom of Standard 314’s emphasis on formative assessment.   After resolving any misunderstandings, I encourage a student to rewrite an answer.  That allows the student to revisit the topic and solidify her understanding.  Indeed, the act of writing itself helps students to embed the rules and concepts more firmly in their memory.[9] 

               What is true for law school is also true for the Bar Exam.   Last summer Sara Berman and I created a podcast for the ABA on practicing tests (essays, Multistate Performance Tests [MPTs], etc.) as some of the most effective ways to prepare for a state’s Bar Exam.[10]  Ideally, a student learns everything she needs to know in law school Bar review is just that–review.  More often, Bar applicants have a vague recollection of rules and concepts from their time in law school.  In other words, Bar applicants often find themselves relearning rules and concepts.  In so doing, they will learn more effectively by practicing an essay or MPT answer and by submitting these answers for grading to their Bar Preparation company and the faculty at their law school who help Bar applicants.  In doing these exercises, they benefit for at least two reasons.  First, it will identify areas in which the applicant’s knowledge of rules and concepts is so weak that she cannot answer a question.   Knowing that, the applicant can review that area and know that she needs to do so.  Second, exercises such as practice essay answers will require applicants to recall the rules they do know and apply them.  The more they do so, the more likely they are to remember them on the Bar. 

               Repeated, active retrieval practice is one of the best ways to learn to perform on exams or in law practice.  Yet, despite the data showing its effectiveness, such practices are not the norm in higher education.[11]  The practice is likely not the norm in law schools.   Standard 314 ought to help to some extent increase active retrieval before the end of the law school semester.  Yet, at present students are not spending their time as wisely as they could. Instead of preparing detailed outlines, and memorizing rules or flash cards, they would learn more from methods that require them to recall and apply legal rules and analysis.   Indeed, one might say this fact is one of the best kept secrets in law school. Perhaps it is time to let this secret out.


[1] Michael Hunter Schwartz, Sophie Sparrow & Gerald Hess, Teaching Law by Design 4­–7 (Carolina Academic Press 2009).

[2] Id.

[3] Id. at 5–6.

[4] Id. at 6.

[5] Jeffrey D. Karpicke, A Powerful Way to Improve Learning and Memory: Practicing Retrieval Enhances Long-Term, Meaningful Learning, American Psychological Association (2016), available at https://www.apa.org/science/about/psa/2016/06/learning-memory (last checked Dec. 6, 2019).

[6] Id.

[7] Id.

[8] Louis N. Schulze, Jr., Using Science to Build Better Learners: One School’s Successful Efforts to Raise its Bar Passage Rates in an Era of Decline, 68 J. Legal Educ. 230, 245 (Winter 2019).

[9] Bryan Goodwyn, The Magic of Writing Stuff Down, 75 Educational Leadership 78–79 (April 2019).

[10] Sara Berman and Ben Madison, Practice Makes Passing, Episode 6 of American Bar Associations Path to Law Student Well-Being Podcast series, available at  https://www.spreaker.com/show/path-to-law-student-well-being  (June 22, 2019).

[11] According to Dr. Karpicke, college students likewise use rote learning methods more than they use active retrieval exercises.  See Karpicke, supra note 5.

Guiding Students From Law School Into the World

It seems that one of the things we law professors can do to help our students develop their identities as professionals and their obligations to the greater society is to incorporate into the law school events that plug students into what’s going on in the “real world.“ I did just this in a small way this week by offering all the students the opportunity to attend and participate in a talk/discussion about the Supreme Court arguments that were heard last week in the DACA case.

The students were invited to attend a portion of my immigration clinic class. Food and pro bono credit hours helped, I’m sure, but the event brought a plentiful group of students I had not interacted with before, who were both knowledgeable about and interested in the issue of the day.  The event lasted only about 45 minutes, but that was long enough to produce a lively and I think informative conversation about oral arguments, professionalism, case theory, the role of policy, administrative law, and of course the specific legal issues raised by the case.

With so much of the law school endeavor focused on exam taking and other tasks that force students into a single-focused, competitive role, bringing them into a discussion about key issues at stake in our country in the moment could likely enhance their connections to their future and help them envision some individual goals they can aim for once out in that “real world.”

Best Practices on Halloween Must Include the Notorious RBG

By Jessica N. Haller, Blog Assistant of “Best Practices for Legal Education” Blog

Last week, Albany Law Students gave back to our host community, enjoyed the mental health break of seeing children in costumes enjoy Halloween, and collaborated with each other in meaningful ways.

Every year, Albany Law School opens wide its doors and creates a safe place for local children to trick or treat on Halloween right in the school’s gymnasium. There are families who now come year after year and who now see the law school as a place their children might one day attend. Most of the student organizations participate, setting up tables, handing out candy to the children or face painting. Many groups often have an activity or game or other treat for the costumed kids. For example, the Student Bar Association had a “Connect Four” game and the Black Law Student Association made “worms in dirt” dessert (chocolate ice cream, Oreos, and gummy worms, yum!).

I am currently on the e-board for the Albany Law School Women’s Law Caucus and this year, we decided to combine fun with a chance to teach children about our favorite U.S. Supreme Court Justice: Ruth Bader Ginsburg (RBG). We created a game called, “Pin the Dissent Collar on RBG.” We knew that many of the children coming to this event might not know who RBG is and we hoepd it might spark some conversation. We printed a large poster of RBG’s iconic Supreme Court photo and purchased three different kinds of black ribbon. The first little girl to participate was very enthusiastic about playing the game and asked about this mysterious “RBG.” You can see her playing the game in the photo below:

Ruth Bader Ginsburg is a feminine icon for women, especially for those of us in law school and in the legal academy. Despite the unending obstacles, she persevered. She was turned away from jobs despite being at the top of her class, and when she was able to get a job, she was paid considerably less than her male counterparts. She argued several cases, making it clear in each that gender equality is a constitutional right. For more information on RBG’s life and career, see her bio on Oyez (I also highly recommend a great documentary called “RBG” found here).

(Author and Blog Assistant, Jessica N. Haller, dressed as Agent “Peggy” Carter)

“Being Human To My Students And Letting Them Know I Care”

This is a wonderful blog post I found on the Institute for Law Learning and Teaching by Jane Korn, Professor of Law at Gonzaga University School of Law. As a current law student, I think that this practice should be implemented in all law schools for first year law students. I had a professor during 1L who did something similar. He would start the class every week with, “so how is 1L going?” and we could spend 10 minutes discussing general concerns about 1L and papers or exams we had coming up. Not only did it ease some of the anxiety, it also showed that the professor really cared about the students. It was like they were saying, “I’ve been there too and I’m here to support you.” Kudos to Professor Korn for setting aside some time in her class to do this!

“I have taught first year law students for a long time.  Please do not ask how long!  But years ago, I became worried about the mental health and stress levels of my first semester, first year students. I teach a four credit, one semester course in Civil Procedure during the first semester of law school.   On the last day of the week that I teach in Civ Pro, I take a few minutes out of class time and ask my students to tell me how they are doing.

The first time I do this, usually at the end of the first week of law school,  I tell my students that it is my custom, from time to time, to take time out from Civ Pro, and talk about anything they would like (with some limits).  In some years, it takes weeks for them to take me up on this offer.  Other years, they start right in.  They ask questions like the following:

  1. When should I start outlining?
  2. How much time should I spend studying every night?
  3. How important is getting involved in extracurricular activities?
  4. What if I don’t know what kind of law I want to practice?
  5. Do professors care about grammar and organization on a final exam? (I only answer what I expect and do not answer for other faculty)

I think that much of the time, they do not get a chance to ask a law professor these kinds of questions, and can usually only ask upper class students.  While we have faculty advisors, students may or may not feel comfortable asking them questions like the above.  They eventually do (and sometimes quickly) feel comfortable asking me a wide variety of questions.  They sometimes ask personal questions and, within reason, I answer them because it makes them feel more comfortable with me.  Questions on gossipy matters about other faculty are off limits. If for example, they complain about another professor,  I handle this question with a smile and say something like – you should ask that professor about this issue.

I set aside class time for several reasons. First, while I do worry about giving up valuable teaching time, lessening the stress of my students may make them more able to learn.  Second, students often feel like they are the only one with a particular concern during this first semester, and they often do not have the ability to know that others have the same concerns or questions.  In the first year, many of our students are not from this area and are far away from support systems, at least at first until they can make friends at law school.  The ability to know that other students have the same problems they do can lessen the feeling of isolation.  Using class time to answer questions to the entire group may help them with this sense of isolation and being the only one who doesn’t know something.  It also lets them see that their concerns are important and credible.

Every year my teaching evaluations reflect this process positively.  Students feel like I care (which I do).  However, the reason I do it is to increase their comfort during those first few exciting, confusing, and terrifying months of law school.”

 

Building an Ethos of Self-Directedness Among First-Year Law Students

By: Mary Walsh Fitzpatrick, Assistant Dean for the Career and Professional Development Center at Albany Law School

Background

I attended a workshop on professional identity formation sponsored by the Holloran Center for Ethical Leadership in the Professions last June. In preparation for the workshop, I read a number of articles on professional identity formation, including “Self-Directedness and Professional Formation: Connecting Two Critical Concepts in Legal Education.” The article posits “[f]or law students to move towards real professional identity formation in their career, they must be self-directed.” Self-directedness, some of the hallmarks of which are self-reflection, goal setting, seeking and receiving feedback, and using sound judgment, is integral to finding meaningful employment and career satisfaction. I know from first-hand experience working within law school career development for the past 13 years, students who are most successful in gaining meaningful employment take ownership of their experiences and make intentional choices early in law school. I believe self-directedness more than any other factor, including grades, leads to meaningful careers for law school graduates. The challenge is to cultivate self-directedness in all law students by creating an ethos of self-directedness with regard to career development beginning in the first year.     

While at the Holloran Center workshop, I devised a career development program to introduce first-year students to professional identity formation with emphasis on self-directedness. I subsequently shared the proposed program with my team at the Albany Law School Career and Professional Development Center and we collaborated on the program presentation and exercises. At Albany Law School, students are assigned an individual career counselor with whom they work one-on-one over the entire course of law school. Thus, in planning and implementing the program my colleagues and I chose to each lead the program for our sections separately, beginning individual relationships with our students and setting expectations.     

We decided upon a method of teaching that would allow students to practice self-reflection, seeking and receiving feedback, and using good judgment in the context of career development. The overarching goal of the program was to help students recognize self-directedness as a key component for successful professional identity formation leading to meaningful careers.

The Program – Setting the Stage

We communicated the program as a mandatory one-hour program and emailed to first-year students several weeks before the program the Individual Career Plan (ICP), a self-assessment tool we created several years ago, and our handbook for developing a legal resume. The students were asked to complete their ICPs and draft their legal resumes in preparation for the program. 

  • Reflection

We began the program by introducing professional identity formation and self-directed learning, emphasizing curiosity, initiative, feedback, self-reflection, resilience, judgment and ethics. We provided students with the Holloran Competency Milestones Assessment of Student’s Ownership of Continuous Professional Development (Self-Directedness) and asked them to take a moment to reflect upon and identify their current stage of development on the continuum. Recognizing each student comes to law school at a different stage of self-directedness we did not ask students to share their findings with the group, rather we called attention to law school providing students with the opportunity to move along the continuum with the goal of graduating competent learners who take full ownership over their careers by setting goals and seeking resources to meet those goals.

Next, we asked students to form small groups and to reflect upon and share with each other why they chose to attend law school, skills they hope to build, and experiences they hope to gain during law school. After this breakout session we asked one student from each group to report back some of the group’s findings. Two distinct motivations for attending law school emerged from this exercise, students: wanting to utilize existing strengths they identify as befitting a legal career; and wanting to acquire the skills necessary to be catalysts for change. Notably, both motivations evidence students’ strong desire to align their skills and values with meaningful employment.

  • Seeking and Receiving Feedback

In the second portion of the program we focused on seeking and receiving feedback in the context of career development. We began by educating students on critical thinking skills sought by legal employers, such as analyzing, evaluating, reasoning, and problem solving. We then asked students to provide peer-to-peer feedback on their resumes utilizing the resume handbook we provided before the program and tools we discussed during the program. Students worked in couples or groups of three to seek and provide each other with constructive feedback on how to better formulate existing resume descriptions for a legal audience. After the exercise we asked students to contribute one piece of valuable feedback they received.    

  • Judgment

In the final portion of the program students were divided into groups and provided three different hypotheticals related to career development decisions. Each group was asked to analyze the issues and report back how they would address the situation presented. The hypotheticals included issues of reneging on a job offer, misrepresenting grade point average on a resume, and failing to follow up with a professional connection. Through dialogue following the exercise we emphasized the importance of reputation, impact of reputational damage, building professional relationships, and the imperative of follow-through.  Many students acknowledged although no single hypothetical scenario would necessarily determine success in finding meaningful employment, the decisions made with regard to these issues could impact one’s professional reputation and future opportunities.   

Conclusion

We hope to have initiated student appreciation for the impact of self-directedness on professional identity formation that is integral to beginning meaningful careers after law school. The next step is for each student to take the initiative to complete an online strengths assessment, the VIA Character Strengths Survey, make a first career counseling appointment where they will receive individualized feedback on the ICP and legal resume and identify next steps in planning their careers.  

Save the Date!

Emory’s Center for Transactional Law and Practice is excited to announce the date for its seventh biennial conference on the teaching of transactional law and skills.  The conference will be held at Emory Law, on Friday, June 5, 2020, and Saturday, June 6, 2020.

More information will be forthcoming on the Call for Proposals, the Call for Nominations for the Tina L. Stark Award for Excellence in the Teaching of Transactional Law and Skills, open registration, and travel accommodations.  We are looking forward to seeing all of you on June 5 and 6, 2020!

Thank you to the Conference Coordinator, Kelli Pittman, for sending us this information!

The Legal Interviewing and Language Access Film Project (LILA)

By: Laila L. Hlass and Lindsay M. Harris

Teaching effective interviewing skills is a perennial problem. Although there are excellent texts on the subject, few examples of real or model interviews exist, particularly ones which incorporate collaboration issues between student partners, language access issues with the client, and how to address issues of bias when they arise in the interview.

In 2018, we designed, screen-wrote, produced and released The Legal Interviewing and Language Access Film Project (LILA), two instructional videos and a teaching guide featuring a law student clinic pair representing two different immigrant clients, in two different introductory meetings, one of which is conducted with interpretation.

Our goal was to better teach interviewing in our own experiential courses, but we also hoped to share this resource with our colleagues. Since the videos were launched, law school clinics and experiential learning programs across the country have adopted the use of the videos. At the time of writing, more than 100 educators at nearly 75 law schools have requested use of the teacher’s guide for these videos. This includes more than 30 immigration clinics, but also educators teaching in a variety of other clinics, purely doctrinal courses, as well as courses focused on client counseling and interviewing skills.

The videos raise a multitude of issues within interviewing including client-centered lawyering, collaboration, interpretation, and addressing bias. Our films enliven and deepen the learning environment by utilizing modeling, as well as stimulating classroom discussion, reflection and role play. 

In Interviewing Victor: The Initial Meeting, two law students Lisa and Max interview a teenage asylum-seeker in removal proceedings, Victor, raising a number of issues relating to initial client interviewing, including: Road mapping and organization of the interview; Building rapport; Confidentiality; Role description, including representation at later stages, and explaining the arc of case; Verbal and nonverbal cues; Tone; Answering client questions or ethical issues that are difficult and unexpected; Recording the interview and seeking permission; Taking notes; Form of questions; Word choice; Approaches to sensitive topics and response to client’s distress; Client-centered lawyering; and Working with a co-interviewer.

In Josefina: Using an Interpreter, two law students Lisa and Max working with interpreters to interview a monolingual Spanish-speaking client seeking a U visa as a victim of a crime in the United States. This video raises questions regarding: Using third person; Pacing of speech; Summarization and  expansion of interpretation; Challenges when one student speaks the client’s language but partner does not; Confidentiality; Use of interested parties, such as family members; Approaches to changing interpreters; and Use of common language words where the interpreter doesn’t know the intended meaning.

For faculty who hope to adopt the videos in a course, pro bono orientation or other training, please email either Laila Hlass lhlass@tulane.edu or Lindsay Harris Lindsay.harris@udc.edu for the teacher’s guide, indicating in which course(s) you are considering using the films.

%d bloggers like this: