The Potential Adjunctification of Law School Faculties

Under proposed ABA Standard 403, future law schools could have only a handful of full-time faculty members.  If those full-time faculty members teach the first year curriculum, the proposed Standard allows for upper level classes to be taught entirely by adjunct faculty as long as the law school ensures the adjunct faculty effectively teach their assigned courses.

As a practical matter, does the proposed standard mean most law schools will become institutions staffed largely by adjuncts?  Probably not.  However, it may mean that for-profit law schools will staff their upper level classes almost entirely with adjunct faculty.

For-profit schools – the beneficiaries of no restrictions on adjunct teaching

If legal education follows the trends seen in undergraduate institutions, for-profit schools are the schools most likely to take advantage of the proposed standard.  A study authored by Phillip W. Magness found that 93% of faculty at for-profit universities are part-time faculty.  This compares with 29.5% part-time faculty at four year public schools, 42% part-time faculty at four year private schools, and 65% part-time faculty at two year non-profit colleges.

At a time when for-profit law schools are facing increased scrutiny for bar pass rates, should the Council on Legal Education be considering a rule change that enables those schools to rely heavily on adjunct faculty?  What impact will an adjunct faculty have on student learning in core doctrinal courses?  These questions should be addressed.

Justification for changing Standard 403 to allow more adjunct teaching

Unquestionably, adjunct faculty serve important roles in law schools.  Practicing attorneys and judges bring a depth of experience into the classroom that full-time faculty may not have.   However, the current Standard already allows for one-third of the upper level classes to be taught by adjuncts.  Why is there a need to allow for more adjunct teaching?

The justification, as stated in the Council’s March 24, 2017 memo, is that elimination of the full-time faculty requirements allows for innovation and flexibility and that the Standard retains the requirement that schools ensure effective adjunct teaching. However, with a smaller subset of full-time faculty, many of whom will have additional institutional service work and have scholarship demands, the ability to also meaningfully supervise a large contingent part-time faculty may not be realistic.

Studies on adjuncts in undergraduate institutions

Of course, not all full-time faculty are better teachers than adjunct faculty.  In fact, on an individual level, often the opposite is true.  The question is what happens when a law school begins to rely heavily on adjuncts for two thirds of the courses students take?

For the past twenty years, undergraduate institutions have moved toward increasing use of adjunct faculty.  Although no definitive studies exist comparing undergraduate student learning outcomes between part-time versus full-time faculty, some studies suggest that student learning declines when students do not have the benefit of full-time faculty teaching doctrinal courses.

For example, a study by Florence Kirk and Charles Spector found undergraduate business students taught by full-time faculty had stronger learning outcomes than students taught by adjuncts.  Another longitudinal  study found that increased use of adjunct faculty correlated to lower undergraduate graduation rates.  However, other studies suggest adjuncts do not negatively affect student learning.

One reason for the Standard change might be that it sets the stage to allow schools to teach more online courses and staff them with adjuncts.  However, at least one study found that full-time undergraduate online faculty produced stronger learning gains than adjunct online faculty.

None of these studies examine the impact of adjuncts on student learning in law school doctrinal courses.  Before moving to a Standard that allows law schools to staff all upper level courses with adjunct faculty, it might be prudent to study law students’ learning outcomes in doctrinal courses taught by adjuncts versus those taught by full-time faculty.

Teaching aside, another study found that, not surprisingly, contingent faculty have less time for class preparation, fewer interactions with students on course and non course related issues, challenge students less, and use less interactive and collaborative teaching methods.  Additionally, adjuncts likely have less access to university teaching resources and less time to take advantage of those resources.

Adjunctification can lead to schools that have few tenured faculty members

The impact of the proposed Standard 403 change  on student learning raises some significant concerns.  Another concern is that part-time faculty are not tenured or tenure-track faculty. In undergraduate institutions, the Magness study notes that the growth in part-time undergraduate faculty resulted in “a decline in the overall percentage (though not in the absolute number) of tenured and tenure track faculty” – with tenure systems “virtually non-existent in for-profit higher education” institutions.

To the extent that tenure ensures both academic freedom and a robust system of faculty governance that protects the interests of students in an era of the monetization of education, a Standard that allows schools to significantly limit, or largely eliminate, tenure and tenure track faculty raises concerns.  This is especially true because the schools most likely to take advantage of a part-time faculty that has little voice in governance are the for-profit schools – schools which may be most likely to put economic interests ahead of educational concerns.

Notice and Comment Period Open Until July 10

 As the ABA Council on Legal Education grapples with a host of issues related to ensuring law students receive a strong legal education, is now the time to deregulate and allow schools to fully staff all upper level courses with adjunct faculty? Are the potential benefits of adjunctification when compared with the potential risks, worth the change?  Should the ABA be enacting regulatory rollbacks that may primarily benefit for profit schools?  All these are questions the Council should address before approving this Standard.

If you have thoughts about the implications of the proposed change to Standard 403 and the potential adjunctification of law schools, written comments or a request to speak at a hearing on the proposed change should be addressed to JR Clark, jr.clark@americanbar.org, by Monday, July 10, 2017.

PUBLIC OPPOSITION TO JEFF SESSIONS RESULTS IN AN OPEN RECORDS REQUEST

Along with 1,400 other law professors, I signed a letter opposing the nomination of Jeff Sessions for Attorney General of the United States. As a law professor, I signed this letter because of my concerns about maintaining the integrity of the legal system.

Shortly after the law professors’ letter was published, my university counsel’s office got an Open Records Act request seeking my emails.

The request, from a reporter working for a conservative political publication, sought: “a copy of each email (inbound, outbound, deleted, or double deleted) for the university email accounts of Andrea A. Curcio and [a colleague who also signed the letter] from the dates of December 15, 2016, to and including January 3, 2017, which includes any of the keywords “Sessions,” or “Jeff Sessions” or “Attorney General.””

A similar request was sent to university counsel for law professor signatories working at other public institutions.

Open records requests are a key to governmental transparency. Being personally subjected to one is unnerving.

How do you avoid such a request if you work at a public law school? You stay silent. Non-involvement with anything in the least bit controversial helps protect you from the possibility that anyone will ever ask to see the content of your emails.

I have often asked myself the theoretical question: if I had lived in Nazi Germany, or in the McCarthy era, would I have remained silent or would I have taken the risk and spoken up. That question is no longer theoretical.

Celebrating Martin Luther King, Jr. Day reminds us of the courage of those who stood up for what they believed was right. Today we again have a very visible choice about whether to step off the curb or to let fear silence us.

 

Adding Your Name to the Professor Watchlist

Most of you have probably heard of the Professor Watchlist, a website that seeks to “expose and document college professors who discriminate against conservative students and advance leftist propaganda in the classroom.”

As the AAUP notes, “This website poses a clear threat to academic freedom. Over the past century, such lists have been used to harass and intimidate faculty, and have a chilling effect on academic freedom and free speech.”

One way to stand in solidarity with those who have been named to this list is to voluntarily place yourselves on this list.  If you would like your name to be added to the list,  the  link is:

https://actionnetwork.org/forms/add-my-name-to-the-professor-watchlist?source=direct_link&

You also could consider spreading the word to colleagues at your school and in other departments.

As our core values come under attack, addressing those attacks both empowers us and demonstrates to our students that lawyers must continue to take the lead in fighting for social justice – especially in times where doing so may present personal or professional risks.

Proposed Change to Bar Pass Accreditation Standard

This post alerts readers to a proposed change in the bar pass accreditation standard and questions that change raises.  It also asks whether the proposed accreditation standard change opens the door to a bigger discussion about the need for a licensing exam that better reflects the competencies needed to practice law.

Proposed Change

A proposed change to ABA Standard 316, the bar accreditation pass rate standard, is out for notice and comment. The proposed change simplifies the standard to a single criteria: “At least 75% of a school’s students who sit for a bar exam must pass an exam within two years of graduation.”

The proposed standard eliminates the current provision that accounts for the significant variations in states’ overall bar pass rates. Currently, schools are in compliance with Standard 316 if “in three of the last five years, a school’s first time bar passage rate is no more than 15 points below the average first-time bar passage rates for graduates of ABA approved law schools taking the bar exam in those same jurisdictions.”

Questions The Proposed Change Raises

A recent Society of American Law Teachers comment letter provides an in-depth exploration of numerous unanswered questions the proposed accreditation standard change raises.  Those questions include:  1. its impact on schools in states with historically low bar pass rates; 2. its impact on schools in states which recently have adopted the UBE that are in the process of adjusting their curriculum to conform to the UBE; and 3. its potential impact on access to legal education if the proposed change causes schools to rely even more heavily on LSAT scores in their admissions process. These questions suggest a need for more information before adopting this proposed Standard.

Others who want to comment on the proposed change have until July 29 to do so.

Proposed Standard Prompts Questions About the Exam

Schools clearly have an obligation to ensure their students are able to obtain a law license and accreditation standards must ensure schools meet that obligation.  That said, this proposed standard change should open the door to discussions about the bar exam itself.

Why do we have an exam that requires memorization of thousands of legal rules, many of which may be inapplicable in an examinee’s own jurisdiction?  Does an examinee’s ability to read, issue spot, analyze and answer two hundred multiple choice questions in 360 minutes [allowing less than two minutes per question] really demonstrate that person’s minimum competence to practice law?    Is an exam that tests a wide range of doctrinal areas but only a few lawyering skills, and that tests lawyering skills in ways largely unrelated to law practice, the best way to  protect the public from incompetent lawyers?

As the SALT comment notes, many states “have attempted to address the disconnect between the bar exam and law practice with alternative licensing requirements designed to ensure newly licensed lawyers possess a broader range of competencies than those tested by bar exams.”   To better protect the public from incompetent newly licensed lawyers, some states have added pro bono work or experiential learning and practice requirements. These additional requirements may help remedy bar exam inadequacies but they also  create additional hurdles for future lawyers, present regulatory and administrative challenges for state licensing agencies, and create a wide range of issues for schools with a national student body.

At the same time there is a push for national uniformity in law licensing via the Uniform Bar Exam [UBE],  we see an array of state licensing requirements.   If we had a bar exam that actually measured minimum competence to practice law, would states need to develop individualized supplementary licensing mechanisms?

A Time For Change?

The NCBE has engaged in a concerted campaign to push states to adopt the UBE.  What would happen if the tables turned and states pressured the NCBE to develop a test that better measured competence to practice law? What would happen if some entrepreneur developed a superior law licensing exam and threatened the NCBE’s virtual monopoly over the law licensing process? Would we then have a test that more accurately reflected and assessed the skills lawyers need?

If there is a demand, a product appears. That certainly is what we saw in the rapid changes to legal education models brought on by public demand.

As colleagues and I wrote about in a recent article, many legal educators and scholars have developed a range of assessments that measure a much wider range of lawyering skills than is currently tested on bar exams.  Why isn’t the NCBE working with those people to develop a better bar exam?

While legal educators adapt what and how we teach to help prepare students for the existing bar exam, we also should be questioning the validity of that exam.  Discussions about the proposed amendments to Standard 316 provide an opportunity to push for much needed changes to the bar exam.

Summer Teaching Innovation Grants

At many schools,  faculty members currently are applying for summer research grants.  This blog posts suggests that in addition to research grants, schools consider summer teaching innovation grants. Just like good scholarship, developing experiential learning courses or course components takes time.

Summer Teaching Innovation Grants – An Experiment that Worked

At Georgia State, like at many schools, our dean has encouraged us to integrate experiential learning throughout the curriculum.  And, he has put his money where his mouth is.

Faculty can compete for  summer teaching innovation grants which are funded at the same level as research grants. Both junior and senior faculty members have taken advantage of the summer grant  opportunities to either revamp existing courses or create new ones.

What A Grant Can Produce

Video Evidence

This  video illustrates some of the courses developed through the innovation grants.

We use  the video as  an admissions tool and at alumni functions to  highlight the law school’s innovative and experiential teaching across the curriculum.

Some Examples

In the video, you will hear about numerous courses that integrate experiential learning.  Many of those courses were developed as a result of summer teaching grants. For example:

Professor Corneill Stephens, after twenty years of teaching first year contracts, radically revamped the second semester of his course.  That semester now devotes substantial time to “hands on” contract drafting and interpretation.

Professor Erin Fuse Brown, who has only been teaching a few years, developed a capstone simulation health care transactions course, taught with practicing attorney adjuncts.

Professor Jessie Gabel Cino,  shortly after she was granted tenure, created a bankruptcy assistance program course in which students work with practicing attorneys to handle simple bankruptcies.

These courses, described in more detail in the video, are just a few examples of courses that came into being as a result of a summer teaching innovation grant.

Grant Criteria

The grant award process is competitive and judged by the following criteria:

  1. The project must result in the creation of a new class or redesign of an existing one that will integrate skills/professional values/experiential components not traditionally taught in conventional courses.
  1. The project proposal must include a statement of learning objectives and outcomes for the course.
  1. The project must include a plan for assessment of learning outcomes, including both formative as well as summative assessment methods, as appropriate.
  1. The project must be capable of being completed over the summer in which the grant is received and implemented preferably in the ensuing academic year.
  1. The project proposal describes a plan for making the new or redesigned course sustainable (capable of being taught on a recurring basis.

Creating a Culture of Teaching Innovation and Excellence

All grant recipients must present to the faculty in the semester following their grant-based course.  This presentation allows other faculty to learn about new courses or changes to existing courses and prompts thinking about one’s own course.

The video describes courses that came into being due to summer teaching innovation grants, and includes some course innovations that resulted simply from a culture which demonstrably values teaching innovation through concrete actions such as summer grants, weekly informal coffees to talk about teaching, and emphasis on teaching in our annual reports.

It’s not too late to for your dean to offer summer  grants as a way to further develop institutional support for teaching innovations.  If these are not already in place at your school, perhaps you can encourage your dean to consider this option.

 

 

 

 

 

 

Collaborative Doctrinal Teaching Across Institutions: A Successful Experiment

By:  Andi Curcio and Eileen Kaufman

For most doctrinal faculty, teaching is a relatively solitary activity. While our clinical and legal writing colleagues regularly collaborate, and we encourage student collaborative learning, doctrinal faculty tend to draft our syllabus, develop our lesson plans, teach our courses and design and administer our assessments largely without input from colleagues.

We operate this way because it is part of legal academic culture, because we deem solitary class planning to be more efficient, and because we value independence in deciding what to teach, how to teach and what and how to assess.

Last Fall, two of us – experienced law professors working at different law schools and both teaching a large section Evidence course- decided to break with the cultural tradition of flying solo when it comes to doctrinal teaching.

We found that collaboration did not take more time than working on our own. Nor did collaboration impinge on our academic freedom. In fact, the collaborative process challenged us to engage deeply with both the material and our pedagogy in ways that made us stronger teachers.

Below, we raise questions to consider when deciding to collaborate, describe our collaborative model, and highlight the benefits that resulted from our collaborative work.

Questions Before Collaborating

Why collaborate? The first question you must answer is whether you want to collaborate and if so, what you want out of the experience. For example, do you want to engage in give-and-take that enhances your teaching, have a sounding board for tough doctrinal or teaching issues, experiment with new teaching materials and techniques, improve your efficiency by a “divide and conquer” approach to class planning and/or assessments?

How extensive a collaboration? Once you identify why you want to collaborate, you can better decide how much collaboration works for you. The collaboration could be as informal as simply using each other as a resource when confronting tough questions. You could take the collaboration one step further and agree to a common syllabus. Or you could dive way in and co-design all teaching materials, assessments, and grading rubrics.

Who should you collaborate with? The final, and perhaps most important, question is who to work with. Do you want to work with someone who has a similar teaching philosophy, prepares in advance or tends to wait until the last minute, has the same or a different level of teaching experience, works at the same institution or a different one? Whatever the answer to those questions, the key to a successful collaboration is finding someone you respect and can communicate with comfortably, freely and honestly.

Our Collaborative Method

We briefly discuss our collaborative method, answering the questions above, to illustrate one way to engage in doctrinal teaching collaboration.

Choosing A Collaborator and Collaboration Method:  We had known each other for years and had worked together on other projects. One of us had taught Evidence for 15 years, the other, although an experienced teacher, was relatively new to the subject. We deeply respected each other as teachers and knew we would learn a lot from each other.

We chose the full-blown model of collaboration that included using the same course materials, employing clickers in the classroom, and co-designing all teaching materials, assessments and grading rubrics.  Our collaborative goals included enhancing student learning of complex and difficult material, improving our teaching, and developing fair and reliable assessments.

Implementing the Collaborative Model: With our collaborative goals in mind, we selected a book that neither of us had previously used, that we thought would work best for both sets of students. We then made decisions about course content and coverage and the number and type of assessments. Once those decisions were made, one of us drafted the syllabus and the other edited it.

Although we each prepared our own classes, we shared power points and hypotheticals so that our classes fairly closely mirrored each other. If one of us taught a class first, and ran into an area or problem that confused students, we alerted the other to the problem and discussed ways to remedy the confusion.

We utilized a number of assessments – TWEN quizzes, practice essays followed by detailed grading rubrics, a self-assessment, a midterm and a final exam. For all of the assessments, one of us took the lead and the other edited the exam. We reversed roles for the grading rubric – if one of us drafted the exam, the other drafted the rubric. This distributed the workload and helped enormously in spotting ambiguities or problems that might undermine the question’s validity.

While we spoke somewhat regularly, most of our collaborative work was done via email so we could do it at our respective convenience.

The Benefits

The collaborative process allowed us to benefit from each other’s knowledge, skills and ideas. We had a sounding board to discuss how best to teach a complex doctrinal issue. We had another set of eyes to help ensure exam questions were clearly written, and that the questions, as well as the overall exam, tested what we sought to assess. When grading, we were able to compare how we were making the “close calls.”

Teaching Benefits:  As the semester progressed, we shared thoughts and ideas about how to shift our teaching when students struggled with either concepts or analyses. For example, we realized early on that the use of clickers in the classroom was helping students on multiple choice questions but was not addressing analytical skills. Since we were using the same classroom materials and assessments, we were able to track and compare performance and make important adjustments as the semester progressed.

We discussed problems we both observed with our respective students’ legal analysis. We brainstormed solutions and decided we should: take a minute of class time for students to write out a rule before launching into hypotheticals or discussions of the rule; require students to provide more analysis in their explanations of clicker questions; and use clicker slides to drill black letter law and application of law to new facts.  We added more practice essays and incorporated teaching methodologies that focused on building students’ analytical and essay writing skills.

These changes to our teaching methodologies may not have occurred had we both been teaching our own courses.

Assessment Benefits:  Working together on assessments was tremendously helpful. The drafting stage took somewhat longer because another set of eyes resulted in changes to the initial draft, but the collaboration unquestionably improved our efficiency when it came to grading.  Because we engaged in fairly extensive give and take in the exam drafting process, we avoided some of the drafting mistakes that sometimes lead to grading issues. Additionally, comparing how we were applying the rubric reassured us that our judgment calls made pedagogical sense and allowed us to more efficiently grade because we did not angst over the judgment calls.

Our Take Away

For the reasons outlined above, we believe that the collaboration improved our teaching, and hopefully improved our students’ learning. It also simply was fun to work together.

While collaboration amongst doctrinal faculty members may not be the norm, and may not be for everyone, we found it incredibly rewarding and we encourage others to experiment with it.

As a true testament to its value, we are doing it again next Fall.

The Power of “Not Yet”, Learning Outcomes & Assessment

Professor Carol Dweck talks about a powerful message:  “not yet”  https://www.ted.com/talks/carol_dweck_the_power_of_believing_that_you_can_improve?language=en.  Her studies demonstrate that when students understand that learning occurs on a continuum and they simply have “not yet” mastered a concept, they develop a “growth mindset” that leads to significant learning gains.

Professor Dweck notes that students with a “growth mind set” engage with the material and develop a passion for learning. They want to see how far they can push themselves. They realize they can improve and that they just have to figure out how to do so. This growth mindset actually engages neurons – a physiological process which paves the way to significant learning gains.

In contrast, she notes that students with a “fixed mindset” seek external validation of their self-worth via a “good grade”.  A fixed mindset causes students to run from failure rather than look at mistakes and failure as opportunities to learn. Students with a fixed mindset literally activate many fewer neurons than those with a growth mindset.

Professor Dweck emphasizes that a growth mindset involves understanding that you will be able to master a problem but you may need to work really hard, try new strategies and seek input from others when you get stuck – http://www.edweek.org/ew/articles/2015/09/23/carol-dweck-revisits-the-growth-mindset.html – all critical components of good lawyering.

Learning outcomes present the opportunity to create a growth mindset in ourselves and in our students. Learning outcomes remind us that our job is to facilitate student growth along the learning continuum.  They are a tool to help students learn how to think deeply about the processes and strategies necessary to tackle new material and challenges throughout their careers. They help students move from “not yet” to “I got this step, bring on a new challenge”.

The cycle of learning outcomes and assessment puts the growth mindset into practice. As educators, we identify the outcomes, gather and interpret evidence about achievement of the outcomes, and we use the evidence to modify our teaching to further improve student learning.  For both student and teacher, learning outcomes present an opportunity for intellectual engagement with the material as we strategize how to improve.

The growth mindset can also be incorporated into our formative assessments. These assessments allow students to see if they have mastered the material “yet” or if they need to work harder and  try different approaches.  Law professors can  use formative assessments to reward the effort and perseverance that lead to mastery of the material and in doing so, we can reinforce the concept of “you don’t have this yet, but you have the ability to figure it out.”

For example, in her doctrinal courses, Professor Sandra Simpson periodically posts a five question multiple choice quiz on TWEN. Each correct and incorrect multiple choice answer comes with an explanation.   She awards points toward the final grade when a student gets all five answers right. The kicker: a student can take the quiz as many times as he or she wants in order to get all five correct answers. This kind of assessment shifts the focus from the need to immediately get an “A” to the process of developing ways to identify the information and strategies needed to master the material in order to get an “A”.  It encourages the growth mindset.

Accreditation standards now require us to identify and measure learning outcomes and engage in formative assessments. When we do so, it is useful to keep in mind the power of “not yet” and  growth mindset principles.

Active Learning Levels the Playing Field

Studies find that active learning methodologies benefit all students, but the greatest benefits may be to women, minorities, low-income and first generation students. See : http://nyti.ms/1i6FRW5   As the NY Times reports, one study found “the active-learning approach worked disproportionately well for black students — halving the black-white achievement gap evident in the lecture course — and for first-generation college students, closing the gap between them and students from families with a history of college attendance.” The studies cited in this NY Times article provide support for increasing active learning in legal education, both to improve all students’ learning and to level the playing field.

A Survey Instrument for Cultural Sensibility Learning Outcomes

As law schools begin to grapple with identifying and measuring law student learning outcomes, cultural sensibility [a.k.a. cultural competence] should be on the learning outcomes list. A validated survey instrument has been developed to help measure some aspects of cultural sensibility learning: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2451300.  The instrument helps measure students’ understanding that we all have multifaceted cultural backgrounds, experiences, and biases that affect how we perceive and analyze legal problems and how we interact with clients and colleagues.

As lawyers, we must recognize and grapple with our own biases and stereotypes, as well as the influence cultural factors and systemic racism have had, and continue to have, upon the US legal system. As I note in a forthcoming Nevada Law Journal article: “While racial categories are artificial constructs, there is a long and ongoing history of real differences in the treatment and, therefore, collective experiences of “racial” groups. Those experiences influence how we perceive and assess facts, attitudes, legal problems and legal processes.”

An integral part of legal education involves developing law students’ abilities to identify their own cultural biases and helping future lawyers understand how those cultural perspectives and biases impact their legal analyses and interactions. There are many learning outcomes that contribute to law students’ cultural sensibility knowledge, attitudes and skills, many of which may be measured in various experiential learning and doctrinal courses.

The survey instrument measures some over-arching cultural sensibility learning outcomes, such as recognizing that: 1. one’s own cultural experiences affect how one views the legal system; 2. legal training in “rational thinking” does not insulate lawyers and judges from our own cultural biases; 3. subconscious cognitive processes hinder our ability to identify when we are acting based upon biases and stereotypes, and 4. we need to withhold judgment about others’ behaviors.

The survey instrument may be administered to students as they enter law school and shortly before they graduate. While we did not administer the survey to the same cohort of law students as they entered and then graduated, we did administer it to 309 entering law students and 281 upper level students. Amongst those students, we found that upper level students had a better understanding that one’s own cultural experiences affect how one views legal problems and interacts with clients. To the extent that cultural sensibility education requires that baseline understanding, the survey instrument is one way to measure some aspects of cultural sensibility learning.

At this June’s AALS Workshop on Measuring Learning Gains, Professor Raquel Aldana and I will continue the dialogue on how else one might measure cultural sensibility learning outcomes across the curriculum.

Consider Pre-testing

The recently passed ABA accreditation standard 302 requires schools to report student learning outcomes. A learning outcome has been defined as something a student can do now that she could not do before [or that she can do better than she did before].

One classic way to measure learning is to give pre-tests. When the class begins, students are tested on  key aspects of learning the professor hopes the students will achieve  during the semester. Pre-test results can be compared to end-of-course results to see if, in fact, students’ learning improved. They also can be used by professors to help identify students’ strengths and weaknesses at the outset and to adjust our teaching accordingly. UNM Dean David Herring’s work on measuring cross-case reasoning is an excellent example of how professors can use pre/post tests to measure learning and improve teaching.  papers.ssrn.com/sol3/papers.cfm?abstract_id=2387855

While pre-tests may provide learning outcome information, the more intriguing aspect of pre-tests is that they may, themselves, be a learning tool. A recent NY Times article reports studies indicating that pre-tests actually improve final exam performance. http://www.nytimes.com/2014/09/07/magazine/why-flunking-exams-is-actually-a-good-thing.html?emc=eta1

The studies’ authors have multiple theories about why pre-tests improve learning. First, they hypothesize that pre-tests help students identify how they will have to think about and synthesize the material. Students begin the course with that information in hand and it shapes their studying.

Another theory is that we suffer from “fluency illusion” – we believe that we truly grasp the material because we have read and highlighted. A pre-test exposes weaknesses in both knowledge and application.

Additionally, there are biological explanations for why pre-tests improve student learning. The brain works via developing networks of associations. Pre-testing primes the brain to develop associations for the material in the pre-test so that when it is later covered in class, the brain can more easily link the new information to existing information.

In the studies presented in the NY Times article, the pre-tests were particularly helpful with multiple choice test performance, and a key to improved performance was providing students with the correct information shortly after they had taken the pre-tests

The value of pretests may depend upon the type of course and the skills and knowledge tested. Yet the idea has intriguing possibilities. Would a pre-test before we covered hearsay improve student learning of that difficult topic? Would a course pre-test on reading/interpreting statutes result in better student performance of this skill at the end of the semester? Would providing 1Ls with a mock exam and an annotated model answer shortly after they began law school improve overall first year exam performance?

Data from other disciplines suggests pre-testing primes students to learn the material and it provides teachers with data we can use to see if the learning occurred. The value of pre-tests in legal education is an idea that certainly merits further study.

The most fun grading ever, really

The words “fun” and “grading” rarely appear in the same sentence.  However, my large section civil procedure and evidence students’ extra credit projects really are fun to grade.

 The Assignment

Students get extra credit if they develop a creative way to explain one of the more complicated concepts we cover in class.  Students receive the following instructions:

You may work alone, or in a team of up to four people, to create a video, comic book, song, game, poster or some other creative way to explain any one of the following concepts: relation back of amendments; work product; expert discovery; summary judgment; claim and/or issue preclusion [in Evidence the list includes hearsay, character evidence, expert testimony and impeachment]. You may choose to focus on particular aspects of these concepts or the entire concept. 

The project will be worth up to 6 raw score points toward your final raw score total. [The most raw score points available in the class toward a final grade, excluding extra credit, is 100].

Points for the extra credit project will be allocated based upon: creativity; content chosen and explained [i.e. if you take a very simple portion of a rule and explain that, you will likely not get many points]; demonstrated understanding of the applicable rule[s]; communication of the rules to other students. Your project will be part of the review session in the final class.

The Projects

About two-thirds of the class normally turns in a project.  The grades usually are between 4 and 6 points, although I have given some projects a 1 or 2.

Some students developed projects based on television shows or movies. For example, in civil procedure, students developed this game show video on work product

The student actors all imitated various faculty members – complete with wigs, hand gestures, and hair flips. Another group of students in civil procedure did a spoof on the Back to the Future movies to explain relation back.

Other students developed a twitter feed on work product.  Some students used board games for inspiration.  A group of civil procedure students produced an elaborate game entitled “Battleship Preclusion

bp photo

The next year, many of those students took my Evidence class and created a new game: “Escape from the House of Hearsay

 bp photo1

Others developed projects using music and poetry.  For example, in Evidence, students developed a song to help explain hearsay, “The Hearsay Saga of Johnny and Sue”:

This stanza from a poem on character evidence made me laugh:

Hope you enjoyed this poem, I sure had fun.
I hope character evidence no longer makes you want to run
Despite this poem I’m sure we will still all cram
So that we do not fail this godforsaken Evidence exam.

These are just a small sampling of the wonderful creative projects.

It’s Not Just Fun and Games

The Best Practices suggestion that doctrinal faculty use multiple methods to assess student learning [chapter 7] prompted me to develop this creative extra credit assessment.

The assessment has multiple learning objectives.  First, the projects require students to learn the material because they cannot communicate creatively if they do not fully grasp the underlying doctrine.  Anecdotally, when polled via anonymous clickers about the assignment, most said working on the project was either very, or somewhat, useful to their learning.

This assessment also seeks to develop students’ abilities to communicate complex material beyond how they would do so in class or on an exam.  Using different mediums to communicate information is a useful skill.

Additionally, this project allows students to express themselves creatively.  Creativity and innovation are amongst the Shultz/Zedeck lawyering effectiveness factors.

Finally, assessing students on what largely end up being visual presentations provides an opportunity to assess the students’ grasp of the material in a format that may be used by tomorrow’s lawyers to communicate information.

The Impact on the Final Grade

Why make it extra credit rather than required?  Although I believe the projects have educational value, I make them extra credit because some students get anxious at the idea of having to engage creatively with the material.  Also, I want students to have some degree of autonomy about where they spend their time and energy.

I have been asked if this type of extra credit project “changes the curve”.   Underlying that question is the assumption that the way we traditionally grade has a validity that may be skewed by a project such as this one. I question that assumption.

This project measures students’ ability to understand, and communicate that understanding, in a different, but not less valid, way than a multiple choice or essay exam questions.  In some ways, giving extra credit for these projects is analogous to giving class participation credit.

Additionally, those who do not participate presumably can use the time students spent on the projects to study the doctrine.  Thus, the non-participants at least theoretically might have a leg up in terms of the material to be tested via a traditional final.

The Take Away

As we explore ways to prepare our students for practice in tomorrow’s world, we should consider alternative ways to assess knowledge and communication skills, and we should encourage creativity and outside the box thinking.  These projects do that.  And, they are fun to grade.

Support for Empirical Research on Teaching/Assessment

The Need for Scholarship About Law Teaching and Learning

When we teach or assess differently, we often wonder: does this actually make a difference?  Does it improve student learning?  If so, does it improve all students’ learning, or does it only help a particular segment of students?  What do students think about this different methodology?  Does it motivate them to work harder and learn more deeply?    If my school has a “performance gap” between students of color and white students, or between men and women, do these different methods lessen or eliminate that gap?  All those questions, and more, can be explored empirically.  

SALT Website Identifies Resources For Scholarship, Especially Empirical Scholarship, on Teaching and Assessment Issues

One significant problem  is that few of us have had any social science training and thus we do not feel equipped to engage in an empirical scholarly exploration of the impact of our teaching or assessment.   Yet hard data is a very persuasive tool as we try to effectuate change.  For these reasons, The Society of American Law Teachers [SALT] has developed two resources for faculty members who want to engage in empirical research involving the “scholarship of teaching and learning.” 

 On the SALT website, go to “SALT at Work” and click on “Issues in Legal Education”.  There you will see a link to a list of academics who have agreed to be social science collaborators.  These academic collaborators will help you design, implement and interpret an empirical study involving law teaching and assessment issues. 

Also at the same website, you will find a  link to an “assessment working group” list serv.  That list serve  is comprised of legal academics who want to support each other in doing teaching and assessment research, especially empirical research.  It’s a great group of people who will help you work through issues in planning, developing or writing articles about your teaching and assessment methods, and is especially a good resource for those delving into empirical research.  

In sum, resources exist to help you as you begin to explore the scholarship opportunities that arise simply from your every day teaching and assessment work.

Assessments Requiring Reading Statutes Not Covered In Class:Lessons I Learned

I teach the second half of a year-long first-year civil procedure class.  We spend the semester reading and interpreting the Fed. R. of Civ. P.   Throughout the semester, I give the students hypothetical questions that require them to read and interpret the applicable Federal Rules.  This semester, I wanted to see if I could assess how well students learned how to read and interpet statutes.  The assessment was motivated, in part, by my desire to assess a broader range of skills than are normally covered in essay exams. 

As a small part of a 72-hour take-home exam, I gave students two short answer questions that required them to read Fed. R. of Civ. P that we had not covered in class and to answer two straightforward questions.  Using the fact pattern that served as the basis for the essay question, I asked students:  #1: Who must be served with this Answer?;  #2:  On what date must you file and serve your Answer? 

Much to my dismay, Continue reading