Best Practices for Legal Education

Formative feedback is giving students information about their performance to help them improve.  Summative assessment is an evaluation of the student’s performance.   It is useful in determining whether the student as achieved the skill or level of knowledge.  Criteria referenced assessment measures whether a student as achieved a certain level of knowledge or skill. Norm referenced assessment compares students performance to each other.  An example of this is when a teacher ranks student’s performance and then “curve” the grades.   Outcome based assessment focuses on ensuring that students acquire and demonstrated a set of core skills and knowledge.  All of this is very well explained and developed in the Carnegie Report and Best Practices.  So, what does this mean in practice? 

Through the collaboration with the UNM Medical School, I had the opportunity to learn a little bit about how the medical school uses criteria-referenced outcome based assessment to assess whether residents have achieved the basic competencies and skills expected of a physician.  As we were designing the problems, the doctors evaluated the problems and identified the specific knowledge, skills and behaviors that they expected the residents to demonstrate.  This ranged from a simple but very important behavior such as washing hands before an examination, to asking about certain symptoms when presented with a particular complaint and ordering the appropriate and relevant tests.   Appropriate demeanor and communication skill was required.  The physician should explain treatment options and get appropriate consents.  Appropriate documentation of the encounter, with an expected “charting” of the patient’s file was required.  I was surprised by how detailed the assessment process can get, and impressed with the easy way in which the doctors were able to identify the expected criteria. 

I had no trouble identifying the substantive issues  that I thought the students should address in the encounter with the standardized client with a domestic violence problem.  They should identify the domestic violence issues as well as the related substantive issues.   I also wanted them to conduct an effective interview and gather sufficient information to be able to counsel the client appropriately.  I felt that it was harder to articulate some of the other criteria that I thought would demonstrate competent performance.  I expected the evaluation to be a challenge as well.  I wanted the students to explain various options to the client, including information about services and shelters as well as the legal options.  And, I wanted them to make the client comfortable enough with the student to want to come back.  

We could evaluate the students’ identification of the issues by having the students write a memo about the encounter.  We could evaluate the interviewing and counseling skills using a combination of a check list and observations of the video.  What was new to me was the ability to use a standardized evaluation of whether the student made the client feel comfortable enough to come back.  We could do this by asking the standardized patient to evaluate the students’ communication skills and affect.   And, the medical school had the capability to assist with all of those measures.  Once I developed a check list, they could use it to have the standardized client check it off.  They used their communication skills measure to have the standardized client evaluate how effectively the student was at communication and even how comfortable the client was with the student.  And, they videotaped each encounter.  They gave me a DVD of the interview and our IT department was able to put it online so that each student could review their own performance and I could also look at the video.  I did not have time to schedule a review of the tape with each student, the group was simply too large, but that would have been helpful in providing the student with feedback.  The Assessment and Learning Department generated a report on each interview detailing the standardized patient’s assessment and collected all of the data so that we could evaluate it multiple ways!    The resources were amazing!  I was overwhelmed with the possibilities for using these tools to help students improve their knowledge and skills, but I was also concerned about the expense.  The medical school put a lot of resources into the standardized patient program!

The medical residents will have to take “Boards” (their certifying and licensing exams) which include an assessment based, in part, on performance with standardized patients.   And, the Liaison Committee on Medical Education (the accrediting body of the American Medical Association) requires medical schools to “develop a system of assessment which assures that students have acquired and can demonstrate on direct observation the core clinical skills and behaviors needed in subsequent medical training”…HMMMMMM…

 Next post I will talk about the value of getting medical residents and law students to talk to each other.

Filed under: Best Practices, Outcomes & Assessment Techniques, Catalysts For Change |