Learning from Medical School—Formative Feedback, Assessment and Resources

Formative feedback is giving students information about their performance to help them improve.  Summative assessment is an evaluation of the student’s performance.   It is useful in determining whether the student as achieved the skill or level of knowledge.  Criteria referenced assessment measures whether a student as achieved a certain level of knowledge or skill. Norm referenced assessment compares students performance to each other.  An example of this is when a teacher ranks student’s performance and then “curve” the grades.   Outcome based assessment focuses on ensuring that students acquire and demonstrated a set of core skills and knowledge.  All of this is very well explained and developed in the Carnegie Report and Best Practices.  So, what does this mean in practice? 

Through the collaboration with the UNM Medical School, I had the opportunity to learn a little bit about how the medical school uses criteria-referenced outcome based assessment to assess whether residents have achieved the basic competencies and skills expected of a physician.  As we were designing the problems, the doctors evaluated the problems and identified the specific knowledge, skills and behaviors that they expected the residents to demonstrate.  This ranged from a simple but very important behavior such as washing hands before an examination, to asking about certain symptoms when presented with a particular complaint and ordering the appropriate and relevant tests.   Appropriate demeanor and communication skill was required.  The physician should explain treatment options and get appropriate consents.  Appropriate documentation of the encounter, with an expected “charting” of the patient’s file was required.  I was surprised by how detailed the assessment process can get, and impressed with the easy way in which the doctors were able to identify the expected criteria. 

I had no trouble identifying the substantive issues  that I thought the students should address in the encounter with the standardized client with a domestic violence problem.  They should identify the domestic violence issues as well as the related substantive issues.   I also wanted them to conduct an effective interview and gather sufficient information to be able to counsel the client appropriately.  I felt that it was harder to articulate some of the other criteria that I thought would demonstrate competent performance.  I expected the evaluation to be a challenge as well.  I wanted the students to explain various options to the client, including information about services and shelters as well as the legal options.  And, I wanted them to make the client comfortable enough with the student to want to come back.  

We could evaluate the students’ identification of the issues by having the students write a memo about the encounter.  We could evaluate the interviewing and counseling skills using a combination of a check list and observations of the video.  What was new to me was the ability to use a standardized evaluation of whether the student made the client feel comfortable enough to come back.  We could do this by asking the standardized patient to evaluate the students’ communication skills and affect.   And, the medical school had the capability to assist with all of those measures.  Once I developed a check list, they could use it to have the standardized client check it off.  They used their communication skills measure to have the standardized client evaluate how effectively the student was at communication and even how comfortable the client was with the student.  And, they videotaped each encounter.  They gave me a DVD of the interview and our IT department was able to put it online so that each student could review their own performance and I could also look at the video.  I did not have time to schedule a review of the tape with each student, the group was simply too large, but that would have been helpful in providing the student with feedback.  The Assessment and Learning Department generated a report on each interview detailing the standardized patient’s assessment and collected all of the data so that we could evaluate it multiple ways!    The resources were amazing!  I was overwhelmed with the possibilities for using these tools to help students improve their knowledge and skills, but I was also concerned about the expense.  The medical school put a lot of resources into the standardized patient program!

The medical residents will have to take “Boards” (their certifying and licensing exams) which include an assessment based, in part, on performance with standardized patients.   And, the Liaison Committee on Medical Education (the accrediting body of the American Medical Association) requires medical schools to “develop a system of assessment which assures that students have acquired and can demonstrate on direct observation the core clinical skills and behaviors needed in subsequent medical training”…HMMMMMM…

 Next post I will talk about the value of getting medical residents and law students to talk to each other.


2 Responses

  1. It sounds like the medical school is a great resource and is using some sophisticated tools. I think the review of video that is described sounds similar to the technology developed by Larry Farmer and Gerry Williams and now available through CALI as Medianotes. You can build the criteria referenced evaluation points into the system to assist and speed up the review process, and the performances are uploaded and available to the student and the faculty member and possibly to other students for peer reviews. I have to say that I see these tools as building minimum competencies and there is the risk of too much standardization–after all, our clients and their problems are not standardized.

    I also compare it somewhat to IRAC and similar formulaic structures for legal analysis and writing–these are useful to get all students to a basic point but run the risk of having students see them as ceilings rather than floors and limit their creativity and ability to find personal voice. That said I don’t mean to suggest that they should not be used; only that they be contextualized within the overall goals of the law school program.

  2. Hi Bob, I think you are absolutely right, we want the objectives to be guidelines and leave room for the art of the practice. Real life is always messier, more complicated and fraught with changes. The practicioner has to know the basics, but become facile enough to adjust for the circumstances. I agree that these tools are just about teaching (and practicing) the basics. I liked that the students got to practice an interview with an actor before meeting a real client. And, I think it makes sense to assess some sort of proficiency with the basics before moving on… Great to have you join the best practices blog. I look forward to more of your insightful posts!


Comments are closed.

%d bloggers like this: