A good place to start is with the questions, “Why do we assess?” and “What makes assessment in professional school different from what many students have experienced previously?” The upcoming section is intended to address these questions and give guidance on a few of the assessment types DDS students experience during their first year at USCSD.
In a nutshell, the assessment program at USCSD is intended to monitor student progress, positively encourage the learning process, motivate learning, and help students develop skills of self and peer assessment. The combination of assessments a student takes while in dental school helps determine whether he/she has achieved the competencies required of a new dentist. To understand why and how the assessment program is organized it’s helpful to consider a few key elements about assessment first.
Assessments are generally designed to ‘measure’ or ‘test’ things that are valued in a curriculum (Shumway and Harden, 2003) Assessment Ch in Fry, 2000). The dental curriculum, in turn, is designed to reflect what is desired in a new dentist. These factors influence what and how things are taught and learned, and later, how they are assessed. When learning outcomes (i.e. the products of learning), teaching and learning methods and assessment methods complement one another this is called ‘constructive alignment’. If for example, decision-making is important for a dentist, teaching and learning activities should encourage a student to develop decision-making ability as it applies to practicing dentistry. It follows then that an assessment that measures decision-making should be built into the assessment program. Such constructive alignment is important in a curriculum because it can help ensure that the curriculum is being addressed holistically and that the assessment is measuring valued things, in this case what is valued in a new dentist. It is also important to strive for alignment because assessment is known to drive the learning process by influencing the approach students take to learning. As an example, if you teach students about decision-making by letting them practice making decisions but then assess them using a multiple-choice exam they may alter their approach to learning about decision making, perhaps taking a superficial rather than deep approach.
Some assessment tools used in the USCSD program may be familiar to students from their undergraduate or graduate studies. Other assessment formats may be quite novel and will require time to get used to (Table 1). A couple of these assessments may be so novel to students that they will get detailed instructions on how to do the assessment before it is done for the first time.
Table 1: Examples Assessments in Undergraduate Education and USCSD
One thing that may be striking looking at the assessments types in the “Shows How” and “Does” rows is that they might be challenging to do because they seem to be testing a broader range of things. They’re not just testing things around acquiring knowledge. Some of these assessments, in fact, are designed to look at an integration of knowledge and skills and attitudes. Other questions that might arise are: “What will be graded with these assessments?” and “How will they be graded?” In most instances, faculty members will use a form or a checklist to assess student performance. This form will be based on criteria that spell out what type of behavior or performance is expected.
Measurement of performance isn’t as black and white as marking an MCQ exam which is why criteria are developed as a benchmark for the assessor. Still, there may be some minor differences in how the criteria are applied by different individuals. Despite its measurement challenges, performance-based assessment has particular strengths, it measures knowledge and skills that are more akin to those a health care professional does in practice and two, it often comes with an opportunity for a student to receive feedback from the instructor. Paraphrasing a measurement-related quote from Albert Einstein, “All that is easily measured is not worth measuring. All that is worth measuring is not easily measured.”
In the first year of dental school assessments will be comprised of the following: PBL Process Grades, Triple Jump Exercise, MCQ Examinations, COMBOT Examinations, Pre-Clinical/Clinical Examinations. Figure 1 illustrates how learning activities are linked to the various assessments. Many of these assessments will be accompanied by some form of feedback to help the student improve his/her performance. Of equal importance, the results of these assessments will feed into the four courses in which the student is enrolled, ultimately determining his/her course grades. The rationale, timing, associated learning activities, course links and further guidance for these assessments can be found in Table 2. Further guidance for the various assessments follows.
Figure 1: Diagram of learning activities and their respective assessments 
Table 2: Year 1 DDS Assessment Program Description
Assessment Tool |
Tool measures |
When Administered |
Associated Learning Activity |
Course Links for assessment |
Further Guidance (links) |
PBL Process Assessment |
The group, reasoning, learning & feedback skills used to work on a PBL case. Assessed by student, peers & facilitator. |
On completion of each case |
PBL Cases- primarily the learning processes engaged in. |
S, F, B, C (weighted by case emphasis) |
|
Triple Jump (T3) |
Student ability to use PBL process to analyze and present a case independently. Skills primarily a mix of reasoning and learning. |
Midterm and final examinations (cases used unlikely to be similar to those studied with group) |
PBL Cases- primarily the learning processes engaged in. |
S, F, B (all Triple Jump cases are biomedical) |
|
MCQ |
Student acquisition of knowledge in subject areas related to the cases worked on. Usually 30-50 items per case studied. 3-5 cases typically studied between exam periods. |
Midterm and final examinations (not comprehensive over trimester) |
PBL Cases- content derived here |
S, F, B, C. Weighted by case emphasis both w/respect to number of items on exam and thus contribution to exam score. |
|
COMBOT |
As above but with fewer items/case. COMBOT items are MCQ items associated with an image. Image is required to answer MCQ. Administered via computer. |
Midterm and final examinations (not comprehensive over trimester) |
PBL Cases- content derived here |
S, F, C. Weighted by case emphasis both w/respect to number of items on exam and thus contribution to exam score. Behavior areas not amenable to getting testing images. |
|
Pre-Clinical/ |
Varies depending on nature of clinical activity (e.g. preclinical laboratory vs clinical rotation) |
Midterm and final examinations but clinical instructors may administer more according to need |
Clinic/preclinical rotations, PBL cases secondarily |
C (great majority) |
See clinical instructors |
Clinics- Daily Clinical Evaluation |
General daily performance with respect to clinical activities. Self-assessment and instructor assessment of same performance |
Daily (or close to daily) |
All learning activities contribute, but especially those experienced in the clinic |
C |
Abbreviations used: S-Human Structure Course, F- Human Function Course, B- Human Behavior Course, C- Clinical Dentistry Course. For further descriptions on DDS Course Descriptions please refer to the USC Course Catalogue and/or the USCSD Curriculum at a Glance document.
The facilitator is responsible for providing each student with an assessment of his/her PBL Process skills after each case. The facilitator will base this assessment on observations of the student’s behavior during the case. The facilitator should use the same PBL Process Guidelines that the student and his/her peers use to assess themselves. The facilitator should deliver his/her PBL Process assessment to each student during a one-to-one feedback session. The student may or may not receive the “score” along with this feedback. The primary purpose of the feedback session is to have a constructive discussion about the student’s strengths and weaknesses related to PBL Process and to identify specific actions that could be taken to improve those skills in the future.
During the first year of dental school each student will complete the Triple Jump exercise at midterm and final exams for each trimester, a total of 6 times. The Triple Jump (T3) assesses the student’s ability to analyze, gather information and apply knowledge to a ‘problem’ or patient case. The goal of the T3 exercise is to advance one’s understanding of a patient problem utilizing PBL process procedures that were modeled and reinforced in the small-group learning environment. As each student works independently, only skills related to reasoning and learning come into play.
Each student obtains a unique problem case and is given a fixed amount of time to independently analyze the case utilizing PBL Process, usually from the afternoon to the morning of the following day. During the initial analysis the student will identify relevant facts, generate ideas relevant to the facts, and identify topics to research. During the second ‘jump’ of the exercise, the student will research the topics and apply that information back to the case and the original ideas. During the final ‘jump’ of the exercise the student will present his/her approach to analyzing the problem and current understanding of it to a pair of faculty assessors. The final score is an average of the two faculty members’ assessment results.
Again, please note that advancing the understanding of the patient problem is the goal of the exercise. Determining the definitive diagnosis for the patient is not the primary goal. Through applying one’s research back to the case the student will likely come up with a selection of diagnostic possibilities, some of which have more compelling support than others. For this stage in the DDS program the student will not be expected to have sufficient knowledge to make a definitive diagnosis. Furthermore, the cases used in the Triple Jump will typically be missing key diagnostic information that would allow one to come to a concrete conclusion. Other cases may even state the diagnosis outright! Another key consideration when completing the T3 is that the time needed to complete the T3 is limited relative to working on a regular PBL case. The student also has the responsibility to do the work on his/her own. Both of these factors require that the student use time carefully. To do this the student will need to think carefully about PRIORITIZING during each stage of the exercise. Think quality not quantity!
Before taking the Triple Jump for the first time, an orientation to the exercise is arranged for the DDS class. Guidance on how to approach the T3 exercise as well as the criteria used to assess performance can be found by going to the link below.
The P3 is an assessment that replaces the Triple Jump in the second year of the curriculum. Although this information will be more relevant during the second year, knowing what is expected in the P3 may help toward understanding what is expected in the T3. As with the Triple Jump, the P3 is administered at midterm and final exams and contributes to the Human Structure, Function, and Behavior courses. While solving the problem is not the objective in the T3, it is the primary objective in the P3. For the P3, each student is given a case to analyze and a specified amount of time to conduct research. The student is expected to arrive at a differential diagnosis that can be defended based on the facts provided in the case. The student’s ability to arrive at a defensible differential diagnosis is related to reasoning skills and ability to draw on the knowledge base developed over the past year. The P3 cases are a mix of biomedical and dental cases during the first and second trimesters. The P3 is a written exercise and is marked by expert faculty using a marksheet and criteria.
The MCQ Exam- the Multiple-choice Question Exam
By this point in one’s academic career, a student has probably taken a number of exams featuring multiple-choice questions. What’s important to consider for MCQ exams in a PBL course is that it might not seem as obvious what will be on the exam. The question of “Will this be on the test?” is not new or limited to PBL courses. It may be the most commonly uttered question posed by students. In a PBL course where the students make more decisions about what gets learned from the cases, they may feel more anxious about knowing the answer to this question.
It may help to know that the types and number of MCQ items on the exam are related to the learning topics built into each of the PBL cases. If the group works on a case, for example, that features a patient with a kidney infection the students will likely end up researching topics related to kidney anatomy, histology, physiology and some pathology. Because the case was directed toward an infectious cause, the group would probably also emphasize topics related to microbiology, immunology and perhaps pharmacology if antibiotics had been mentioned. The questions selected for the MCQ would be linked to similar topics and the weight placed on each topic (e.g. the number of items) would be related to how much emphasis was built into the specific cases covered on the examination.
All MCQ (and COMBOT) items are categorized at several levels. This categorization scheme is drawn directly from the NBDE examinations system and reflects the key subject areas considered of importance in dental education. The USCSD PBL program also utilizes this categorization scheme in the Outcomes Assessment process, where the student groups map the major and minor topics studied against this category outline. The NBDE category outline can be found here.
The same overall considerations for the MCQ exam apply to the COMBOT exam, namely the types and numbers of items will be based on the emphasis of the cases under study. But because the COMBOT items are images linked to an MCQ, this exam may seem a little different. Below is an example item to demonstrate what a COMBOT question looks like.
Which of the following labeled vascular structures would most likely contribute to the formation of esophageal varices?
1. Structure 1
2. Structure 2
3. Structure 3
4. Structure 5
5. Structure 8
The OSCE –Objective Structured Clinical Examination
The OSCE is an assessment of the student’s ability to apply the clinical knowledge and skills that have been acquired through cases, pre-clinical and clinical experiences. There can be multiple OSCEs given in a trimester and at times outside of scheduled exam week. These exams generally coordinate with the pre-clinical sessions and clinical rotations.