The first and perhaps most important recommendation for constructing exams is to consider what should be on it before you start writing. What content should the exam cover? What skills should students demonstrate? What fraction of your test should require higher order thinking, and what fraction should measure basic knowledge? Show To clarify the answers to these questions, it is good practice to construct a table of specifications (aka, a test blueprint) to guide your test writing (Brame, 2019; see chapter 12). A test blueprint characterizes the distribution of points across content areas and types of thinking you want the test to target. It can be easiest to start with a distribution of points across content areas. The table shows one way to do that, with the number of points corresponding to the amount of time spent in class.  The biggest benefit of a test blueprint is that it can help you stay on track as you write the test, making sure that you test the content and ways of thinking that matter to you. It can also be a good way to communicate with your students. Sharing the test blueprint–or an abbreviated form of it–with them can help them prepare and reduce their uncertainty about what to expect. Include opportunities for partial credit that are efficient to gradeQuestion clusters. In a question cluster, students are given an initial prompt, such as a graph or a scenario, and are then asked multiple short questions about the prompt. These questions can be MC or short answer; the key is that each has an answer that students can provide independently of other questions and that you can grade efficiently. Two examples are given below.
When creating question clusters, it’s important that mistakes students make on the first or second question don’t result in all incorrect answers on later questions. Ideally, the questions in a cluster should be independent of each other; if that’s not possible, it’s wise to grade the later questions based on the first false assumption (while clearly identifying to students the problematic thinking on the earlier question). Multiple true/false. Another way to create opportunities for partial credit is to use “choose all correct” questions, which might more accurately be regarded as multiple true/false items. That is, the student can evaluate each response as a true or false response to the stem, and you can grade it as such. Thus a student could get from 0-5 points for a question with 5 alternatives. These items have additional benefits: they are often easier for the instructor to write, and Brian Couch and colleagues have also shown that they more effectively reveal partial understanding. Two examples are shown, one from a biochemistry course and one from a physics course. Multiple choice questions with more and less correct distractors. Some faculty provide opportunities for partial credit in multiple choice questions by offering distractors (or incorrect answers) that are more and less correct. For example, if the one best answer for a MC question is a particular dehydrogenase, then a distractor that named another dehydrogenase might be worth partial credit, whereas a distractor that named a different type of enzyme might be worth no credit. In another example, if the question required computation, then a distractor that changed the sign might be worth partial credit, whereas a distractor that had the wrong digit might be worth no credit. . Sequence questions intentionallyExam questions can be sequenced in the order in which topics were discussed in class, by difficulty, or randomly. While a recent review of the literature on psychology exams indicated that ordering did not impact overall student performance, it did find that some studies reported cognitive and/or affective benefits of intentional question sequencing (Hauck et al., 2017). Specifically, ordering questions by lecture sequence can improve student performance, and sequencing questions from easier to harder can improve student confidence during and after test-taking (Weinstein and Roediger, 2010; Jackson and Greene, 2014). To improve student experience during test-taking, it therefore makes sense to choose an intentional order to present exam questions.
Use good practice in question writingOf course, aligning the exam with what you have taught and intend to assess is only half the battle; the other half is writing questions that are valid measures of your students’ knowledge. If you use multiple choice questions, the key is to write questions that 1) don’t give unintentional clues to the correct answer, and 2) that assess the desired skill and knowledge and not something else, such as working memory or non-target vocabulary. This guide provides detailed guidance, but in essence, you can follow these rules : The stem (that is, the question or prompt) should
The alternatives should
It’s important to avoid complex multiple choice (e.g., A only; A and B; A and C; A, B, and C). “Choose all correct” questions that essentially function as multiple true/false can have the same benefits without the excessive demand on working memory. It’s also important to know that the number of alternatives can vary among items; there is little difference in difficulty, discrimination, or reliability among items with three, four, or five alternatives. For free response questions, Hogan and Murphy provide a few recommendations to maximize the effectiveness of constructed response items (Hogan and Murphy, 2007).
|
