Guide to Writing Short Answer and Select-Response Items
Based on suggestions from your textbook and Haladyna (1999). Developing and validating multiple choice test items. Mahwah, NJ: LEA Associates and Measurement and Assessment in Teaching, Linn and Gronlund
Keep the test blueprint/spec in front of you as write items.
Short Answer Items
Word the item so that the required answer is both brief and specific:
Poor: An animal that eats the flesh of other animals is _____________.
Better: An animal that eats the flesh of other animals is classified as _____________. (carnivorous)
Do not use quotes taken from the book in short answer items:
Poor: Chlorine is a _____________.
Better: Chlorine belongs to a group of elements that combine with metals for form salts. It is therefore called a _____________. (halogen)
Use only one blank per item:
Poor: President _____________ emancipated the _______ in _____ .
Use direct questions:
Poor: John Glenn made his first orbital flight around the earth in _____________.
Better: In what year did John Glenn make his first orbital flight around the earth? ________. (1962)
Binary Choice (T/F)
Avoid items that use the words usually, sometimes, never, always, all, none:
Poor: T F The president of the US is usually elected to that office.
Avoid trivial statements.
Poor: T F The US declared war on Japan on December 7, 1941.
Avoid long, complex sentences. These items tend to test reading skills rather than content.
Avoid using opinion not attributed to some source.
Poor: T F Adequate medical care can best be provided through socialized medicine
Better: T F The American Medical Association favors socialized medicine as the best way to provide adequate medical care.
Matching Items
Use only homogeneous material in a single matching exercise:
Include an unequal number of responses and premises, and instruct the student that responses may be used once, more than once, or not at all.
Keep the items to be matched brief, and place the shorter responses on the right.
e.g.,
___ 1. Boston Tea Party | A. 1765-1769 |
___ 2. Battle of Lexington | B. 1775-1779 |
___ 3. Repeal of the Stamp Act | |
___ 4. First Continental Congress |
Indicate in the directions the basis for matching the responses and premises.
Multiple Choice Items
The "stem" of the item should be meaningful by itself and should present a definite problem:
Poor:
1. South America
is a flat, arid continent
imports coffee from the US
has a population less then Australia
was settled mainly by colonists from Spain.
Better: Most of South America was settled by colonists from
England
France
Australia
Spain.*
The "stem" of the item should include as much of the item as possible and should be free from irrelevant material:
Good:
1. Spanish colonists settled most of South America in search of
adventure
wealth
lower taxes
religious freedom
Avoid negatively stated stems
Poor:
1. Which of the following states is NOT located north of the Mason-Dixon line
Maine
New York
Penn
Virginia *
All of the alternatives should be grammatically consistent with the stem of the item.
Good:
1. An electric transformer can be used to
store electricity *
increase the voltage of alternating current
convert electrical energy into mechanical energy
change alternating current to direct current
Items used to measure understanding should contain moderate novelty.
All of the distracters should be plausible.
Poor:
1. Which explorer first landed at the North Pole?
Roberty Peary *
Robert Peebrain
Your teacher
Marco Polo
The relative length of the alternatives should not provide a clue to the answer
"All of the above" and "None of the above" should be used sparingly .
More Suggestions
Keep the specific content of items independent from one another.
The following questions come from the story "Stones from Ybarra."
Who was Lupe's best friend?
Kate
Dolores
Roxie*
Who was always quarreling with Lupe?
Kate
*Sara
Roxie
It is easy to see that Roxie can't be correct in both
items.
Avoid overly specific and overly general knowledge.
Who wrote the Icon of Seville?
Lorca
Ibanez
Rodriguez
Avoid opinion-based items.
Unqualified open item:
What is the best comedy film ever made?
Abbott and Costello go to Mars
Young Frankenstein
A Day at the Races
Qualified opinion item:
According to American Film Institution, what is the greatest American film?
It Happened One Night
Citizen Cane
Gone With the Wind
Star Wars
Avoid Trick Items
Is there a fourth of July in England?
Some months have 31 days. How many have 28?
How many animals of each gender did Moses bring on his ship?
A butcher in the meat market is 6 feet tall. What does he weigh?
Avoid complex multiple-choice formats.
Which of the following words describes you?
1. Kind
2. Considerate
3. Easy on the eyes
1 only
2 only
2 and 3
1,2,3
Keep your tests as pure a measure of achievement as possible. .
Format the items vertically instead of horizontally.
Below is a vertically formatted item:
You draw a card from a deck of 52 cards. What is the chance you will draw a card with an odd number on it?
36/52
32/52
About one half
Here is an item that is formatted horizontally:
You draw a card from a deck of 52 cards. What is the chance you will draw a card with an odd number on it?
A. 36/52 B. 1/2 C. 1/3 D. Not given
Edit and proof items.
Keep vocabulary simple for the group of students being tested.
Use correct grammar, punctuation, and spelling
Minimize the amount of reading in each item.
Use either a question stem or a partial sentence.
An example in the question format is:
Which retirement investment is most recommended by investment counselors for young investors?
U.S. Bonds
Certificate of Deposits
C.* Fast growth stock funds
An example in the partial-sentence (completion) format is:
Regarding aging people, in other modern societies,
Programs for the elderly are generally a low priority item.
Programs for the elderly are more advanced when compared to the United States.
Programs for the elderly are about the same as in the United States.
The elderly have been shown to have different needs, based on cultural differences.
Don’t use these "fill in the blank" type items.
The __________ format is the best way to format a MC item.
completion
question
fill-in-the-blank
Ensure that the directions in the stem are very clear.
Unclear directions in the stem:
Regarding gravitation:
Clearer directions in the stem:
Which of the following best shows the concept of gravitation?
Avoid Window Dressing (Excessive Verbiage).
Poor:
High temperatures and heavy rainfall characterize a humid climate. People in this kind of climate usually complain of heavy perspiration, even moderately warm days seem uncomfortable. Which climate is described?
Savanna
*Tropical rainforest
Tundra
Better:
Which term below describes a climate with high temperatures and heavy rainfall?
Savanna
Tropical rainforest
Tundra
Word the stem positively; Avoid negative phrasing when possible
Which is NOT an advantage of the MC format over the CR format for measuring knowledge?
Better sampling of content
Greater reliability of test scores
Ease of construction
Which are benefits of MC testing? Mark A if true or B if false.
Greater potential for content sampling (A)
Greater potential for high reliability (A)
More appropriate for measuring aspects of performance (B)
Objectively scored (A)
Able to measure higher level thinking (A)
Subject to guessing (A)
Free of bias (B)
Use as many distractors as possible but three seems to be a natural limit.
Make sure that only one of these choices is the right answer.
Vary the location of the right answer according tot he number of options.
Place options in logical or numerical order.
Wrong:
What is the cost of an item that normally sells for $9.99 that is discounted 25%?
$5.00
*$7.50
$2.50
$6.66
Right:
What is the cost of an item that normally sells for $9.99 that is discounted 25%?
$2.50
$5.00
$6.66
*$7.50
Keep choices independent; Choices should be mutually exclusive.
Wrong:
What age range represents the physical "peak" of life?
11-15
15-19
19-25
25-32
Avoid using the choices None of the Above, All of the Above.
Phrase choices positively.
Avoid various clues to the right answer, such as
Clang associations
Who were known as the Magnificent Seven?
A touring softball team.
*A group of seven cowboys.
A rock and roll group.
Grammatical inconsistencies
The most effective strategy in playing winning tennis for a beginner is:
hit with more pace.
*to keep the ball in play.
volley at the net as often as possible.
Blatantly absurd, ridiculous options.
Who is best known for contributions to electronics?
Comedian Jay Leno
*Robert Sveum
Actor Bruce Willis
Make all distractors plausible.
Use common errors of students.
Avoid the use of humor.