Skip to main content
All CollectionsAssessmentsGeneral Assessment Information
Item Writing Guidelines
Item Writing Guidelines

Assessment Item Guidelines

Martine Holland avatar
Written by Martine Holland
Updated over a week ago

Guiding Principle:

A good test item can be answered by a qualified candidate by reading only the context and stem. If more information is needed, the item probably needs revising.

The following guidelines are based on those presented by Rodriguez, M.C. (2016). They have been adapted to apply to DataCamp’s specific use and format of assessment.

Content

  1. Each item should test only one concept as specified by the associated learning objective

  2. Items should not use content from DataCamp course/practice/project materials, new materials should be used to test higher-order thinking

  3. Items should be independent of one another

  4. Items should test important concepts and not overly specific content e.g. the difference between two very similar functions should not be tested

  5. Avoid opinions

  6. Avoid trick items

Style

The following are additions to DataCamp’s style guidelines which should also be followed.

  1. Linguistic complexity should be kept appropriate to the level of the test takers

  2. Minimize the amount of reading in each item, avoid window dressing

  3. Items should be based around a real life-situation to test test higher-order thinking

Writing the stem

  1. State the central idea clearly and concisely in the stem and not in the options

  2. Word the stem positively, avoid negative phrasing

Multiple-Choice Item Guidelines

  1. Use only options that are plausible and discriminating. There should be four options unless there is a strong argument for more or fewer options.

  2. Broadly speaking, a good question can be answered without looking at the options. A question that needs the candidate to read and choose between options is often not tightly enough aligned around a learning objective, and risks testing reading skills rather than subject knowledge.

  3. Ensure that only one of the options is correct

  4. Vary the location of the correct answer

  5. Give options in logical or numerical order

  6. Keep options independent. They should not overlap

  7. Avoid the use of “none-of-the-above”, “all-of-the-above” and “I don’t know”

  8. Word options positively, avoid negative words

  9. Avoid giving clues to the right answer

    1. Keep the length of options roughly equal

    2. Avoid specific determiners (e.g. always, never)

    3. Avoid pairs of triplets of options that clue the test taker to the correct choice

    4. Avoid blatantly absurd or ridiculous options

    5. Keep options homogeneous in content and grammatical structure

  10. Make all distractors plausible. Common mistakes and misconceptions can be used to write good distractors

  11. Avoid the use of humor

Fill-in-the-blank item Guidelines

  1. Use only options that are plausible and discriminating, three options are sufficient, four are preferred

  2. Ensure that only one of the options, or combination of options, is correct

  3. Only leave blanks that test the learning objective.

Typing item Guidelines

  1. Ensure that all possible correct solutions will be accepted as correct, test takers should not be penalized for use of an equally correct answer

  2. The expected output should always be present.

  3. Ensure that no answer, a partial answer, or the wrong answer will always return an incorrect answer

  4. Ensure that the correct answer is not revealed in the output code e.g. a model formula isn’t printed in the output

  5. The expected output should be consistent. Do not include non-deterministic elements that would mean the expected output would vary.

Graphics in items

  1. Avoid reference to specific colors (e.g. “The red line shows the median”)

Reference

Rodriguez, M.C. (2016) ‘Selected-Response Item Development’ in Lane, S (ed), Raymond, M.R. (ed), Haladyna, T.M. (ed) Handbook of Test Development 2nd Edition, Routledge, pp259-273

Did this answer your question?