Make sure that you are in the subskill that you need to add an item to, then press “Add Challenge” (We currently share an interface with practice, which uses the term challenge instead of item).

From the menu select “Blanks” (note the description does not correctly reflect assessment formats used by this item).

You will then see something similar to this:


This is where you write the context and stem for the item, in markdown. If you want to include a preview of the data, we recommend that you format this as a markdown table. The preview does not typically need to include more than around 5 rows of the data. Be mindful of the number of columns. Remember that test takers will never see the full data so you can keep it quite small.

Pre-challenge code

In this section you should include any code that is needed for your item but does not need to be seen by the test taker. This could be loading packages, importing data, defining the database (SQL), performing any pre-cleaning, model fitting (e.g. for modeling items focused on metrics).

Code 1

The next thing you will probably do is jump to the bottom and write the code for the item. We recommend that you keep this short and related to the point being tested. Only include things that are relevant. An example would be including a library call so the test taker knows it is available to them but not including clean that we happen to have done to make the test item possible.

Within your code block you will need to include an encoding for the section that will be left blank. Let’s suppose we are writing an item where the correct measure of center has to be selected. It may look something like this:


We need to give a label name to the blank. You could give this blank a helpful name so that you can easily identify it later, but there are some reserved words in the build system that may lead to a build failure. To make things easier for ourselves, we have developed a habit of sticking to `expr1`, `expr2` etc. You don’t need to follow this convention but be aware you may have unexpected build failures caused by the naming.

The label itself is enclosed in double curly brackets and proceeded by an underscore. This looks strange but will become second nature to you.


Now that we have the code defined and the label names, we need to define what the label should be filled with. We do this in the variables section. If you have left more than one blank they will all be defined in the same variables block. The variables block is written in markdown, so code needs to be inside tick marks. Below is how we would format the example above.


- 'mean'


For a fill in the blanks item, this is where we define the alternative options that will be shown to test takers. Many of the same guidelines as for multiple choice distractors apply here, the options should be plausible for example. The one that doesn’t apply is the ordering. This is handled by the build so you don’t need to worry about it here. You define distractors in much the same way as the variable itself, just with additional bullets.


- 'median'

- 'mode'

In this example we have only included two distractors, as they are the only plausible alternatives. But just like multiple choice, if there are three you can include them.

Once the distractors are defined, you are done. You can preview the item using the preview button in the bottom right corner.

We recommend that you preview before saving. Once you save it will trigger a build that will fail if you haven’t resolved issues first. In general you can pick up issues in preview first.

Did this answer your question?