Make sure that you are in the subskill that you need to add an item to, then press “Add Challenge” (We currently share an interface with practice, which uses the term challenge instead of item).

From the menu select “Blanks” (note the description does not correctly reflect assessment formats used by this item).

You will then see something similar to this:


This is where you write the context and stem for the item, in markdown. If you want to include a preview of the data, we recommend that you format this as a markdown table. The preview does not typically need to include more than around 5 rows of the data. Be mindful of the number of columns. Remember that test takers will never see the full data so you can keep it quite small.

Pre-challenge code

In this section you should include any code that is needed for your item but does not need to be seen by the test taker. This could be loading packages, importing data, defining the database (SQL), performing any pre-cleaning, model fitting (e.g. for modeling items focused on metrics).

Code 1

The next thing you will probably do is jump to the bottom and write the code for the item. We recommend that you keep this short and related to the point being tested. Only include things that are relevant. An example would be including a library call so the test taker knows it is available to them but not including clean that we happen to have done to make the test item possible.

Within your code block you will need to include an encoding for the section that will be left blank. Let’s suppose we are writing an item where the correct measure of center has to be selected. It may look something like this:


We need to give a label name to the blank. You could give this blank a helpful name so that you can easily identify it later, but there are some reserved words in the build system that may lead to a build failure. To make things easier for ourselves, we have developed a habit of sticking to `expr1`, `expr2` etc. You don’t need to follow this convention but be aware you may have unexpected build failures caused by the naming.

The label itself is enclosed in double curly brackets and proceeded by an underscore. This looks strange but will become second nature to you.


Now that we have the code defined and the label names, we need to define what the label should be filled with. We do this in the variables section. If you have left more than one blank they will all be defined in the same variables block. The variables block is written in markdown, so code needs to be inside tick marks. Below is how we would format the example above.


- 'mean'

For a typing challenge that is all you need to do. Once you have defined the code and variable you can preview the item using the preview button in the lower right corner.

We recommend that you preview before saving. Once you save it will trigger a build that will fail if you haven’t resolved issues first. In general you can pick up issues in preview first.

Common Problems

If the preview or build fails there are a few common issues that may be the cause.

  • Have you defined a variable that will be the blank in the code?

  • Does the code run in your local environment? If it does, what is the difference?

    • Have you missed out loading a package?

    • Is the data the same?

Did this answer your question?