In this article, you will learn how to write the solution and submission correctness tests for Drag and Drop exercises.
The Context, Instructions, and Hint blocks are written in the same way as Coding exercises. See
The Solution and SCT blocks are written in YAML. Note that you need to be very careful with indentation to avoid parsing errors. Make sure you use the space bar, rather than the tab key for indentation.
Solution block
The solution for a Drag and Drop exercise is written in YAML. It consists of drop zones and items that belong to each drop zone. For every drop zone you will need to define values for the following keys:
id: The id of the drop zone. This can consist of letters and numbers but cannot contain whitespaces.title: A human-readable reminder of what the task is about. The students will not see this; it is there to help you maintain the exercise.items: Contains key-value pairs for every item. The keys that require a corresponding value are:id: The id of the item. This can consist of letters and numbers but cannot contain whitespaces.content: The content that needs to be shown on the card. The students will see these.offset(optional): Used in Parsons Drag and Drop exercise to indicate the correct indentation level for that item (line of code).maxOffset(optional): Used in Parsons Drag and Drop exercise to indicate the maximum indentation level.
Note that the ids need to be unique and play a key role while writing SCTs.
Classify Drag and Drop exercise
Shown below is a sample solution block for a Classify Drag and Drop exercise. There is one drag zone, that holds all the options. You can add up to three drop zones. Each drop zone contains a list of items, each with an id and content. These will be shown in a random fashion.
# Drag Zone
- id: options
title: "Options"
# Drop Zone 1
- id: dropzone_r
title: "R"
items:
- content: "stringr"
id: stringr
- content: "dplyr"
id: dplyr
# Drop Zone 2
- id: dropzone_python
title: "Python"
items:
- content: "pandas"
id: pandas
- content: "numpy"
id: numpy
Order Drag and Drop exercise
In the case of an Order Drag and Drop exercise, there is no drag zone and only a single drop zone. The different options that need to be ordered are in the list of items.
# One Drop Zone
- id: data_science_process
title: Data Science Process
items:
- id: question
content: Ask an interesting question
- id: data
content: Get the data
- id: explore
content: Explore the data
- id: model
content: Model the data
- id: communicate
content: Communicate and visualize the results
Parsons Drag and Drop exercise
In the Parson's variant of the Drag and Drop exercise, learners can order as well as indent the items to complete the code. Accordingly, the solution can specify the maximum indentation level using maxOffset as well as the correct indentation level for each line of code using offset.
# Drag Zone
- id: a
title: Program
language: python
maxOffset: 2
items:
- content: 'def is_true(boolean_value):'
id: id_0
- content: 'if boolean_value:'
id: id_1
offset: 1
- content: 'return True'
id: id_2
offset: 2
- content: 'return False'
id: id_3
offset: 1
SCT block
The submission correctness for Drag and Drop exercises is automatically determined from the solution, and a visual indication of which items are correctly or incorrectly placed is provided on the right panel.
SCTs augment this visual feedback with custom feedback messages that help the learner correct their mistakes. SCTs for Drag and Drop exercises are written in YAML, and support checking the target (drop zone), index and offset of each item.
Item-level checking
SCTs can specify custom checks at the item level and provide custom feedback by checking for the target, index, or offset of an item. You can specify a correctMessage that is returned if the condition is satisfied, or an incorrectMessage if it is not. Some examples are shown below.
Check if item is in target:
Checks:
- condition: check_target(item) == target
incorrectMessage: 'Wrong. This item should be in target.'
Check if the index of item is > 0:
Checks:
- condition: check_index(item) > 0
incorrectMessage: 'Wrong. This line isn't nested.'
Check if the offset (indentation) of item is 2:
Checks:
- condition: check_offset(item) == 2
correctMessage: 'Wrong. The offset of the item should not be 2.'
Note that positions are zero-based ("Python style, not R style").
Success message
The successMessage is the message displayed when everything is correct. For more information on writing good success message you can look at this article.
Failure message
An optional failureMessage can also be specified to display a generic failure message when a submission is incorrect. The generic failure message is overridden by the custom checks at the item level.
Checking the order
A boolean field isOrdered indicates if the order of the items within the drop zone needs to be considered to determine correctness. It is usually false for classify exercises and true for order and parsons exercises.
Examples
Classify Drag and Drop exercise:
Checks:
# Check if the item is in the drop zone, display message if not
- condition: check_target(spam) == supervised
incorrectMessage: 'Spam is NOT unsupervised learning'
- condition: check_target(movie) == supervised
incorrectMessage: ‘Think again! We usually have labelled data for this.’
- condition: check_target(patterns) == unsupervised
incorrectMessage: ‘Incorrect! Finding patterns in data is not supervised learning.’
- condition: check_target(similar_users) == unsupervised
incorrectMessage: ‘Try again! You don’t know the group the users belong to beforehand.’
# Success message, displayed when the correct solution is submitted
successMessage: 'Well done!'
# Generic failure message
failureMessage: 'Wrong! Try again!'
# The order in the drop zones doesn’t matter
isOrdered: false
Order Drag and Drop exercise:
Checks:
# Check if the index of the item is the same as the solution
# Display message if not
- condition: check_index(extract) == solution
incorrectMessage: 'Wrong! Try again! This should go first.'
- condition: check_index(load) == solution
incorrectMessage: 'Try again! Loading takes place after extracting the data.'
- condition: check_index(audit) == solution
incorrectMessage: 'We believe in you, try again! This should come later.'
- condition: check_index(complete) == solution
incorrectMessage: 'We believe in you, try again! This should come last.'
# Success message, displayed when the correct solution is submitted
successMessage: 'Well done!'
# Generic failure message
failureMessage: 'Wrong! Try again!'
# The items need to be placed in the correct order
isOrdered: true
Parsons Drag and Drop exercise:
Checks:
# Check if the index of the item line_0 is the same as the solution
# Display message if not
- condition: check_index(line_0) == solution
incorrectMessage: "Start by defining the function"
# Check if indentation of item line_0 is the same as the solution
# Display message if not
- condition: check_offset(line_0) == solution
incorrectMessage: "The function definition isn't nested"
- condition: check_index(line_1) == solution
incorrectMessage: "Start with a check on the function argument"
- condition: check_offset(line_1) == solution
incorrectMessage: "The internals of a function have to be indented"
- condition: check_index(line_2) == solution
incorrectMessage: "Try again, you can do it!"
- condition: check_offset(line_2) == solution
incorrectMessage: "return True should have an offset of 2"
- condition: check_index(line_3) == solution
incorrectMessage: "Try again, you can do it!"
- condition: check_offset(line_3) == solution
incorrectMessage: "return False should have an offset of 1"
- condition: check_target(distractor) == solution
incorrectMessage: "We don't need this, it's not valid Python."
# Success message, displayed when the correct solution is submitted
successMessage: 'Well done!'
# Generic failure message
failureMessage: 'Wrong! Try again!'
# The items need to be placed in the correct order
isOrdered: true
Note
It is very important to use custom feedback messages with care. Note that submission correctness is automatically determined from the solution, and SCTs only control the messages that get displayed.
SCT Guidelines
1. Every item has to be bound to at least one condition.
2. A condition can only refer to a maximum of one item:
❌ check_target(id_1) == solution and check_target(id_2) == solution is not valid because the items id_1 and id_2 are referenced in the same condition
✅ check_target(id_1) == solution and check_index(id_1) == solution and check_offset(id_1) == solution is valid because only one item is referenced.