In this article, you will learn how to write the solution and submission correctness tests for Drag and Drop Exercises.
The solution for a Drag And Drop Exercise is written in YAML as an array of target zones (either drag zone or drop zone), and the items that belong to each target zone. Every target is defined as a map (key-value pairs) with the keys id, title, items, and an optional maxOffset. The items object is a YAML array of items, each of which is a map with keys id and content. Note that the ids need to be unique and play a key role while writing SCTs.
Shown below is a sample solution block for a classify drag-and-drop exercise. Each zone has an id and title, and consists of items, each of which has an id and content key.
# Drag Zone ----
- id: tasks
# Drop Zone 1 ----
- id: supervised
title: Supervised Learning
- id: spam
content: Detect if an email is spam or not.
- id: movie
content: Predict movie rating
# Drop Zone 2 ----
- id: unsupervised
title: Unsupervised Learning
- id: patterns
content: Find patterns in data
- id: similar_users
content: Discover groups of similar users
In the case of an order drag-and-drop exercise, there is only a single drag/drop zone.
- id: etl
- content: Extract the data from the source system(s).
- content: Load extracted data
- content: Audit the data
- content: Complete the real-life ETL cycle.
In the Parson's variant of the drag-and-drop exercise, learners can order as well as indent the items to complete the code. Accordingly, the solution can specify the maximum indentation level using maxOffset as well as the correct indentation level for each line of code using offset.
Any items included in the drag zone behave as distractors and are not a part of the final solution. If you are using distractors, please make sure it is amply clear in the instructions that not all items need to be dragged out of the drag zone.
# Drag Zone ---
- id: parts
- content: return true
# Drop Zone ---
- id: program
- content: 'def is_true(boolean_value):'
- content: 'if boolean_value:'
- content: 'return True'
- content: 'return False'
Submission Correctness Tests (SCTs)
The submission correctness for drag-and-drop exercises is automatically determined from the solution, and a visual indication of which items are correctly or incorrectly placed is provided.
SCTs augment this visual feedback with custom feedback messages that help the learner correct their mistakes. SCTs for drag-and-drop exercises are written in YAML, as a map, and support checking the the target, index and offset of each item.
successMessage: 'Well done!'
- condition: check_target(spam) == unsupervised
correctMessage: 'Spam is NOT unsupervised learning'
- condition: check_target(movie) == solution
Think again! We usually have labelled data for this.
failureMessage: 'Wrong! Try again!'
The successMessage is the message displayed when everything is correct. A boolean field isOrdered indicates if the order of the items within the target zone needs to be considered to determine correctness. It is usually false for classify exercises and true for order and parsons exercises.
SCTs can specify custom checks at the item level and provide custom feedback by checking for the target, index, or offset of an item, and returning a correctMessage if the condition is satisfied (or an incorrectMessage if it is not)
check_target(item) == target Check if item is in target
check_index(item) > 1 Check if index of item is > 1
check_offset(item) == 2 Check if offset/indentation of item is 2
In the example above, the two custom feedback checks work in the following manner:
- If the target of the spam item is unsupervised, a message gets displayed telling the learner that ‘Spam is NOT unsupervised learning’
- The target of the movie item is expected to match the solution. If it does NOT match the solution, the custom message ‘Think again! …' is displayed.
An optional failureMessage can also be specified to display a generic failure message when a submission is incorrect. The generic failure message is overridden by the custom checks at the item level.
It is very important to use custom feedback messages with care. Note that submission correctness is automatically determined from the solution, and SCTs only control the messages that get displayed.
- Every item has to be bound to at least one condition.
- A condition can only refer to a maximum of one item:
- ❌ check_target(id_1) == solution and check_target(id_2) == solution is not valid because the item id_1 and id_2 are referenced in the same condition
- ✅ check_target(id_1) == solution and check_index(id_1) == solution and check_offset(id_1) == solution is valid because only one item is referenced.