In this article, you will learn how to write the solution and submission correctness tests for Drag and Drop exercises.

The Context, Instructions, and Hint blocks are written in the same way as Coding exercises. See

The Solution and SCT blocks are written in YAML. Note that you need to be very careful with indentation to avoid parsing errors. Make sure you use the space bar, rather than the tab key for indentation.

Solution block

The solution for a Drag and Drop exercise is written in YAML. It consists of drop zones and items that belong to each drop zone. For every drop zone you will need to define values for the following keys:

  • id: The id of the drop zone. This can consist of letters and numbers but cannot contain whitespaces. 
  • title: A human-readable reminder of what the task is about. The students will not see this; it is there to help you maintain the exercise.
  • items: Contains key-value pairs for every item. The keys that require a corresponding value are:
  • id: The id of the item. This can consist of letters and numbers but cannot contain whitespaces.  
  • content:  The content that needs to be shown on the card. The students will see these.
  • offset (optional):  Used in Parsons Drag and Drop exercise to indicate the correct indentation level for that item (line of code).
  • maxOffset (optional): Used in Parsons Drag and Drop exercise to indicate the maximum indentation level.

Note that the ids need to be unique and play a key role while writing SCTs.

Classify Drag and Drop exercise

Shown below is a sample solution block for a Classify Drag and Drop exercise. There is one drag zone, that holds all the options. You can add up to three drop zones. Each drop zone contains a list of items, each with an id and content. These will be shown in a random fashion.

# Drag Zone
- id: tasks
  title: Tasks

# Drop Zone 1
- id: supervised
  title: Supervised Learning
  items:
    - id: spam
      content: Detect if an email is spam or not.
    - id: movie
      content: Predict movie rating

# Drop Zone 2
- id: unsupervised
  title: Unsupervised Learning
  items:
    - id: patterns
      content: Find patterns in data    
    - id: similar_users
      content: Discover groups of similar users

Order Drag and Drop exercise

In the case of an Order Drag and Drop exercise, there is no drag zone and only a single drop zone. The different options that need to be ordered are the in the list of items.

# Single Drop Zone
- id: etl
  title: Tasks
  items:
    - id: extract
      content: Extract the data from the source system(s).
    - id: load
      content: Load extracted data      
    - id: audit
      content: Audit the data
    - id: complete
      content: Complete the real-life ETL cycle.    

Parsons Drag and Drop exercise

In the Parson's variant of the Drag and Drop exercise, learners can order as well as indent the items to complete the code. Accordingly, the solution can specify the maximum indentation level using maxOffset as well as the correct indentation level for each line of code using offset

Any items included in the drag zone behave as distractors and are not a part of the final solution. If you are using distractors, please make sure it is amply clear in the instructions that not all items need to be dragged out of the drag zone. Note that you don’t need to add distractors and can only add a single drop zone.

# Drag Zone
- id: parts
  title: Parts
  language: python
  maxOffset: 0
  items:
    - content: return true
      id: distractor

# Drop Zone
- id: program
  title: Program
  maxOffset: 2
  items:
    - id: line_0
      content: 'def is_true(boolean_value):'      
    - id: line_1
      content: 'if boolean_value:'      
      offset: 1
    - id: line_2
      content: 'return True'
      offset: 2
    - id: line_3
      content: 'return False'      
      offset: 1

SCT block

The submission correctness for Drag and Drop exercises is automatically determined from the solution, and a visual indication of which items are correctly or incorrectly placed is provided on the right panel.

SCTs augment this visual feedback with custom feedback messages that help the learner correct their mistakes. SCTs for Drag and Drop exercises are written in YAML, and support checking the target (drop zone), index and offset of each item. 

Item-level checking

SCTs can specify custom checks at the item level and provide custom feedback by checking for the target, index, or offset of an item. You can specify a correctMessage that is returned if the condition is satisfied, or an incorrectMessage if it is not. Some examples are shown below.

Check if item is in target:

Checks:
   - condition: check_target(item) == target
     incorrectMessage: 'Wrong. This item should be in target.'

Check if the index of item is > 0:

Checks:
   - condition: check_index(item) > 0
     incorrectMessage: 'Wrong. This line isn't nested.'

Check if the offset (indentation) of item is 2:

Checks:
   - condition: check_offset(item) == 2
     correctMessage: 'Wrong. The offset of the item should not be 2.'

Note that positions are zero-based ("Python style, not R style").

Success message

The successMessage is the message displayed when everything is correct. For more information on writing good success message you can look at this article

Failure message

An optional failureMessage can also be specified to display a generic failure message when a submission is incorrect. The generic failure message is overridden by the custom checks at the item level. 

Checking the order

A boolean field isOrdered indicates if the order of the items within the drop zone needs to be considered to determine correctness. It is usually false for classify exercises and true for order and parsons exercises.

Examples

Classify Drag and Drop exercise:

Checks:
   # Check if the item is in the drop zone, display message if not
   - condition: check_target(spam) == supervised
     incorrectMessage: 'Spam is NOT unsupervised learning'
   - condition: check_target(movie) == supervised
     incorrectMessage: ‘Think again! We usually have labelled data for this.’
   - condition: check_target(patterns) == unsupervised
     incorrectMessage: ‘Incorrect! Finding patterns in data is not supervised learning.’
   - condition: check_target(similar_users) == unsupervised
     incorrectMessage: ‘Try again! You don’t know the group the users belong to beforehand.’
# Success message, displayed when the correct solution is submitted
successMessage: 'Well done!'
# Generic failure message
failureMessage: 'Wrong! Try again!'
# The order in the drop zones doesn’t matter
isOrdered: false

Order Drag and Drop exercise:

Checks:
  # Check if the index of the item is the same as the solution
  # Display message if not
  - condition: check_index(extract) == solution
    incorrectMessage: 'Wrong! Try again! This should go first.'
  - condition: check_index(load) == solution
    incorrectMessage: 'Try again! Loading takes place after extracting the data.'
  - condition: check_index(audit) == solution
    incorrectMessage: 'We believe in you, try again! This should come later.'
  - condition: check_index(complete) == solution
    incorrectMessage: 'We believe in you, try again! This should come last.'
# Success message, displayed when the correct solution is submitted
successMessage: 'Well done!'
# Generic failure message
failureMessage: 'Wrong! Try again!'
# The items need to be placed in the correct order
isOrdered: true

Parsons Drag and Drop exercise:

Checks:
  # Check if the index of the item line_0 is the same as the solution
  # Display message if not
  - condition: check_index(line_0) == solution
    incorrectMessage: "Start by defining the function"
  # Check if indentation of item line_0 is the same as the solution
  # Display message if not
  - condition: check_offset(line_0) == solution
    incorrectMessage: "The function definition isn't nested"
  - condition: check_index(line_1) == solution
    incorrectMessage: "Start with a check on the function argument"
  - condition: check_offset(line_1) == solution
    incorrectMessage: "The internals of a function have to be indented"
  - condition: check_index(line_2) == solution
    incorrectMessage: "Try again, you can do it!"
  - condition: check_offset(line_2) == solution
    incorrectMessage: "return True should have an offset of 2"
  - condition: check_index(line_3) == solution
    incorrectMessage: "Try again, you can do it!"
  - condition: check_offset(line_3) == solution
    incorrectMessage: "return False should have an offset of 1"
  - condition: check_target(distractor) == solution
    incorrectMessage: "We don't need this, it's not valid Python."
# Success message, displayed when the correct solution is submitted
successMessage: 'Well done!'
# Generic failure message
failureMessage: 'Wrong! Try again!'
# The items need to be placed in the correct order
isOrdered: true

Note
It is very important to use custom feedback messages with care. Note that submission correctness is automatically determined from the solution, and SCTs only control the messages that get displayed.

SCT Guidelines

1. Every item has to be bound to at least one condition.

2. A condition can only refer to a maximum of one item:

       ❌ check_target(id_1) == solution and check_target(id_2) == solution is not valid because the items id_1 and id_2 are referenced in the same condition

       ✅ check_target(id_1) == solution and check_index(id_1) == solution and check_offset(id_1) == solution is valid because only one item is referenced.

Did this answer your question?