In this article, you will learn how to write the solution and submission correctness tests for Drag and Drop Exercises.

Solution

The solution for a Drag And Drop Exercise is written in YAML as an array of target zones (either drag zone or drop zone), and the items that belong to each target zone. Every target is defined as a map (key-value pairs) with the keys id, title, items, and an optional maxOffset. The items object is a YAML array of items, each of which is a map with keys id and content. Note that the ids need to be unique and play a key role while writing SCTs. 

Shown below is a sample solution block for a classify drag-and-drop exercise. Each zone has an id and title, and consists of items, each of which has an id and content key.  

# Drag Zone ----
- id: tasks
  title: Tasks

# Drop Zone 1 ----
- id: supervised
  title: Supervised Learning
  items:
    - id: spam
      content: Detect if an email is spam or not.
    - id: movie
      content: Predict movie rating

# Drop Zone 2 ----
- id: unsupervised
  title: Unsupervised Learning
  items:
    - id: patterns
      content: Find patterns in data    
    - id: similar_users
      content: Discover groups of similar users

In the case of an order drag-and-drop exercise, there is only a single drag/drop zone. 

- id: etl
  title: Tasks
  items:
    - content: Extract the data from the source system(s).
      id: extract
    - content: Load extracted data
      id: load
    - content: Audit the data
      id: audit
    - content: Complete the real-life ETL cycle.
      id: complete

In the Parson's variant of the drag-and-drop exercise, learners can order as well as indent the items to complete the code. Accordingly, the solution can specify the maximum indentation level using maxOffset as well as the correct indentation level for each line of code using offset

Any items included in the drag zone behave as distractors and are not a part of the final solution. If you are using distractors, please make sure it is amply clear in the instructions that not all items need to be dragged out of the drag zone.

# Drag Zone ---
- id: parts
  title: Parts
  language: python
  maxOffset: 0
  items:
    - content: return true
      id: distractor

# Drop Zone ---
- id: program
  title: Program
  maxOffset: 2
  items:
    - content: 'def is_true(boolean_value):'
      id: line_0
    - content: 'if boolean_value:'
      id: line_1
      offset: 1
    - content: 'return True'
      id: line_2
      offset: 2
    - content: 'return False'
      id: line_3
      offset: 1

Submission Correctness Tests (SCTs)

The submission correctness for drag-and-drop exercises is automatically determined from the solution, and a visual indication of which items are correctly or incorrectly placed is provided.

SCTs augment this visual feedback with custom feedback messages that help the learner correct their mistakes. SCTs for drag-and-drop exercises are written in YAML, as a map, and support checking the the target, index and offset of each item. 

successMessage: 'Well done!'
isOrdered: false
checks:
   - condition: check_target(spam) == unsupervised
     correctMessage: 'Spam is NOT unsupervised learning'
   - condition: check_target(movie) == solution
     incorrectMessage: >
       Think again! We usually have labelled data for this.
failureMessage: 'Wrong! Try again!'

The successMessage is the message displayed when everything is correct. A boolean field isOrdered indicates if the order of the items within the target zone needs to be considered to determine correctness. It is usually false for classify exercises and true for order and parsons exercises.

SCTs can specify custom checks at the item level and provide custom feedback by checking for the target, index, or offset of an item, and returning a correctMessage if the condition is satisfied (or an  incorrectMessage if it is not)

check_target(item) == target    
Check if item is in target

            check_index(item) > 1    Check if index of item is > 1

         check_offset(item) == 2    Check if offset/indentation of item is 2

In the example above, the two custom feedback checks work in the following manner:

  1. If the target of the spam item is unsupervised, a message gets displayed telling the learner that ‘Spam is NOT unsupervised learning’
  2. The target of the movie item is expected to match the solution. If it does NOT match the solution, the custom message ‘Think again! …' is displayed.

An optional failureMessage can also be specified to display a generic failure message when a submission is incorrect. The generic failure message is overridden by the custom checks at the item level. 

It is very important to use custom feedback messages with care. Note that submission correctness is automatically determined from the solution, and SCTs only control the messages that get displayed.

SCT Guidelines

  • Every item has to be bound to at least one condition.
  • A condition can only refer to a maximum of one item:
  • ❌ check_target(id_1) == solution and check_target(id_2) == solution is not valid because the item id_1 and id_2 are referenced in the same condition
  • ✅ check_target(id_1) == solution and check_index(id_1) == solution and check_offset(id_1) == solution is valid because only one item is referenced.

Did this answer your question?