Approaching Course Maintenance can feel intimidating and overwhelming - so here’s a guide on how to get started.
Step One:
Log into the Course Dashboard and find your Course
Step Two:
Scan through your Performance Metrics to see if anything appears off. Some red flags may include:
Average rating is at or below 4.2 / 5
Average twenty-eight day completion is below 25%
A large number of feedback messages compared to number of completions (If your course is high volume (over 100 starts a week), and number of feedback messages is over 5% of starts for that exercise, this is a good indicator that there is an issue with that exercise.)
Step Three:
Review the Course and Exercise Metrics. Are there any Chapters that have a significantly lower rating than the others? If so, review the Exercise View to isolate which exercise users are having trouble with. A problematic exercise will likely have a high percentage of ‘Asked Hint’ and ‘Asked Solution’ percentage.
For “Asked hint”: above 20% is a warning, above 35% is a red alert
For “Asked Solution”: above 20% is a red alert
Step Four:
After isolating a problematic exercise, read through the Issues, Feedback, and Incorrect Attempts,
Start with Incorrect attempts, to see what users are actually submitting using the ‘Diff’ link
Then look at ‘Issues’ to see the messages reported by users
Lastly, look at “Feedback” (these messages are submitted in response to the question “Was this Feedback (or Hint) helpful”
Questions to ask:
Are many users submitting the same incorrect answer?
Are the incorrect answers due to typos, missing parenthesis, extra spaces, incorrect punctuation or other common typing errors?
Does the correct answer require knowledge that is not presented in the question or instructions?
Could the question or instructions be misleading or unclear?
Does the feedback focus on the question or code itself, or on the hint or SCT? (Remember, SCTs are done by DataCamp, so let us know via a Github issue if they could be improved!)
Keep In Mind:
The solution to the issue may NOT be more code scaffolding! We want our questions to be challenging, and not giving away the answer.
Step Five:
Fix it! After using the dashboard to locate problems, use the Teach editor to make your changes. All edits must be reviewed by the Content team before going live, so you will need to create and work in another branch.
When you're finished, open a pull request on GitHub to the master branch and tag @datacamp-contentquality as the reviewer. They'll let you know if they have any questions or concerns about your updates and will merge the changes live.
If you are not sure how to best fix a problem with your course, open a Github issue, describe the problem, and tag @datacamp-contentquality.
Additional Resources