What does a low completion rate or low rating mean?
A lower completion rate or lower rating can result from a variety of problems with the course, such as too many difficult exercises, technical issues, or erroneous/uncompelling content/structural issues (e.g. incorrect or out-of-date material).
How can I figure out why my completion rate or rating is so low?
Check the exercises table in the Course Dashboard (for more information, refer to this article) for difficult or problematic exercises. Those with a high number of exercises where a lot of students asked for the hint and/or solution can indicate a higher level of difficulty than may be optimal.
Look for exercises with a high number of feedback messages (i.e. relative to the other exercises in the course).
- Are there lots of reports of broken videos or expired sessions?
- Are students complaining about typos, explanations, etc? While a few typos or errors in a course are unlikely to be the cause, persistent errors in the course could be an explanation for a low completion rate.
- Are students complaining about correct answers being rejected? Even if students are mistakenly reporting correct answers being rejected, this can often indicate that the instructions are not clear or another issue with the exercise.
Chapter-level metrics. While chapter-level metrics are not always instructive, it is worth checking whether there are particular chapters that receive a lower rating or completion rate.
- With respect to the latter, completion rates are typically slightly higher in later chapters as students become more invested in the course.
- If you see that a later chapter dips significantly lower than the preceding chapter, this may be an indication that there is a problem with that chapter. In this case, paying particular attention to feedback messages and exercise metrics in this chapter may help to identify the problem.
I think I’ve located the problem, what do I do now?
If you think that the completion rate or rating may be low because of particular problematic exercises, you can refer to the sections above on high asked-hint and high asked-solution rates to try and modify the problematic exercises. After making changes to address these issues, create a GitHub pull request.
If you think that the completion rate may be affected by technical issues, try to replicate the problem.
- If you are able to replicate the issue, raise an issue in the GitHub repo that specifies the problem, affected exercise number(s), and steps to replicate the problem, and assign @datacamp-contentquality.
- If you are not able to replicate the issue, check if there are enough reports to make you believe that it is a genuine issue. If there are only one or two reports of an issue that you can’t replicate, it is likely an isolated incident that does not require intervention. If there are many reports of an issue, create an issue in the GitHub repo and assign @datacamp-contentquality.
If you think that the problem may not be due to problematic exercises or technical issues, but due to other problems with the content that you can't fix in a pull request (e.g., incorrect videos), create an issue and assign @datacamp-contentquality. In the issue, describe what you think the problem may be, and we will get back to you as soon as possible.