What does a low completion rate mean?
A lower completion rate can result from a variety of problems with the course, such as too many difficult exercises, technical issues, or erroneous/uncompelling content/structural issues (e.g. incorrect or out of date material).
How can I figure out why my completion rate is so low?
Check the exercises table in the Content Quality Dashboard (for more information, refer to this article) for difficult or problematic exercises. Those with a high number of exercises where a lot of students asked for the hint and/or solution can indicate a higher level of difficulty than may be optimal.
Look for exercises with a high number of feedback messages (i.e. relative to the other exercises in the course).
- Are there lots of reports of broken videos or expired sessions?
- Are students complaining about typos, explanations, etc? While a few typos or errors in a course are unlikely to be the cause, persistent errors in the course could be an explanation for a low completion rate.
- Are students complaining about correct answers being rejected? Even if students are mistakenly reporting correct answers being rejected, this can often indicate that the instructions are not clear or another issue with the exercise.
Chapter-level metrics. While chapter metrics (viewable in the CQ Dashboard) are not always instructive, it is worth checking whether there are particular chapters that receive a lower rating or completion rate.
- With respect to the latter, completion rates are typically slightly higher in later chapters as students become more invested in the course.
- If you see that a later chapter dips significantly lower than the preceding chapter, this may be an indication that there is a problem with that chapter. In this case, paying particular attention to feedback messages and exercise metrics in this chapter may help to identify the problem.
I think I’ve located the problem, what do I do now?
If you think that the completion rate may be low because of particular problematic exercises, you can refer to the sections above on high-asked hint and high-asked solution rate to try and modify the problematic exercises.
If you think that the completion rate may be affected by technical issues, try to replicate the problem.
- If you think that the problem may not be due to problematic exercises or technical issues, but due to other problems with the content (e.g. incorrect videos, the course is too easy, structural problems), create an issue and assign @datacamp-contentquality. In the issue, describe what you think the problem may be, and we will try to get back to you with our feedback as soon as possible.
- If you are also able to replicate the issue, or if there are enough responses to make you believe that it is a genuine issue (i.e. an isolated report that you can’t replicate may be something specific to the user’s environment), raise an issue in the GitHub repo and assign @datacamp-contentquality.
If you think that the problem may not be due to problematic exercises or technical issues, but due to other problems with the content (e.g. incorrect videos, the course is too easy, structural problems), create an issue and assign @datacamp-contentquality. In the issue, describe what you think the problem may be, and we will try to get back to you with our feedback as soon as possible.