What does a high solution rate mean?

In general, we aim to have our exercises have an asked solution rate below 20%. A high asked solution rate means that the majority of students are giving up, and has had the answer provided to them. This means that a high number of students don’t grasp the learning objective, or the exercise is not testing the material in a fair way. The key element to inspect when faced with a high solution rate is whether the hint is useful. If it is, then it is worth addressing the same types of issues prevalent in exercises with a high asked hint rate.

How can I figure out why so many students are asking for the solution?

To help answer these questions, you can make use of the Content Quality Dashboard to get an idea where students might be going wrong. Documentation for the CQ Dashboard can be found here.

  • Compare the asked hint rate to the asked solution rate. If the hint is effective, then there should be a lower number of students asking for the solution after using the hint. If these numbers are similar, then chances are the hint is not providing the necessary help.
  • Compare the hint to the incorrect responses in the CQ Dashboard (for more on interpreting code-diffs, you can refer to this article). Is the hint addressing the most common issues, or is it unrelated to what students struggle most with?
  • If it looks as though the hint is reducing the number of people asking for the solution, and the solution rate is still high, you can refer to the section on high asked hint rate above to explore alternative explanations.

I’ve located the cause, what should I do?

The hint is not helpful.

  • If you have compared the hint(s) to the most common incorrect responses, and see that it is not addressing students’ most common errors, adding a new hint or replacing the unhelpful one is a good step.
  • As described below, try to avoid giving away the solution in the hint. It should be helpful while still encouraging students to think about and understand the problem at hand.There are other problems with the exercise (unrelated to the hint).

If the hint does address the common errors and students still struggle, one option is to be more explicit or precise. Note, in these cases it is often good practice to also look at other ways of helping students out, rather than giving them the code directly.

  • One option is to provide an example of the code which will force students to understand and adapt it for the exercise (rather than letting them copy and paste the solution).
  • Improvements to instructions and introductory text are often another preferable way to assist students without handing the answer to them.

There are other problems with the exercise (unrelated to the hint).

Refer to this article on High % Asked Hint to address other common issues.

Did this answer your question?