You are responsible for helping us to keep a high standard in our assessments. Do not allow items to proceed if they don’t meet the requirements.
Three things you must review for every item:
Does it test the required objective?
Is it technically correct?
Can it be answered by reading only the context and stem?
Three things that would help us out:
Is it suitable for an international audience?
Does it use any topics that should be avoided?
Are all of the item writing guidelines being followed?
How does a review happen?
You will be assigned a review as a pull request submitted by another item writer
You should review the items listed in the pull request using the expert review link in the pull request
The linked task will tell you the requirements the items should meet
Review decision for each item:
Accepted: Meets all criteria and could be put live with no change required
Revision required: Meets most criteria but requires a small change to be accepted. If you would be happy for the change to be made with no re-review, this would be considered a small change.
Rejected: Does not meet enough criteria to be accepted without significant changes. This item will be immediately dropped.
Your feedback:
Should be specific
Should enable the item writer to revise the item so that it will be accepted (if needed)
Should make a clear case for rejection (if needed)
The following are examples of what we expect in your review comments:
[ ] `123456` Accepted
[ ] `abc123` Revisions Required. The item does not meet guidelines multiple choice 4. This item can be accepted if the options are put into numerical order. In addition, there is a typo in the context “teh” should be “the”.[ ] `ligawjegb` Rejected. The item does not test the learning objective. The objective requires that we test the ability to fit a linear model but this item is testing random forest regression. Significant rework requiring further review would be required to resolve the issue.