Certainty-Based Marking (CBM) in CBM Self-Tests
How CBM works
- After each answer, you say how sure you are that your answer is correct.
- In LAPT and Moodle, this is on a 3-point scale: C=1 (low), C=2 (mid) or C=3 (high)
- We don't rely on words like 'sure' or 'very sure' because these mean different things to different people
- The mark scheme and the risk of a penalty determine when you should use each C level:
- Certainty levels 1, 2, 3 always give you marks 1, 2, or 3 when you are correct
- If you are wrong, then unless you opted for C=1 you will lose marks: -2 at C=2 and -6 at C=3
- * The 'No Idea' option gives you feedback, but no marks, without forcing you to guess. It is not available on all Moodle versions.
Why use CBM?
- To make you think about how reliable your answer is
- To encourage you to try to understand the issues, not just to
react immediately to a question
- To encourage you to think laterally: other pieces of knowledge
may help to validate or question your answer
- To challenge you - if you won't risk losing marks if wrong, then
you don't really know the answer.
- If you are a careful thinker, but not very confident, you will
gain in confidence
- It is more fair - a thoughtful and confident correct answer
deserves more marks than a lucky hunch
- You need to pay attention if you make confident wrong answers -
to think, reflect and learn!
- Efficient study requires that you constantly question how your
ideas arise, and how reliable they are
How to decide on the best certainty level
- If you're sure, obviously you do best with C=3. But you will lose twice over (-6) if you are actually wrong!
- If unsure, you should avoid any risk of penalty by choosing C=1
- In between, you are best to use C=2: you gain 2 or lose 2 depending on whether you are right.
- People are good at games, where you weigh up risks like this. If you look for justifications and queries
about your answer you can weigh these against the possible outcomes. You can also think about probabilities:
- The graph below shows how the average mark at each C level depends on the probability that your answer will be right.
- Suppose you think you only have a 50% chance of being right:
The highest graph for 50% on the bottom scale is black, for C=1. So you
will expect to boost your marks on average most by acknowledging your low
- If you think you can justify your answer well, with less than an 80%
chance of being correct, then the red graph is highest, for C=3. Opt
- Note that you can't ever expect to gain by misrepresenting your certainty.
If you click C=3 (the red line) when you aren't sure, you will expect to do badly - with very likely a
negative mark on average. If you understand the topic well, and think
your answer is very probably right, then you will lose if you opt for C=1 or C=2 rather than C=3.
You do best if you can distinguish which answers are reliable and which uncertain.
- Sometimes you have the option to click "No idea", if any answer would be a pure guess.
This will count as a response (Mark=0) and you will get feedback about the correct answer.
You may choose only to respond to certain questions when testing yourself, perhaps because
these are the topics you are studying, or you are just picking a sample. In that case your marks will
be expressed in two ways: (1) as a percentage of the entire quiz, and (2) as a percentage of the questions
you chose. You won't see feedback for questions you haven't chosen.
Explanation of Final Scores with CBM: Accuracy, CBM Bonus and Knowledge
Tony Gardner-Medwin (UCL)
CBM at UCL (LAPT open resource)
You always do best by trying to maximise your accumulated CBM marks. But it is important at the end to relate your scores
to conventional ways of evaluating your performance. Click here for a full explanation
of the scores presented in LAPT and Moodle.