CAPQuiz
Maintained by
Sebastian Gundersen, George Schaathun, André Storhaug
CAP is short for Computer Adaptive Practice. In CAPQuiz, the proficiency is measured by a rating. Good answers increase the rating, and bad answers decrease it. To increase the rating, students need to give good answers more of than bad ones over time. Estimating question difficulty is known to be difficult. CAPQuiz automates this process to some extent. The question author must provide an initial estimate, but CAPQuiz improves the estimates based by comparing how the same student answers different questions. Hence the rated question sets will improve over time.
Comments
Comments are no longer open for new posts. Existing comments remain available to read.
Thank you for your quick and comprehensive response.
1. I hadn't thought of that. Thank you for the tip (numeric with its own data set).
2. Student level = 1200 | Question level = 1200 ==> success 50%
I get that
Klinkenberg and others say: prefer a higher chance, for example 75%.
I also understand that
You write: "This is configurable."
Do you mean the setting at "Configure N-closest":
Desired user win probability 0.75
I am a physics teacher in secondary education. This is perfect for practicing and grinding formulas, calculations etc.
I am also a trainer of teachers and organizations / schools in the field of Education & ICT.
Many thanks for this digital elaboration of a grand idea.
Sincerely,
Kees Koopman.
First of all, the plugin is brand new; only just ready for field testing, and I am preparing to do so in my module in the Autumn this year. I did, however, use a similar system (in-house, standalone implementation) i 2017, so I know the idea is sound.
In terms of how it is supposed to work,
1. The intention is to use parameterised questions, which is simple to do in mathematics and physics, using questions types such as STACK or (I think) numeric. In this case, it makes sense to give the students the same question over and over again, just with different numbers. You are absolutely right that if each question is unique, as would be common in many non-mathematical subjects, an enormous number of questions is needed.
2. Question delivery is usually randomised and it can be tuned somewhat according to pedagogic requirements. If a student at 1200 gets a question at 1200, it means that the success probability is 50%. Klinkenberg et al (2011)¹ argue that the student should get odds better than 50%, maybe 75%, which means that he should ideally have a question at lower rating. This is configurable.
3. Optimising the question selection algorithm, I take it, is an open, and very interesting, research problem. We are working on this, and I know of other teams, too, who have similar ideas and challenges.
4. The rating model which is used, is the same one used in chess. This means that you increase in rating when you answer better than expected, and the more surprising the result, the greater the change. Similarly, bad answers make you drop. At every point in time, you get random questions to give you approximately the ideal success probability.
Klinkenberg's paper explains the underlying mechanism very well, although there are several points open to debate and further research.
So essentially, you are right, the intention is what you suggest. I shall review the manual when I prepare my own module, and make sure that it is better explained. I hope this helps.
What subjects to you teach? Do you think the idea could work in your subject?
¹ https://www.sciencedirect.com/science/article/pii/S0360131511000418
I came across the CAPQuiz activity in the plug-in collection, a plug-in that allows adaptive testing.
A student has a certain 'rating', is offered vagues of various levels and, depending on the answers, goes up or down in 'rating' (or stays the same).
Well, that requires a considerable amount of various questions, each with its own level rating ('rating').
Suppose the candidate receives a basic rating of 1200 at the start of the quiz.
QUESTION Is it the intention that he / she will first be offered questions with a rating of 1200? And that questions with a higher or lower rating follow, depending on the answers given? The manual is unclear about this.
Who else has dived here and can / wants to provide clarity?
Thanks in advance.
Kees Koopman