Adaptive Quiz: CAT (Computer-Adaptive Testing) implementation for Moodle
Maintained by
Vitaly Potenko, Adam Franco
Create tests that efficiently measure users' abilities by adapting the questions difficulty to the estimation of user's ability.
Comments
Comments are no longer open for new posts. Existing comments remain available to read.
We're very happy to hear that you're considering improving the report for the test takers as this has been one of the features that our students and teachers have been missing when using adaptive tests. We've actually been thinking about adding a dynamic type of feedback. We'd propose adding optional fields to the settings where teachers can describe (some of) the difficulty levels of the questions they included, outlining the competencies students need to master each level. These descriptions could then be used to generate the report by selecting the closest levels below (X) and above (Y) the student's ability measure, providing feedback such as: "You are currently developing the following skills: ... (level description X) and are expected to progress toward these abilities: ... (level description Y)." What do you think about it? Would that be viable?
Best,
Konstanze
1. No option and there are no plans for it so far (it's basically a debatable question - whether you should set any time limits in CAT).
2. There is some work going on in background currently to provide a more informative report to the user (optionally). Also, would appreciate your input on this - what would you like to see in this report?
Regards,
Vitaly
1. Is there an option for a timer anywhere?
2. Is there a way to report out results besides just a xxx/xxxx? When a student passes or fails I would like to show them a more comprehensive report.
Thoughts on the above? Thank you!
In short, technically this is doable, however, it'll require significant amount of effort, because the SEB logic implemented in the regular quiz mod isn't pluggable. You cannot just borrow the code from there and hope it'll work in the adaptive quiz mod, you'll basically have to replicate the entire thing.
How are you?
May you always be healthy.
Firstly, I want to thank you very much for your efforts in maintaining this Adaptive Quiz plugin, so that we can use it other than in class to map student groups based on their ability to answer CAT-based questions. We can also use it in new student admission tests at our school.
Furthermore, I hope that in the future it will be a form of security in working on adaptive quiz type questions. Is it possible to provide security features such as regular quiz modules (like those in Moodle currently) that allow Safe Exambrowser to be integrated into the Quiz settings?
Currently, the security in the Adaptive Quiz module is still at browser security level, namely: "Full screen pop-up with some Javascript security".
So by providing security features with Safe Exambrowser (SEB), students will be completely protected when working on Adaptive Quiz type questions, from fraudulent acts such as cheating via tabs on the browser used, or other browsers that could run behind the adaptive quiz page. which is taking place.
Maybe that was our hope, so that it can be followed up immediately in the future.
So, thank you very much, Vitaly.
God bless you.
Thank you so much for your responses. It is a great help.
It doesn't matter how the taker answers questions - correct or wrong, the stoppage conditions always remain the same, event if all questions are answered correctly or all are answered wrong.
Before I further read the study by Linacre, I will just ask if what you mean on your answer is that the questions will just continue until the end if the test taker is answering the questions without any wrong answer? Is that right?
1. It may stop because you might have set the 'maximum questions' parameter to 40 in the adaptive quiz settings. Check what value you have there.
2. There's no such stopping condition as 'maximum correct answer'. No matter how the taker is answering the questions, the test will stop 1) when the standard error value matches the one you configure in the quiz settings, 2) the maximum number of questions has been reached, again, you configure it in the quiz settings.
You can see the stoppage reason for each taker's attempt when viewing quiz report as a manager. It'll give you understanding of why the test has stopped in this particular attempt.
I'd recommend to follow those links in the plugin's description above to learn how CAT works, in particular the one with Linacre (2000) title.
Thank you for your response. I think I kinda getting it now but I still need guidance with this. Correct me if I'm wrong:
1. Is the test stopping on the 40th question because it is out of question to pull on that level? If that's the case, I need to upload more questions per level.
2. How the test would stop if the test taker reach the maximum correct answer? Is that possible to set or the test will just go until the maximum question reached even if the test taker is answering the questions correctly?
Regarding SCORM - no, it's not possible, it's an entirely different thing.
As for the quiz config - please, check the 'The Testing Process' section in the plugin's description above, in particular, where it says 'The possible stopping conditions are as follows:..' and the bullet list below. There you'll find out when the testing process stops.
Also, check the 'Test Parameters and Operation' section. You should have configured the minimum amount of questions and the maximum amount of questions. The maximum amount is when the test may stop (one of the possible conditions).
So, in general, the test may stop when the standard error for the ability estimation is within the configured boundaries, or when the maximum amount of administered (that is, presented to user) questions has been reached.
You cannot stop the test when 'the taker answered 85 correct/wrong' questions, this is not how it works.
Also, How are we going to set the standard error if we want to stop the test if the test taker answered 85 correct answers and vice versa, will stop also with 85 wrong answers? We are using 150 questions overall and has 4 levels of questions. We followed the rule on this documentation but it is always stopping on the 40th question. and sometimes, it is not stopping at all.