Essay (auto-grade)

Hi,
We have found a bug.
To recreate it:
1. Create a rubric and publish it as a template.
2. Add a question to a quiz. When you select a rubric, each template appears twice in dropdown.
Hope you can fix this issue! Thank you!
BR
Nikola

"Bug: The plugin ignores the 'Show grade bands = No' setting. When a student receives a sub-optimal grade, the 'Hints to improve your grade' section is forcibly displayed. It should respect the 'Show grade bands' setting in all cases."

We discovered with version 2020073192, unless a file type is selected, users are able to upload exe files.

Could you please add capability to turn of AI Autograding as an option within Admin settings here (admin/settings.php?section=aiplacement) or in plugin settings for this question type? Thank you! -Ben

Hi Wayne,
[1] the AI reviews the student submission agaisnt any criteria or rubric included in the teacher-defined prompt
[2] the type, quantity and level of feedback returned also depends on what is specified in the prompt
[3] any text-generating AI models that are active on the Moodle site where the question resides
HTH, Gordon

Hi Gordon,
We’re currently reviewing your Autograde plugin on behalf of a client who is using an older version and evaluating potential upgrade paths. I noticed from recent discussions that you've introduced OpenAI capabilities within the tool and I wanted to reach out directly for a bit more context.
Would you happen to have any documentation or references explaining how AI is integrated in the plugin? Specifically, we're interested in understanding:
- Whether the AI reviews students’ responses against predefined correct answers or expected patterns.
- What kind of feedback, if any, is returned to the student.
- Which AI models are supported currently?
Any documentation, example use cases, or technical overview you could share would be greatly appreciated.
Thanks
Wayne

Hi Dan,
the OpenAI key needs to be set in the Moodle admin settings. Also, admin needs to allow text generation by AI.
If that has been done, you should see "AI Settings" within the settings for each Essay (auto-grade) question. In those settings, you can specify the AI prompt and AI grade for an individual question.
For your reference, a prompt that I used developing the AI functionality in this plugin is given below.
Note the use of {{double braces}} to allow the inclusion of questiontext (supplied by the teacher) and answertext (supplied from the student)
Also, note instruction [7] which specifies the required format of the output.
============================
You will receive an English essay created by a Japanese student of English.
Your task is to generate formative feedback and a suitable grade for the essay.
Use the following rubric to make your assessment:
[1] Vocabulary: 0-5 points
[2] Grammar: 0-5 points
[3] Flow of Ideas: 0-5 points
[4] Task Completion: 0-5 points
The following prompt for the writing task was given to the students:
{{questiontext}}
Follow these important instructions:
[1] Do not include any HTML, markdown formatting, or backticks in your response.
[2] Do not include any greetings or platitudes in your response.
[3] Use simple English (CEFR A1-A2) for the formative feedback.
[4] Flag only true spelling mistakes (i.e. words that do not exist in standard English), and do not mark correctly spelled words as errors, even if they seem unusual.
[5] Calculate the grade as the sum of the scores for each category in the rubric.
[6] Calculate a "grademax" value that is the sum of the maximum scores for each category.
[7] Return a JSON object containing three properties: "feedback", "grade" and "grademax".
Here is the answer text that the student submitted for this task:
{{answertext}}

For the Openai, do I need to install an additonal plugin? In setting it just shows pluginname for the quiz qeustion. I purged cache. Also have the openai api in ai settings in Moodle 4.5+ version. Thank you

Hi Gordon,
very nice work thanks a lot but now I get another error in moodle5 (dev) which means that I can only display the questions but can no longer edit them or create new ones.
Error: Non-static method core_ai|manager::get_providers_for_actions) cannot be called statically
Of course this is only my problem, because I am testing in moodle5, maybe I will find a correlation, then I would get back to you.
In the meantime, thanks again for the update.

Hi Norbert, sorry about those errors. They are fixed in the most recent release. Please upgrade at the next convenient opportunity.
Hi Gabor, thanks for the feedback. This plugin now allows for an AI feedback and grade, which can be used as part of the grade for a question. Please try it out and let me know if it suits your needs.

I think the plugin is great! People should definitely try it.
I was wondering that since it is available for Moodle 4.5, would it be possible to integrate AI autograding based on model answers? For example, you provide the question and the expected answer. Students' answers would be evaluated by AI based on the provided sample answer. And if AI is unsure about the grade, alert the teacher to mark it themselves. That would be a game changer for this activity.

I got a lot of deprecation-hints in moodle 5: Deprecated: Creation of dynamic property qtype_essayautograde_question::$responsesample, $responsesampleformat, $allowsimilarity, $enableautograde, $itemtype, $itemcount, $showfeedback, $showcalculation, $showtextstats, $textstatitems, $showgradebands, $addpartialgrades, $showtargetphrases, $errorcmid, $errorpercent, $errorfullmatch, $errorcasesensitive, $errorignorebreaks is deprecated in... but still works ­− nice plugin anyway, thank you for that, Gordon.

I like using this plugin more and more with my courses. It would be good if you could assign either individual marks or % to the target phrases. For some mark schemes, %ages don't always work

Hi Gordon,
I am getting an exception during import of xml moodle format. This is exactly the same file as the export:
Exception : Call to undefined method qtype_essayautograde::check_ordering_combined_feedback()
Unfortunately I don't know which version of moodle my university is on (probably 4.2)

Hi Gordon,
Any plans to bring this up to working with Moodle 4.2?