When AI Grades the Essays: What Rubric-Based Scoring Reveals About Student Writing

I want to talk about something that most AI education companies would rather you did not think too hard about: what actually happens when AI grades a student’s work. My postgraduate research at Liverpool John Moores University examined algorithmic bias in AI-driven educational systems, and the findings shaped directly how we approach assessment at SAIL (Scholastic Artificial Intelligence Learning).

What AI grading is doing technically

When an AI system scores a piece of writing, it is performing pattern recognition — comparing structural, lexical, and syntactic features against a model of what high-scoring writing tends to look like. Here is the analogy: imagine identifying a great restaurant without visiting one. You build a model based on observable signals. Your model gets quite good at predicting ratings. But it has never tasted the food. A study by Perelman (2019) found that some automated grading systems rewarded length and lexical complexity over actual argument quality. Research by Binns et al. (2018) found non-native English speakers were disproportionately penalised.

Where rubric-based AI scoring is different

The solution is not to abandon AI scoring. It is to design it around explicit rubrics. Rubric-based AI scoring evaluates a submission against defined criteria — each criterion scored separately, with feedback generated for each dimension. According to a 2024 study from the University of Chicago Education Lab, the AI approaches that produced the strongest learning gains combined specific rubric-tied feedback with Socratic prompting — not those that simply returned a score.

What this means for IELTS preparation

For IELTS — where writing is scored across four official criteria: task achievement, coherence and cohesion, lexical resource, and grammatical range and accuracy — generic AI scoring is of limited value. AI scoring aligned to those exact criteria is a fundamentally different preparation tool. The quality of AI grading is entirely dependent on the quality of its design.

---