AI Oral Assessment vs AI Detection: Which Question Are You Asking?

Two different questions#

Faculty who worry about AI-written work usually mean one of two things. The first worry is "did this student write this?" The second is "does this student understand this?" The two questions sound alike and lead to different tools.

• AI detection addresses the first question. It analyzes text and reports a likelihood that a model produced it.
• Oral assessment addresses the second. It asks the student to explain and defend the work and records how they respond.

A grade is supposed to reflect understanding, so the second question is usually the one that matters for assessment. The first question matters for misconduct processes, where the burden of proof is high.

What the evidence says about detection#

Weber-Wulff and colleagues tested a set of tools that claim to detect AI-generated text. Their conclusion was that the available tools are neither accurate nor reliable, and that they lean toward classifying text as human-written. That means detectors can miss machine-written text and can also be unreliable on human-written text, depending on the tool and the conditions.

Liang and colleagues examined detectors on writing by non-native English speakers. They found that detectors consistently misclassified that writing as AI-generated, while native-speaker writing was classified more accurately. This is a fairness problem. A student who writes carefully in a second language is more likely to be flagged.

Detectors also face a moving target, because the models that produce the text keep changing. None of this means a detector can never be informative. It means that a score is not evidence you can hold up in an appeal.

What oral assessment collects#

In an oral assessment, an examiner asks the student about their work, follows up on the answers and judges the responses against criteria. The record may include a transcript, a recording and rubric scores. If a student cannot explain a claim, a source or a design choice in their own submission, that tells you something about understanding. If they can, it tells you something too.

Oral assessment does not prove authorship. A student could study a generated paper and explain it well. What it does show is that the student can discuss the content, which is usually the outcome the course cares about. Some tools report an "authorship" result in addition to understanding. Whatever the label, check how the tool reaches it and whether you can see the evidence.

Side by side#

When detection might still be used#

Some institutions license a detector and some instructors look at the scores. If you do, use the score as a reason to talk to the student, not as proof. State in your syllabus how scores will and will not be used. Never base a grade or a misconduct finding on the score alone.

Using oral assessment as an alternative#

Oral assessment is one of several alternatives to detection. Others include in-class writing, process portfolios with drafts and decision logs, and tasks that require students to apply AI output critically. Oral assessment works well alongside a submitted artifact because the questions can refer directly to the student's work.

To run one fairly:

1. Share the criteria in advance.
2. Offer a practice attempt.
3. Provide accommodations and a text option.
4. Review the evidence yourself before releasing grades.
5. Keep a person responsible for the final decision.

The guide to assessing AI-assisted student work sets out a full method, and the explainer on AI oral exams describes the format.

Our blog essay on alternatives to AI detection covers the same ground from the assignment-design side.

What a good tool should show you#

If you evaluate an oral assessment tool, ask to see a real session record. You should be able to see the questions asked, the student's answers, how the scores relate to the evidence, and how you change a score. Ask what the tool says about authorship and how it supports that statement. The comparison guide lists criteria drawn from vendors' public pages.

Where Rocketproof.ai fits#

Rocketproof.ai takes the oral assessment route. It asks students questions about their own submissions and keeps a transcript and rubric evidence for the instructor to review. It assesses whether the student can explain and defend the work, which is separate from detecting AI-written text, and it leaves the grade to the instructor. The oral defense platform page describes the workflow.