Beyond AI Detection: Better Ways to Verify Student Learning

When generative AI first became widely available to students, one of the first questions in higher education was predictable: How are we going to detect it?

I understood why that became the immediate concern. Faculty were suddenly receiving papers that did not always look or sound like the work they were accustomed to seeing. ChatGPT could produce a reasonably organized essay in seconds, and the technology was improving quickly. Before long, generative AI could help students research, analyze information, write code, work through problems, build presentations, and revise their writing. At the same time, AI-generated work was becoming much harder to distinguish from human work simply by reading it.

So we looked for another technology to tell us whether AI had been used.

I think we have reached the point where we need to ask whether that is actually solving the problem we care about most.

If our goal is to assess learning, the question we ultimately need to answer is not simply, "Did AI write this?" What we really need to know is, "Does this student understand this?"

Those questions can overlap, but they are not the same question.

AI detection can only tell us so much

Imagine two students submit similarly polished papers.

One student researched the topic, developed an argument, used AI to help organize some ideas, rewrote several sections, checked the sources, and ultimately understands the subject very well. Another student gave an AI system the assignment instructions, accepted most of what it produced, made a few edits, and submitted the result without developing much understanding of the material.

From an educational standpoint, those are dramatically different situations. Looking only at the final documents, however, may not tell us much about the difference between them.

Even if we had a tool that could perfectly identify every sentence that involved AI, we would still have another question to answer: What did the student actually learn?

That is why I think higher education needs to move beyond treating AI detection as the center of the academic integrity conversation. Detection may provide one piece of information, depending on how an institution chooses to use it, but it does not give us direct evidence of student understanding.

For that, we need assessment.

AI use is also becoming much harder to define

There is another problem with building assessment around detecting AI. The distinction between "AI-generated" and "human-generated" work is becoming increasingly messy.

Think about how a student might actually write a paper today. They could use AI to brainstorm possible topics, conduct the research themselves, ask AI to explain a difficult concept from one of the readings, write the first draft on their own, ask AI to critique the argument, revise the paper, use another tool to correct grammar, and finally ask for suggestions for a better title.

At that point, asking "Who wrote the paper?" becomes surprisingly complicated.

A course can and should establish rules about which of those activities are acceptable. Students should know those expectations and follow them. But even with a perfectly clear AI policy, the instructor still has to determine whether learning took place.

A student can follow every AI rule and misunderstand the material. Another student might be allowed to use AI extensively and develop a sophisticated understanding of the subject in the process.

Either way, the instructor needs evidence of learning.

Start with what you actually want students to know

One of the most useful things we can do when redesigning assessment for generative AI is to forget about AI for a moment and go back to the learning outcome.

What am I actually trying to determine about this student?

If I assign a research paper, perhaps I want to know whether the student can evaluate evidence, develop an argument, synthesize sources, and communicate a defensible conclusion. If I assign a business case, I may want to know whether the student can analyze a situation, identify relevant information, weigh alternatives, and defend a recommendation.

The same idea applies across disciplines. With a coding assignment, I may care whether the student understands the logic behind the program and can explain important design decisions. In a clinical or professional scenario, I may care as much about the reasoning behind a decision as I do about the final decision itself.

Once we identify what we actually want students to demonstrate, the assessment question becomes much more useful:

What evidence would convince me that this student can actually do this?

A submitted artifact can absolutely be part of that evidence. I just do not think it should automatically be treated as all of the evidence.

Ask students about the decisions they made

One of the simplest ways to get closer to student understanding is also one of the oldest: ask the student why.

Why did you choose this source? Why did you structure the argument this way? Why did you reject another approach? Why does this evidence support your conclusion? Why did you write the code this way instead of another way?

These are fairly ordinary questions, but they can tell us a great deal because they move the assessment away from the finished product and toward the reasoning that produced it.

A student who understands the work will usually have something meaningful to say about the decisions behind it. That does not mean every answer will be perfect. Students hesitate, forget details, get nervous, and sometimes understand something better than they can articulate it on the spot.

We are not trying to catch someone with the wrong answer to a surprise question. We are gathering another piece of evidence and considering it alongside the work the student already submitted.

Give the student something slightly different

Explanation is useful, but I think transfer can be even more revealing.

If students have developed a real understanding of something, they should often be able to use that knowledge when the situation changes. This does not require designing an entirely new exam. Sometimes changing a single variable is enough.

Suppose a student in an emergency management course develops a response plan for a hurricane scenario. After reviewing the plan, I might ask, "Now assume the hospital loses backup power. What changes?"

For a business case, I might say, "You recommended this strategy based on your projected customer acquisition cost. What happens to your recommendation if that cost doubles?"

A programming student could be given a small change to the original requirements and asked which part of the code would need to change and why.

These questions get at something that is difficult to see in a polished final product. The student has to work with the underlying knowledge rather than simply repeat what is already on the page.

Ask students where their own work is weak

Another question I like is some version of, "What is the weakest part of this?"

Which source are you least confident about? What assumption are you making here? What would someone who disagrees with you say? What part of the project gave you the most trouble? If you had another week, what would you change?

Students who understand their work often recognize its limitations. In fact, recognizing uncertainty, tradeoffs, competing interpretations, and weak points is often part of developing expertise.

The final product tends to hide those things because we teach students to submit the cleanest version possible. A conversation about the work can bring some of that thinking back into view.

Look at some of the process behind the product

We can also get useful evidence by looking at how the work developed.

Depending on the assignment, that could mean outlines, drafts, source notes, design iterations, code commits, preliminary calculations, peer feedback, or revisions. I would be careful here, because it would be very easy to turn this into a mountain of required documentation that creates more work for students and faculty without actually improving assessment.

I am more interested in the parts of the process that tell us something meaningful.

For a research assignment, perhaps the student explains how the argument changed as they encountered new evidence. A design student could show an earlier version and explain why it was changed. A student working on code could walk through a particularly difficult problem and explain how they eventually solved it.

Those moments give us access to thinking that often disappears once the final version is submitted.

Reflection can help, but it needs to be specific

Reflection assignments are another common response to generative AI, and they can certainly be useful. The problem is that AI can write a reflection too.

A generic prompt such as "Describe what you learned from this assignment" is just as easy to hand to an AI system as many other writing prompts.

Reflection becomes more useful when it is tied closely to the student's actual experience with the work. Ask about a specific decision. Ask what changed between two versions. Ask which source made the student reconsider something. Ask about a mistake and how it was corrected.

The more specific the reflection is to what actually happened during the assignment, the more useful it becomes as evidence.

Add a short oral defense when the assignment warrants it

This is the approach I have become particularly interested in.

After the student submits the normal assignment, ask a few questions grounded in that student's work.

It does not need to become a 30-minute oral examination. For many assignments, I would rather have a short conversation about a few important decisions, claims, sources, calculations, or conclusions.

If a student relies heavily on one source in the conclusion, ask why that source mattered. If the student chose one approach over another, ask about the choice. If an argument depends on an assumption, change the assumption and see what happens.

Now I have two pieces of evidence in front of me. I have the work the student submitted, and I have the student's explanation of that work.

That gives me a much better picture of what the student understands.

The obvious problem is scale

None of this is a new idea. Academia has used oral assessment for a very long time through dissertation defenses, comprehensive examinations, viva voce examinations, clinical assessments, and other forms of oral evaluation.

There is a reason we still use them. A conversation gives an evaluator the ability to move beyond a prepared answer and ask the student to explain, clarify, defend, or reconsider something.

The problem is that oral assessment becomes difficult very quickly when you move from a handful of students to a large undergraduate course. I might be able to meet individually with 20 or 30 students. Doing the same thing with 100 or 200 students is a very different proposition.

This is where I think AI can play a much more interesting role in assessment.

Instead of primarily using AI to determine whether another AI wrote something, we can use it to make richer forms of assessment more practical.

That idea is at the center of Rocketproof. A student submits the normal assignment, and Rocketproof uses that student's actual work to help conduct a structured oral defense. The conversation is recorded and transcribed, and the instructor can review the submission, the student's responses, the transcript, the rubric, and supporting evidence together.

AI can help generate relevant questions and organize the evidence that comes out of the conversation. The academic judgment still belongs to the instructor.

That last part matters to me. If our response to AI-generated student work is simply to have another AI make all of the academic decisions, I am not sure we have solved very much.

Verification should not feel like surveillance

I also think we need to be careful about the kind of classroom culture we create around this.

If every student becomes a suspect, every polished sentence becomes suspicious, and every assignment turns into an investigation, we are going to create a fairly miserable learning environment for everyone involved.

There is a much more constructive way to frame verification.

I can tell my students, "I care about the work you submit, but I also care about the thinking behind it. You may be asked to explain some of your choices, respond to questions about your argument, or apply what you learned to a slightly different situation."

That seems perfectly reasonable to me.

In fact, students who genuinely did the work and understand it now have an opportunity to demonstrate that understanding directly. The conversation does not have to begin with an accusation that they used AI. It begins with an expectation that students should understand the work they submit.

The same approach works with different AI policies

One of the things I like most about verification is that it does not depend on every instructor adopting the same AI policy.

One course might prohibit generative AI for a particular assignment. Another might allow students to use it for brainstorming and editing. A third might actively require students to use AI and then critique its output.

Those are legitimate pedagogical choices, and different assignments may require different rules.

But there is one expectation that can remain consistent across all three:

The student should understand the work they submit.

That gives us a much more durable foundation for assessment than trying to design every assignment around the current capabilities of AI detection.

Academic integrity and student learning are related, but they are not identical

Academic integrity still matters. Generative AI has not changed that.

What it has done is expose an important distinction between determining whether a student followed the rules and determining whether a student learned something.

A student can follow every rule and still fail to understand the material. A student can also violate a course policy while understanding the subject very well. An instructor may need to address either situation, but they are different educational problems.

If our only response to generative AI is better detection, we risk spending enormous amounts of energy investigating the origin of student work while learning surprisingly little about the student's actual knowledge.

Verification brings the focus back to learning.

You do not need to redesign every assignment tomorrow

I would not suggest adding an oral defense to every homework assignment, paper, discussion post, and quiz. That would create a different problem.

Start with the assignments where authorship and understanding matter most: a major research paper, capstone, final project, case analysis, substantial presentation, piece of code, or professional scenario.

Then try a simple thought experiment:

If I had five minutes with this student after reading the submission, what would I ask?

Maybe you would ask about the strongest source. Maybe you would challenge an assumption. Maybe you would change one detail in the scenario or ask the student to explain a complicated part in plain language.

Or perhaps you would simply ask:

"What was the hardest decision you made here, and why?"

I think questions like that often tell us something a polished final product cannot.

We need better evidence of learning

Generative AI has exposed a weakness that was already present in some of our assessment practices. We sometimes treated the submitted artifact as though it gave us a direct window into the student's mind.

It never really did. AI has simply made that limitation much harder to ignore.

I do not think the answer is to abandon writing, projects, presentations, coding, or other meaningful student work. Those activities still have tremendous educational value. I also do not think we should spend the next decade trying to build a perfect machine for determining which machine helped write which sentence.

We can ask for better evidence.

Students can research, write, build, design, calculate, create, and use AI when the instructor determines that it is appropriate. Then, when the learning matters enough to verify, we can ask students to explain what they did, defend the decisions they made, respond to a new problem, and show us what they understand.

That approach does not depend on winning an arms race with the next generation of AI models. It depends on something much more familiar to education: asking good questions and paying attention to the answers.

That is also the idea behind Rocketproof:

Assess the student, not just the submission.

If you are exploring alternatives to AI detection or reconsidering how you assess student learning in the age of generative AI, you can explore the Rocketproof demo or contact us about using structured oral defense in your course, program, or institution.