ProofPaths pairs AI grading with questions about students’ reasoning

A student can submit a correct calculation while misunderstanding the assumption that makes it valid. AI assistance makes the distinction between a finished answer and demonstrated understanding particularly important. ProofPaths addresses it by combining automated rubric-based grading with a written interview about the student’s own solution.

Mohsen Hariri and Vipin Chaudhary of Case Western Reserve University describe the framework in a preprint posted on November 2, 2026. The manuscript also states that it was accepted at the EduHiPC 2026 workshop. Its reported validation concerns software-controller behavior using synthetic inputs and scripted agent responses; it does not establish educational effectiveness or agreement with human assessors.

Three reviews, followed by revision

Instructors supply the assignment, a reference solution and grading criteria. Three grading agents examine the submission from different perspectives: correctness, rigor of reasoning and the risk of awarding unwarranted credit. Each produces an overall score, criterion scores, feedback and a confidence estimate.

The initial reviews run concurrently. In a second concurrent round, each agent reads the other reviews and revises its judgment. After checking score ranges and requiring a shared positive maximum score, the controller selects the median overall score. The review supplying that score also supplies the feedback and criterion scores; the other reviews remain available for inspection.

This rule limits the numerical influence of one outlier. In a reported probe, scores of 80, 82 and 95 produced a median of 82, which stayed unchanged when 95 became 100. That arithmetic protection does not establish grading accuracy. Because the agents exchange critiques, their eventual agreement is not independent corroboration, even if they use different models.

Questions anchored to submitted work

The interview runs independently of grading and can begin after document parsing while grading continues. It uses the submission, assignment context and earlier exchanges, without designated inputs for the reference solution, rubric or grading results. Assignment credit comes from grading; interview explanations and feedback are retained for separate instructor review.

Each question carries an identifier for a prepared excerpt from the submission. The controller checks the identifier and retrieves the corresponding text rather than relying on the agent to reproduce it. An excerpt cannot be reused while unused excerpts remain. Before advancing, the controller also checks that earlier excerpts still occur in the normalized submission text.

The paper illustrates the approach with least squares. A student gives a correct formula but incorrectly explains that every quadratic loss has a unique minimizer. A follow-up asks what happens when a feature is duplicated, probing the missing condition of linearly independent feature columns. This exchange, including its understanding labels, is an authored illustration—not an observed student outcome.

The interview agent labels answers as understood, partially understood or not understood and provides feedback. The framework also uses multidimensional item response theory, implemented through the Scorio library, to estimate ability and guide follow-up selection. The authors identify calibration against instructor judgments as necessary for the understanding labels and grading confidence estimates.

What the tests actually establish

Nine deterministic probes exercised the controllers with a synthetic algebra solution, three prepared excerpts and scripted responses. All behaved as specified. Checks returned exact excerpts for valid identifiers, rejected unknown identifiers and altered historical source text, enforced excerpt-reuse rules and produced an explicit failure after three successive invalid interview outputs.

Two accepted cases expose important boundaries: an unrelated question passed when paired with a valid excerpt identifier, and a review passed despite inconsistent criterion and overall scores. Source membership and valid formatting therefore do not guarantee a relevant question or a coherent assessment.

Coverage is limited too. Excerpt preparation follows text order and is capped at 24 excerpts, so passages can overlap or omit later parts of long solutions. Written responses also differ from human-led oral interviews, requiring attention to language accessibility. ProofPaths has no program-execution stage.

Records and instructor authority

The architecture separates course administration, assessment workflows and institutional GPU-backed inference. Stored records retain inputs, reviews, transcripts and progress. Versioned operations prevent results from an older submission or interview attempt from being accepted as current, while caching can reuse matching validated outputs. These mechanisms address execution and traceability, not the educational validity of model judgments.

Actual latency depends on inference capacity and contention; parallel grading alone guarantees no fixed speedup. Deployment also requires policies for access to and retention of student content. Although the browser records certain activity events and can invalidate attempts, the authors caution that those reports alone cannot establish unauthorized assistance or justify a misconduct finding.

ProofPaths offers instructors a record of both submitted work and explanations. Whether its judgments reliably assess understanding remains a question for educational validation; instructors can inspect the evidence and override grades.