AI Grading of Nursing Students' Clinical Skills Videos Against a 55-Item Rubric
My company, DeepRails, built this for one of the largest nursing colleges in the United States. Nursing students film themselves doing clinical skills like handwashing, sterile gloving, putting PPE on and off, and moving a patient from bed to chair, then submit the video in Canvas. The system grades each video against a checklist of up to 55 items and writes feedback for every item. We started on Gemini 2.5 Pro and moved to Gemini 3 Flash in January after an A/B test showed similar accuracy at much lower cost. 1. The nursing faculty rewrote the rubric so each line is one thing you can see on camera. We also tightened filming instructions: 720p or better, hands close to the camera, and both sides of the gloves shown. 2. Cache the video in Vertex once. The video is over 99% of every prompt and cached tokens bill at 10%, so everything after is cheap. Sample at 4 to 6 frames per second, because the default 1 fps misses too much. 3. The first prompt builds a timeline and tags every event as PERFORMED, PRETENDED or MENTIONED. Before this, the model kept giving credit for steps students said out loud but never did. 4. Grade small groups of related rubric items in parallel. Running the whole rubric in one prompt got 30 to 40% of grades wrong. Each item gets pass, fail or not demonstrated, a confidence level, and feedback written to the student. Confidence starts at Low and only goes up if the model clearly sees the relevant hands or body part. 5. A second prompt checks every grade, passes included. Its key line is: "The grader is generally reliable but can hallucinate or miss details. Review the video yourself." It watches the video before it reads the grade. Re-checking only the fails was noticeably worse. 6. Disputed items go to a fresh grader that never sees the first attempt, then get checked again. If still disputed, a judge prompt reads the whole history and decides. Under 2% of items get this far. 7. Low-confidence items go to a human, along with the roughly 10% of checks the AI still isn't reliable on, mostly fine finger work like glove placement and taking PPE off. Across 7 skills validated in January and February 2026 (272 videos, 11,143 rubric checks), the AI was right on 92% of rubric checks against 94% for trained human graders, with all 7 skills clearing the 85% bar. The system handles about 90,000 videos a year, work that used to take thousands of hours of grading.
$1.2M savings in 2025, $2.1M in 2026, scaling to $4M in 2029.
0 comments
No comments yet. Be the first to weigh in.