05 · Case Study

SONIQ AI Course-Review Auditor

PythonFastAPILLM Integration (AWS Bedrock)Next.js / TypeScriptAccessibility Engineering

Developed at Cal Poly San Luis Obispo’s Digital Transformation Hub (DxHub), powered by Amazon Web Services (AWS), as part of an AI innovation fellowship sponsored by the California Virtual Campus (CVC@ONE) that selected ~100 students from 1,200+ applicants.

No California community college course reaches the CVC Exchange, where roughly 60,000 students cross-enrolled last year, until a peer reviewer certifies it against the CVC Online Course Design Rubric. That review runs 5 to 10 hours. Accessibility is the largest single cost: captions on every video, heading order on every page, alt text on every image, all required by state and federal law. One course held 85 videos, each opened individually just to confirm captions existed. The objective was an automated first pass over the objective standards, against a sponsor target of 2 to 5 hours per course, with a certified reviewer deciding every call.

1.0

Ingest

Input is the Canvas IMSCC export cartridge. parse_imscc() flattens it into a course-object dictionary with a fixed contract spanning modules, pages, assignments, quizzes, discussions, files, videos, and syllabus. Every key is always present, using empty arrays and null where data is genuinely absent, so no downstream check branches on key existence.

Pages carry raw HTML with tags intact, since accessibility checks read tag and attribute structure, alongside a plain-text extraction for semantic checks. Archives validate on ZIP magic bytes rather than file extension, because faculty rename files. course_settings/module_meta.xml overrides the manifest tree for true module ordering, falling back to imsmanifest.xml with a warning.

Output is deterministic: sorted resource order, byte-identical JSON across runs, which matters when a reviewer re-runs a course. A CourseSourceAdapter boundary makes Moodle .mbz or the Canvas REST API new adapters rather than a rewrite.

Fig. 1 · The ingest step: a Canvas IMSCC cartridge is dropped in, and extraction and rubric analysis start automatically from there.
Fig. 1 · The ingest step: a Canvas IMSCC cartridge is dropped in, and extraction and rubric analysis start automatically from there.
2.0

Analysis

Two tracks run over the course object. A deterministic rule engine applies roughly twenty accessibility checks across eight families: images, headings, links, tables, contrast, media, documents, and page structure. On real courses it caught a heading jumping from h2 to h6, alt text that was only a filename, and a link whose entire visible text was the word “Link.” Identical findings group rather than repeat, so a sample generating 131 instances of one error reads as a single issue.

The second track evaluates the rubric semantically through an LLM on AWS Bedrock, returning a rating, confidence score, quoted evidence, and a suggested fix per standard; it is credential-gated, and without AWS access audits still complete on the deterministic checks with the report stating the AI analysis was unavailable.

Ambiguity is surfaced rather than guessed: caption detection returns true on a <track kind="captions"> element and null for iframe embeds that HTML alone cannot resolve.

Fig. 2 · Every standard carries a rating and a finding count, filterable by rubric family or by status, so a reviewer can jump straight to what is incomplete.
Fig. 2 · Every standard carries a rating and a finding count, filterable by rubric family or by status, so a reviewer can jump straight to what is incomplete.
Fig. 3 · An expanded standard: the reasoning behind the rating and the specific items found missing, traced back to the page they came from.
Fig. 3 · An expanded standard: the reasoning behind the rating and the specific items found missing, traced back to the page they came from.
3.0

Delivery

A FastAPI backend exposes POST /audit, GET /history, and per-section comment endpoints, persisting runs to SQLite. Uploads post straight to the backend rather than through the Next.js proxy, which buffers request bodies in memory and would cap large cartridges; the backend streams to disk, starts the pipeline on a background thread, and returns a run_id while the dashboard polls.

The reviewer dashboard (Next.js 15 App Router, React 19, TypeScript strict) holds to WCAG 2.1 AA, the same standard it audits: status is triple-encoded as color, icon, and label, and the status palette was validated against a color-vision-deficiency simulator rather than by eye.

Fig. 4 · The reviewer dashboard: every audited course with its score, flag count, and status, rolled up against the standards across the whole review queue.
Fig. 4 · The reviewer dashboard: every audited course with its score, flag count, and status, rolled up against the standards across the whole review queue.
4.0

Results

A full audit of a real course returns in 5 to 10 seconds. Validation ran against sponsor-curated aligned and non-aligned sample courses, backed by more than 300 automated tests covering the pipeline end to end, including malformed-archive and golden tests. The semantic track flagged a missing AI policy, a requirement new to the 2027 rubric, at 95 percent reported model confidence. Architecture and interface were scoped against the sponsor’s operating scale of 116 colleges and more than 30,000 users.

Authorship here covers the reviewer dashboard and the client-facing work: discovery and demo calls with CVC directors and certified faculty reviewers, translating their requirements into product features, and presentation of the completed build to CVC staff, Cal Poly faculty, and AWS engineers.

The full run, end to end: ingesting a cartridge, waiting out the pipeline, then filtering the returned report down to what still needs a reviewer’s judgment.
The full source — parser, check suite, FastAPI backend, and the test set — is public at github.com/luminenn/Soniq.