Deck · Pipeline & prompts · The simple version · Before and after · Luna vs Muse · Theme

The final CFA lesson pipeline

One lesson, station by station: who does the work, what comes out, how long it takes, and what happens when it fails. Then how 1,233 lessons get through it and into staging. Numbers are from the 22 September run of 20 lessons (10 topic overviews, 10 chapters, Levels I to III, nine subjects), all 20 in flight at once.

Every word a candidate reads is written by GPT-5.6 Luna on Bedrock. Every number is computed by our code before the writer sees it. Every check that can be deterministic is deterministic. Jev judges each section on its own, and every section that fails gets its own repair, in parallel. There is no whole-lesson reviewer whose verdict nobody acts on.

1. One lesson, station by station

Timings are per-lesson means from the run above. "Repair" means the exact problem is sent back with the text, at most once per section, and a repair that shrinks the work, changes a heading, or touches a fence it was not asked to change is refused.

1
PlannerLuna · high
Reads the LOS, the subject playbook and the source packet. Decides the learner task, the worked-example fixtures, the figures (Mermaid type and the fixture each draws from, one illustration), the trap brief, the exam framing, the practice formats.Out: plan JSON, validated by code; an invalid plan gets one bounded repair.
74 sp90 114 s
2
Fixturescode
Compiles every number the lesson may use: inputs, derived terms, calculation results with display strings.
<1 s
3
WriterLuna · max
The tutor. Gets the plan as a brief, the fixtures, the VOICE rules, the section template, the Mermaid, chart and table syntax rules, the source packet.Out: the lesson Markdown. 81k output tokens on average, 84% of them hidden reasoning. This is the bottleneck.
308 sp90 393 s
4
Normalise + static checkscode
Heading levels, template labels, blank lines and separator rows around tables. Then the deterministic rules: math delimiters and commands, table width, Mermaid parse and type, chart values bound to fixtures, prose runs, pipeline-language leaks, register tells.Out: findings with severity; a major finding gets one bounded whole-draft repair (12 of 20 lessons needed it).
43 swhen triggered
5
Fan out: Mermaid stylist ∥ imagesLuna · medium gpt-image-2.5-flare
At the same time: one stylist call per flowchart redraws it in the house shape (code refuses a redraw that changes meaning), and each illustration request becomes a wordless flat picture. Styled fences are applied to the media output afterwards.
12 sboth together
6
Jev review per sectionJev · gate
The lesson is split into sections (H2 for an overview; the LOS intro, each ### subsection and the snapshot for a chapter). Each section is judged on its own against the plan, the fixtures and a rubric: numbers from fixtures, exam framing, teaches not lists, visual earns its place, worked example is a scenario, steps show work, changed condition complete, trap told as a story, practice items defensible with rationales, overview stays an overview, register rules, voice. Static findings are attributed to the section that contains their evidence.Out: per section, the failed questions with confidence and the pass criterion. 9.4 calls per lesson, about a second each.
14 s$0.02
7
Fan out: repair agents per sectionLuna · high
Every failing section gets its own repair prompt: the section, its static findings, the failed criteria, the voice rules, and the order to rewrite in place. All sections repair in parallel. A repair is spliced back only if the heading is unchanged, no heading was added, it did not shrink, fences and image lines are byte-identical (practice fences may change, then re-validate), and it did not add a blocking finding; an offending section is reverted alone. Repaired sections are judged again.Out: 4.8 repairs per lesson, 207 model-seconds summed but about 90 s of wall time because they run together.
~90 swall
8
Donecode
Practice fences re-validated, calculators materialised, a review object synthesised from the section verdicts (status, rubric, issues) so the publisher and the deck keep working, evidence and traces written.
<1 s

Per lesson

529 s
mean wall time; p50 520 s, p90 756 s. Topics 514 s, chapters 544 s.

Cost per lesson

$0.133
mean, p90 $0.18: writer $0.051, section repairs $0.031, Jev $0.019, planner $0.012, images $0.04 each. Luna at gateway list price.

Throughput

88 / h
20 lessons in 13.7 min at concurrency 20; zero retries, zero throttling. 1,233 lessons ≈ 14 h at this concurrency.

Pass rate

17 / 20
the three failures are old static collisions (two prose runs, one malformed table) the bounded repair did not clear.

2. What the section review did on those 20 lessons

MeasureCountReading
Sections judged1175.9 per lesson
Sections with a gated failure79advisory questions excluded
Repairs attempted / applied / reverted / refused by a guard96 / 75 / 13 / 8refusals were "shrank", "fence changed", "heading changed"
"Talks to the candidate" failures before → after16 → 4the voice repair works
Practice item rationale failures before → after15 → 4fixed once practice fences were allowed to change
"Numbers from fixtures" before → after26 → 23half real: a changed-condition section invented a fresh scenario (10,000,000; 800,000) that no fixture holds, which a rewrite cannot fix. Half is Jev being conservative on tables.
"Overview stays an overview" before → after15 → 13the Learning outcomes section keeps teaching; a rewrite does not remove content. This needs a template change, not a repair.

"Nothing repeated" was asked on every prose section and failed nearly all of them, repaired or not; it is now recorded but never a trigger until calibrated.

3. Against the previous flow

Previous (whole-lesson Luna reviewer)Now (Jev per section + section repairs)
Wall time per lesson530 s at concurrency 5529 s at concurrency 20
Cost per lesson≈ $0.12$0.133
Throughput measured26 / h88 / h
Reviewer output used for anythingNo: 8 majors on 8 lessons, none fixedYes: 75 section repairs applied
Writer share of time49%58%; the writer's reasoning effort is the one lever left

4. What keeps a long run alive

5. From one lesson to all 1,233

sealed run cfa-v3-all: 203 topic overviews + 1,030 chapter lessons (2027 curriculum, staging hierarchy)

batch ladder   --review-mode jev-sections --max-lessons N --concurrency C --batch-label cC  →  batches.jsonl
               20 done (88/h, no throttling); next 50 and 100 to find the ceiling

continue       at the best concurrency with zero throttling, --resume until done; Jev token refreshed every 12 h

publish        publish-staging-v3 --run-name cfa-v3-all [--apply | --verify]
               per lesson, one transaction: lock the stub → concept groups (one per H2), blocks, mappings →
               assets to the notes bucket (sha256 paths) → assets + concept_group_assets → lesson metadata
               {stub:false, curriculum ids, cfaRevision} → read back → COMMIT. Dry-run is read only.

later          migrate-to-prod: copy by cfaRevision.runName, re-upload assets, remap URLs

6. Decisions locked

DecisionWhy
Luna writes; Luna repairsMuse Spark matched Luna on the judge's rubric at a tenth of the cost, but read in full its quantitative drafts are thin, it broke table syntax, and its reviews miss issues. Its voice is the target, not the model. GLM equalled Luna; MiMo failed two calls in three.
Jev judges, per section, with a rubricAbout a second and a fifth of a cent per call; precise because it sees one section against the plan and fixtures; its answer is a location for a repair, not a verdict to file.
Whole-lesson Luna reviewer99 s and $0.02 per lesson for a verdict nothing acted on. Its one irreplaceable job, spotting finance errors like a conflated forward value, is not covered by Jev; that is the open item below.
Repairs per section, in parallel, guardedA whole-draft rewrite for one defect lost teaching 11 times in 37 on the benchmark. A section rewrite cannot lose what it never saw.
Deterministic before AITables, headings, template labels, step lists and cut streams are handled in code, so no model call is spent on them.
Mermaid for diagrams and charts, illustrations for pictures, one per lessonChart numbers are checked against fixtures; an image model is never asked for a number.

7. Still open, in order