CFA lessons: before and after the fixes

Five lessons — two module overviews, three chapters, three levels, four subjects — generated twice on the same day with the same model: once with the pipeline as it was, once after the changes below. Every lesson, prompt and verdict on this page is the real thing; click to read it.

Runs: cfa-v3-before (before) and cfa-v3-after5m (after), 2026-09-22, gpt-5.6-luna on bedrock-mantle.responses.

5 → 11
figures across the 5 lessons
24 → 40
tables
16 → 10
longest run of paragraphs with no table or figure
4 → 5 of 5
reach 16/24 on the tutor rubric, both runs scored by the same reviewer after the fact

How a lesson is made

Four stations. Three are the model writing; one is our code checking. When a check fails, the exact error goes back to the model once for a repair.

AI · planPlannerReads the official learning outcome and how this subject teaches. Writes the blueprint: examples, visuals, the trap, the exam angle.
AI · writeWriterTurns the blueprint into the lesson. Every number comes from fixtures — values our code computed, not the model.
code + jev · checkChecksStatic rules (headings, math, tables, figures, the visual floor), then Jev — a small judge asked yes/no and graded questions.
AI · judgeReviewerA tutor scores the twelve-criterion rubric and lists every numeric claim so code can recompute it. It never rewrites.

One real lesson, step by step Pricing and valuing equity forwards and futures, before run — click any step for its full input and output

#StepRuns onTime
1plannerLuna on Bedrock70.4 s
2fixturescode0.0 s
3writerLuna on Bedrock231.7 s
4static-checkscode0.0 s
5writer-presentation-repairLuna on Bedrock57.6 s
6static-checkscode0.0 s
7jev-structuralJev1.0 s
8calculatorscode0.0 s
9mediacode0.0 s
10practice-validationcode0.0 s
11reviewerLuna on Bedrock96.8 s
12jev-item-qaJev0.7 s
13jev-item-qaJev0.6 s

How a chart gets onto the page

ChannelThe model writesWho draws itAllowed to carry
MermaidDiagram syntax in a mermaid fenceThe app, on the learner's phoneDecisions and processes. Never numbers.
Mermaid xychartChart syntax with the fixture values typed inThe app, same engine as the flowchartsCurves, payoffs, bridges, waterfalls. Code checks every value against the fixtures; a stylised shape must be titled “Illustrative”. Cash flows over time are tables.
Illustration![description](Image: query)GPT Image, from the queryStructure and shape only: a fund structure, a cycle, a hierarchy. A request with a number in it is rejected.

A Mermaid fence gets a second pass. The writer's plain diagram goes to a stylist call (same Bedrock model, medium reasoning, with a validate-syntax tool it must call) that redraws it the way the medical lessons look: vertical layout, a bold title plus one or two detail lines per box taken from the surrounding section, colours by meaning (blue input, yellow decision, grey step, purple rule, red risk, green outcome, orange party). Code accepts the redraw only if it parses, keeps the diagram type and box count, uses only the whitelisted HTML and adds no number; otherwise the original stays. All fences of a lesson are styled at the same time.

Diagrams: as written, and after the stylist

11 of 11 diagrams in the after run were restyled; each pair below is drawn by the same Mermaid renderer the app uses.

Pricing and valuing equity forwards and futures chapter · flowchart-v2 1 · styled in 11 s
As written
flowchart TD
A["Future purchase"] --> B["Long hedge"]
C["Future sale"] --> D["Short hedge"]
After the stylist
flowchart TD
A["<b>📜 Future purchase</b><br><span class='detail' style='display:block; text-align:left'>• Need shares at maturity</span>"]:::input --> B["<b>📈 Long hedge</b><br><span class='detail' style='display:block; text-align:left'>• Replicate delivered stock</span><br><span class='detail' style='display:block; text-align:left'>• Dividend offsets funding</span>"]:::step
C["<b>📜 Future sale</b><br><span class='detail' style='display:block; text-align:left'>• Need to sell later</span>"]:::input --> D["<b>📉 Short hedge</b><br><span class='detail' style='display:block; text-align:left'>• Hedge the future sale</span>"]:::step
Pricing and valuing equity forwards and futures chapter · xychart 2 · styled in 0 s
As written
xychart-beta
    title "Illustrative: Long forward payoff"
    x-axis ["90", "101", "108", "112"]
    y-axis "Payoff" -1100 --> 1100
    line [-1100, 0, 700, 1100]
After the stylist
xychart-beta
    title "Illustrative: Long forward payoff"
    x-axis ["90", "101", "108", "112"]
    y-axis "Payoff" -1100 --> 1100
    line [-1100, 0, 700, 1100]
Code of Ethics and Standards of Professional Conduct topic · mindmap 1 · styled in 0 s
As written
mindmap
  root((Ethics answer))
    Process
      Investigate
      Review
    Duties
      Code
      Standards
    Decision
After the stylist
mindmap
  root((Ethics answer))
    Process
      Investigate
      Review
    Duties
      Code
      Standards
    Decision
Code of Ethics and Standards of Professional Conduct topic · flowchart-v2 2 · styled in 9 s
As written
flowchart TD
A["Possible violation"] --> B["Professional Conduct"]
B --> C["Evidence sufficient"]
C -->|no| D["Close inquiry"]
C -->|yes| E["Statement of Allegations"]
E --> F["Statement of Charges"]
F --> G["DRC hearing panel"]
G --> H["Written decision"]
After the stylist
flowchart TD
A["<b>Possible allegation</b><br><span class='detail' style='display:block; text-align:left'>• Opens a review</span>"]:::input
B["<b>Professional Conduct</b><br><span class='detail' style='display:block; text-align:left'>• Reviews the information</span>"]:::step
C{"<b>Evidence sufficient?</b><br><span class='detail' style='display:block; text-align:left'>• Supports a violation</span>"}:::decision
D["<b>Close inquiry</b><br><span class='detail' style='display:block; text-align:left'>• No hearing finding</span>"]:::outcome
E["<b>Statement of Allegations</b><br><span class='detail' style='display:block; text-align:left'>• Preliminary findings</span>"]:::step
F["<b>Statement of Charges</b><br><span class='detail' style='display:block; text-align:left'>• Recommended sanction</span>"]:::step
G["<b>DRC hearing panel</b><br><span class='detail' style='display:block; text-align:left'>• Considers allegations</span>"]:::step
H["<b>Written decision</b><br><span class='detail' style='display:block; text-align:left'>• Violation and sanction</span>"]:::outcome
A --> B
B --> C
C -->| No | D
C -->| Yes | E
E --> F
F --> G
G --> H
Recommending portfolio allocations and investments for a private client chapter · flowchart-v2 1 · styled in 23 s
As written
flowchart TD
A["After-tax goals"] --> B["Check liquidity"]
B --> C["Check risk capacity"]
C --> D["Check family objective"]
D --> E["Check jurisdiction"]
E --> F["Set allocation"]
F --> G["Select investments"]
After the stylist
flowchart TD
A["<b>💵 After-tax goals</b><br><span class='detail' style='display:block; text-align:left'>• After-tax spending</span><br><span class='detail' style='display:block; text-align:left'>• Goals by horizon</span>"]:::input
B{"<b>🧮 Liquidity adequate?</b><br><span class='detail' style='display:block; text-align:left'>• F = s x y + G</span><br><span class='detail' style='display:block; text-align:left'>• Compare with liquid A</span>"}:::decision
C{"<b>⚖️ Risk capacity sufficient?</b><br><span class='detail' style='display:block; text-align:left'>• Separate willingness</span><br><span class='detail' style='display:block; text-align:left'>• Assess risk capacity</span>"}:::decision
D{"<b>👤 Family objective clear?</b><br><span class='detail' style='display:block; text-align:left'>• Governance and transfer</span><br><span class='detail' style='display:block; text-align:left'>• Family objectives</span>"}:::decision
E{"<b>📜 Jurisdiction clear?</b><br><span class='detail' style='display:block; text-align:left'>• Tax and legal limits</span><br><span class='detail' style='display:block; text-align:left'>• Use applicable advice</span>"}:::decision
F["<b>📊 Set allocation</b><br><span class='detail' style='display:block; text-align:left'>• State mandate effect</span><br><span class='detail' style='display:block; text-align:left'>• Funding need, no target</span>"]:::step
G["<b>✅ Select investments</b><br><span class='detail' style='display:block; text-align:left'>• Within set allocation</span><br><span class='detail' style='display:block; text-align:left'>• Reflect mandate</span>"]:::step
A --> B
B -->| Yes | C
C -->| Yes | D
D -->| Yes | E
E -->| Yes | F
F --> G
Recommending portfolio allocations and investments for a private client chapter · timeline 2 · styled in 0 s
As written
timeline
    title Near-term funding path
    Now : Liquid assets available
    Each year-end : After-tax spending
    End of horizon : Home purchase
After the stylist
timeline
    title Near-term funding path
    Now : Liquid assets available
    Each year-end : After-tax spending
    End of horizon : Home purchase
Fixed-Income Bond Valuation: Prices and Yields topic · flowchart-v2 1 · styled in 8 s
As written
flowchart TD
A["Map cash flows"] --> B["Calculate price"]
B --> C["Read relationships"]
C --> D["Use comparables"]
After the stylist
flowchart TD
A["<b>Map cash flows</b><br><span class='detail' style='display:block; text-align:left'>• Future coupons</span><br><span class='detail' style='display:block; text-align:left'>• Face-value receipt</span>"]:::step --> B["<b>Calculate price</b><br><span class='detail' style='display:block; text-align:left'>• Match yield period</span><br><span class='detail' style='display:block; text-align:left'>• Discount cash flows</span>"]:::step
B --> C["<b>Read relationships</b><br><span class='detail' style='display:block; text-align:left'>• Check price direction</span><br><span class='detail' style='display:block; text-align:left'>• Use bond structure</span>"]:::step
C --> D["<b>Use comparables</b><br><span class='detail' style='display:block; text-align:left'>• Compare similar bonds</span><br><span class='detail' style='display:block; text-align:left'>• Evidence may be thin</span>"]:::step
Fixed-Income Bond Valuation: Prices and Yields topic · xychart 2 · styled in 0 s
As written
xychart-beta
    title "Illustrative: Price against yield"
    x-axis ["0.04", "0.06", "0.08"]
    y-axis "Price" 991.50 --> 1039.08
    line [1039.08, 1014.89, 991.50]
After the stylist
xychart-beta
    title "Illustrative: Price against yield"
    x-axis ["0.04", "0.06", "0.08"]
    y-axis "Price" 991.50 --> 1039.08
    line [1039.08, 1014.89, 991.50]
Fixed-Income Bond Valuation: Prices and Yields topic · flowchart-v2 3 · styled in 15 s
As written
flowchart TD
A["Target bond"] --> B{"Comparable bonds?"}
B -->|yes| C["Match credit"]
C --> D["Match maturity"]
D --> E["Estimate yield"]
E --> F["Estimate price"]
B -->|no| G["Recheck evidence"]
After the stylist
flowchart TD
A["<b>Target bond</b><br><span class='detail' style='display:block; text-align:left'>• Maturity and credit</span>"]:::input --> B{"<b>Comparable bonds?</b><br><span class='detail' style='display:block; text-align:left'>• Check observable anchors</span>"}:::decision
B -->| Yes | C["<b>Match credit</b><br><span class='detail' style='display:block; text-align:left'>• Check credit traits</span>"]:::step
C --> D["<b>Match maturity</b><br><span class='detail' style='display:block; text-align:left'>• Check maturities</span>"]:::step
D --> E["<b>Estimate yield</b><br><span class='detail' style='display:block; text-align:left'>• Use relative estimate</span>"]:::step
E --> F["<b>Estimate price</b><br><span class='detail' style='display:block; text-align:left'>• Use relative estimate</span>"]:::outcome
B -->| No | G["<b>Recheck evidence</b><br><span class='detail' style='display:block; text-align:left'>• Check observable data</span>"]:::risk
Bond price from yield-to-maturity between coupon dates chapter · flowchart-v2 1 · styled in 14 s
As written
flowchart TD
A["Read settlement"] --> B["Set coupon periods"]
B --> C["Find elapsed fraction"]
C --> D{"Between dates"}
D -->|yes| E["Discount cash flows"]
D -->|no| E
E --> F["Choose clean or dirty"]
F --> G["Report price"]
After the stylist
flowchart TD
A["<b>Read settlement</b><br><span class='detail' style='display:block; text-align:left'>• Use settlement date</span>"]:::input --> B["<b>Set coupon periods</b><br><span class='detail' style='display:block; text-align:left'>• Convert to period basis</span><br><span class='detail' style='display:block; text-align:left'>• Match coupon language</span>"]:::step
B --> C["<b>Find elapsed fraction</b><br><span class='detail' style='display:block; text-align:left'>• Days since last coupon</span><br><span class='detail' style='display:block; text-align:left'>• Divide by period days</span>"]:::step
C --> D{"<b>Between coupon dates?</b><br><span class='detail' style='display:block; text-align:left'>• Test settlement timing</span>"}:::decision
D -->| Yes | E["<b>Discount cash flows</b><br><span class='detail' style='display:block; text-align:left'>• Use periodic YTM</span><br><span class='detail' style='display:block; text-align:left'>• Use t minus a timing</span>"]:::step
D -->| No | E
E --> F["<b>Choose clean or dirty</b><br><span class='detail' style='display:block; text-align:left'>• Clean excludes AI</span><br><span class='detail' style='display:block; text-align:left'>• Dirty includes AI</span>"]:::step
F --> G["<b>Report price</b><br><span class='detail' style='display:block; text-align:left'>• Report selected price</span><br><span class='detail' style='display:block; text-align:left'>• Clean or dirty price</span>"]:::outcome
Bond price from yield-to-maturity between coupon dates chapter · xychart 2 · styled in 0 s
As written
xychart-beta
    title "Illustrative: Price against annual YTM"
    x-axis ["0.04", "0.06", "0.08"]
    y-axis "Price" 963.70 --> 1038.08
    line [1038.08, 1000.00, 963.70]
After the stylist
xychart-beta
    title "Illustrative: Price against annual YTM"
    x-axis ["0.04", "0.06", "0.08"]
    y-axis "Price" 963.70 --> 1038.08
    line [1038.08, 1000.00, 963.70]

Illustrations: the third channel, tried on four lessons

Where a table or a chart cannot show a structure — who owns what, what sits between whom, which phase follows which — an image can. The first run proved the channel works and showed its failure mode in one picture.

Organizational forms and the liability wall Corporate Issuers, Level I chapter · same lesson, two ways of asking
First run: a three-word query

(Image: business forms liability walls)

Generated infographic that invents LLC, S corporation and C corporation content

The image model filled the gap itself: an infographic of LLCs, S and C corporations and pass-through taxation — none of which is in the CFA curriculum — plus a slogan. Every word in it is an unverified claim in the learner's lesson.

After the fix: a descriptive query, no text allowed

(Image: three side-by-side panels, each showing one or more owner figures standing above a small business building; … a solid brick wall standing between them and the building)

Three panels: one owner, three owners and many owners above a shop; dotted lines in the first two, a brick wall in the third

The writer must now describe exactly what to draw in at least twelve words, naming only the lesson's own entities (image_request_too_thin otherwise), and every request is framed as a wordless flat illustration. The caption and the prose carry the words.

The run below (cfa-v3-images2) is the four candidate lessons generated after the fixes. The same run also showed the old chart channel failing silently: writers emitted JSON chart specs in shapes the renderer could not read and the media stage dropped them with only a warning. That channel is gone; charts are now Mermaid xychart fences the app draws, with every value checked against the fixtures.

LessonStatusFigures drawnFigures droppedImagesDiagrams styledRubric
Sole proprietorships, partnerships and corporations
Level I · Corporate Finance · chapter
Owners, the business, and the liability boundary across three organizational forms
Owners, the business, and the liability boundary across three organizational forms
ok 00 10 / 0 23
Private investment methods and structures
Level III · Private Markets Pathway · chapter
ok 00 03 / 3 22
Phases of the business cycle
Level I · Economics · chapter
ok 00 01 / 1 22
Bond price from yield-to-maturity between coupon dates
Level I · Fixed Income · chapter
failed 00 00 / 0 21

What was wrong

The lessons read like walls of text because nothing made them not. The subject config already says a fixed-income lesson needs a cash-flow timeline and a price curve; the writer prompt said "do not force a visual" and never said where one goes.
Baseline lessonWordsTablesFiguresLongest prose run
Pricing and valuing equity forwards and futuresLevel II · Derivatives and Risk Management · chapter1,4096116 paragraphs
Code of Ethics and Standards of Professional ConductLevel III · Ethical and Professional Standards · topic735324 paragraphs
Recommending portfolio allocations and investments for a private clientLevel III · Private Wealth Pathway · chapter757226 paragraphs
Fixed-Income Bond Valuation: Prices and YieldsLevel I · Fixed Income · topic2,2028014 paragraphs
Bond price from yield-to-maturity between coupon datesLevel I · Fixed Income · chapter1,2295013 paragraphs

"Longest prose run" is the most consecutive paragraphs a reader meets with no table, figure or heading between them. Medical lessons mandate a table or diagram in every section; these had none for 13, 14 and 16 paragraphs at a stretch.

Three other things the earlier hand audit found, still true in the baseline: the planner prompt was a rule dump the model could not follow (it contradicted itself on headings), a repair was free to return less than it was given (one repair dropped eight sections and both worked examples over a single bad key), and the reviewer requested changes on every lesson for source reasons while flagging none of the learner-facing defects.

What changed

Prompts written as a brief

Modelled on the medical lesson prompts: a role, the job in plain words, how this subject teaches, then a section-by-section template that names the table or figure each section owes, then the constraints code enforces. The plan reaches the writer as a brief, not a JSON dump it used to transcribe.

Visuals are a floor, not a suggestion

Each subject's config sets how many tables and figures a lesson owes. Two new static rules enforce it — visual_budget_unmet and prose_run_too_long (more than four paragraphs with nothing between them) — and a failing lesson gets one repair told to add the planned figure where the prose compares things, not to cut prose.

Jev asks graded questions from the rubric

Instead of "is a trap present?" (answered yes 96% of the time), each unit is scored 0–2 on LOS verb alignment, teaching vs reference, whether the worked example is a real scenario, and whether the visuals do work. Overviews are asked whether they orient, not whether they teach. Advisory until calibrated; recorded on every lesson.

Diagrams styled like the medical lessons

The writer keeps emitting plain Mermaid — a decision or a process, at most eight boxes, no numbers. A separate stylist call then redraws each fence phone-first with titles, detail lines and semantic colours, and code refuses any redraw that changes the diagram. See the pairs below.

A repair cannot shrink the work

A repaired plan is compared with the first attempt; losing fixtures, snapshot lines, traps or visuals is fed back as a defect for one more try. A repaired lesson that loses a unit section, a table or figure, a practice block, or a sixth of its words is refused and the original kept, with the refusal on record.

What the first re-run taught

The first run on the new prompts failed three of the five lessons, and both reasons were mine. The prose-run rule counted a worked example's alternating steps and $$ equations as an eight-paragraph wall and failed a chapter that had already gained a figure and a table; display math no longer counts as prose. And the brief that replaced the plan JSON dump lost one fact the dump carried implicitly — which unit owns the practice fence — so two overviews wrote a chapter id where overview-3 belonged. The templates now say it. The run below is the corrected one; the first is kept in cfa-v3-after.

Before and after, lesson by lesson

Green is better, red is worse; word counts are prose only (tables and fences excluded) and are a fact, not a score. The rubric is the tutor's 0–2 score on twelve criteria; both runs were scored after the fact by the same reviewer prompt, so the column compares like with like. "Repairs" counts bounded repair calls; "(refused)" means the gate kept the original.

LessonWordsTablesFiguresLongest runMajor findingsRepairsRubricAfter
Pricing and valuing equity forwards and futuresLevel II · Derivatives and Risk Management · chapter 1,409 → 1,023 6 → 6 1 → 2 16 → 4 0 → 0 1 → 1 22 → 25 / 24 ok
Code of Ethics and Standards of Professional ConductLevel III · Ethical and Professional Standards · topic 735 → 866 3 → 10 2 → 2 4 → 4 0 → 0 1 → 0 13 → 22 / 24 ok
Recommending portfolio allocations and investments for a private clientLevel III · Private Wealth Pathway · chapter 757 → 847 2 → 4 2 → 2 6 → 8 0 → 0 1 → 0 22 → 23 / 24 ok
Fixed-Income Bond Valuation: Prices and YieldsLevel I · Fixed Income · topic 2,202 → 1,362 8 → 14 0 → 3 14 → 6 0 → 2 1 → 1 19 → 20 / 24 failed
Bond price from yield-to-maturity between coupon datesLevel I · Fixed Income · chapter 1,229 → 1,153 5 → 6 0 → 2 13 → 10 0 → 1 0 → 1 20 → 24 / 24 failed

What is next