CFA lessons: before and after the fixes
Five lessons — two module overviews, three chapters, three levels, four subjects — generated twice on the same day with the same model: once with the pipeline as it was, once after the changes below. Every lesson, prompt and verdict on this page is the real thing; click to read it.
Runs: cfa-v3-before (before) and cfa-v3-after5m (after), 2026-09-22, gpt-5.6-luna on bedrock-mantle.responses.
How a lesson is made
Four stations. Three are the model writing; one is our code checking. When a check fails, the exact error goes back to the model once for a repair.
One real lesson, step by step Pricing and valuing equity forwards and futures, before run — click any step for its full input and output
| # | Step | Runs on | Time | |
|---|---|---|---|---|
| 1 | planner | Luna on Bedrock | 70.4 s | |
| 2 | fixtures | code | 0.0 s | |
| 3 | writer | Luna on Bedrock | 231.7 s | |
| 4 | static-checks | code | 0.0 s | |
| 5 | writer-presentation-repair | Luna on Bedrock | 57.6 s | |
| 6 | static-checks | code | 0.0 s | |
| 7 | jev-structural | Jev | 1.0 s | |
| 8 | calculators | code | 0.0 s | |
| 9 | media | code | 0.0 s | |
| 10 | practice-validation | code | 0.0 s | |
| 11 | reviewer | Luna on Bedrock | 96.8 s | |
| 12 | jev-item-qa | Jev | 0.7 s | |
| 13 | jev-item-qa | Jev | 0.6 s |
How a chart gets onto the page
| Channel | The model writes | Who draws it | Allowed to carry |
|---|---|---|---|
| Mermaid | Diagram syntax in a mermaid fence | The app, on the learner's phone | Decisions and processes. Never numbers. |
| Mermaid xychart | Chart syntax with the fixture values typed in | The app, same engine as the flowcharts | Curves, payoffs, bridges, waterfalls. Code checks every value against the fixtures; a stylised shape must be titled “Illustrative”. Cash flows over time are tables. |
| Illustration |  | GPT Image, from the query | Structure and shape only: a fund structure, a cycle, a hierarchy. A request with a number in it is rejected. |
A Mermaid fence gets a second pass. The writer's plain diagram goes to a stylist call (same Bedrock model, medium reasoning, with a validate-syntax tool it must call) that redraws it the way the medical lessons look: vertical layout, a bold title plus one or two detail lines per box taken from the surrounding section, colours by meaning (blue input, yellow decision, grey step, purple rule, red risk, green outcome, orange party). Code accepts the redraw only if it parses, keeps the diagram type and box count, uses only the whitelisted HTML and adds no number; otherwise the original stays. All fences of a lesson are styled at the same time.
Diagrams: as written, and after the stylist
11 of 11 diagrams in the after run were restyled; each pair below is drawn by the same Mermaid renderer the app uses.
flowchart TD
A["Future purchase"] --> B["Long hedge"]
C["Future sale"] --> D["Short hedge"]flowchart TD
A["<b>📜 Future purchase</b><br><span class='detail' style='display:block; text-align:left'>• Need shares at maturity</span>"]:::input --> B["<b>📈 Long hedge</b><br><span class='detail' style='display:block; text-align:left'>• Replicate delivered stock</span><br><span class='detail' style='display:block; text-align:left'>• Dividend offsets funding</span>"]:::step
C["<b>📜 Future sale</b><br><span class='detail' style='display:block; text-align:left'>• Need to sell later</span>"]:::input --> D["<b>📉 Short hedge</b><br><span class='detail' style='display:block; text-align:left'>• Hedge the future sale</span>"]:::stepxychart-beta
title "Illustrative: Long forward payoff"
x-axis ["90", "101", "108", "112"]
y-axis "Payoff" -1100 --> 1100
line [-1100, 0, 700, 1100]xychart-beta
title "Illustrative: Long forward payoff"
x-axis ["90", "101", "108", "112"]
y-axis "Payoff" -1100 --> 1100
line [-1100, 0, 700, 1100]mindmap
root((Ethics answer))
Process
Investigate
Review
Duties
Code
Standards
Decisionmindmap
root((Ethics answer))
Process
Investigate
Review
Duties
Code
Standards
Decisionflowchart TD
A["Possible violation"] --> B["Professional Conduct"]
B --> C["Evidence sufficient"]
C -->|no| D["Close inquiry"]
C -->|yes| E["Statement of Allegations"]
E --> F["Statement of Charges"]
F --> G["DRC hearing panel"]
G --> H["Written decision"]flowchart TD
A["<b>Possible allegation</b><br><span class='detail' style='display:block; text-align:left'>• Opens a review</span>"]:::input
B["<b>Professional Conduct</b><br><span class='detail' style='display:block; text-align:left'>• Reviews the information</span>"]:::step
C{"<b>Evidence sufficient?</b><br><span class='detail' style='display:block; text-align:left'>• Supports a violation</span>"}:::decision
D["<b>Close inquiry</b><br><span class='detail' style='display:block; text-align:left'>• No hearing finding</span>"]:::outcome
E["<b>Statement of Allegations</b><br><span class='detail' style='display:block; text-align:left'>• Preliminary findings</span>"]:::step
F["<b>Statement of Charges</b><br><span class='detail' style='display:block; text-align:left'>• Recommended sanction</span>"]:::step
G["<b>DRC hearing panel</b><br><span class='detail' style='display:block; text-align:left'>• Considers allegations</span>"]:::step
H["<b>Written decision</b><br><span class='detail' style='display:block; text-align:left'>• Violation and sanction</span>"]:::outcome
A --> B
B --> C
C -->| No | D
C -->| Yes | E
E --> F
F --> G
G --> Hflowchart TD
A["After-tax goals"] --> B["Check liquidity"]
B --> C["Check risk capacity"]
C --> D["Check family objective"]
D --> E["Check jurisdiction"]
E --> F["Set allocation"]
F --> G["Select investments"]flowchart TD
A["<b>💵 After-tax goals</b><br><span class='detail' style='display:block; text-align:left'>• After-tax spending</span><br><span class='detail' style='display:block; text-align:left'>• Goals by horizon</span>"]:::input
B{"<b>🧮 Liquidity adequate?</b><br><span class='detail' style='display:block; text-align:left'>• F = s x y + G</span><br><span class='detail' style='display:block; text-align:left'>• Compare with liquid A</span>"}:::decision
C{"<b>⚖️ Risk capacity sufficient?</b><br><span class='detail' style='display:block; text-align:left'>• Separate willingness</span><br><span class='detail' style='display:block; text-align:left'>• Assess risk capacity</span>"}:::decision
D{"<b>👤 Family objective clear?</b><br><span class='detail' style='display:block; text-align:left'>• Governance and transfer</span><br><span class='detail' style='display:block; text-align:left'>• Family objectives</span>"}:::decision
E{"<b>📜 Jurisdiction clear?</b><br><span class='detail' style='display:block; text-align:left'>• Tax and legal limits</span><br><span class='detail' style='display:block; text-align:left'>• Use applicable advice</span>"}:::decision
F["<b>📊 Set allocation</b><br><span class='detail' style='display:block; text-align:left'>• State mandate effect</span><br><span class='detail' style='display:block; text-align:left'>• Funding need, no target</span>"]:::step
G["<b>✅ Select investments</b><br><span class='detail' style='display:block; text-align:left'>• Within set allocation</span><br><span class='detail' style='display:block; text-align:left'>• Reflect mandate</span>"]:::step
A --> B
B -->| Yes | C
C -->| Yes | D
D -->| Yes | E
E -->| Yes | F
F --> Gtimeline
title Near-term funding path
Now : Liquid assets available
Each year-end : After-tax spending
End of horizon : Home purchasetimeline
title Near-term funding path
Now : Liquid assets available
Each year-end : After-tax spending
End of horizon : Home purchaseflowchart TD
A["Map cash flows"] --> B["Calculate price"]
B --> C["Read relationships"]
C --> D["Use comparables"]flowchart TD
A["<b>Map cash flows</b><br><span class='detail' style='display:block; text-align:left'>• Future coupons</span><br><span class='detail' style='display:block; text-align:left'>• Face-value receipt</span>"]:::step --> B["<b>Calculate price</b><br><span class='detail' style='display:block; text-align:left'>• Match yield period</span><br><span class='detail' style='display:block; text-align:left'>• Discount cash flows</span>"]:::step
B --> C["<b>Read relationships</b><br><span class='detail' style='display:block; text-align:left'>• Check price direction</span><br><span class='detail' style='display:block; text-align:left'>• Use bond structure</span>"]:::step
C --> D["<b>Use comparables</b><br><span class='detail' style='display:block; text-align:left'>• Compare similar bonds</span><br><span class='detail' style='display:block; text-align:left'>• Evidence may be thin</span>"]:::stepxychart-beta
title "Illustrative: Price against yield"
x-axis ["0.04", "0.06", "0.08"]
y-axis "Price" 991.50 --> 1039.08
line [1039.08, 1014.89, 991.50]xychart-beta
title "Illustrative: Price against yield"
x-axis ["0.04", "0.06", "0.08"]
y-axis "Price" 991.50 --> 1039.08
line [1039.08, 1014.89, 991.50]flowchart TD
A["Target bond"] --> B{"Comparable bonds?"}
B -->|yes| C["Match credit"]
C --> D["Match maturity"]
D --> E["Estimate yield"]
E --> F["Estimate price"]
B -->|no| G["Recheck evidence"]flowchart TD
A["<b>Target bond</b><br><span class='detail' style='display:block; text-align:left'>• Maturity and credit</span>"]:::input --> B{"<b>Comparable bonds?</b><br><span class='detail' style='display:block; text-align:left'>• Check observable anchors</span>"}:::decision
B -->| Yes | C["<b>Match credit</b><br><span class='detail' style='display:block; text-align:left'>• Check credit traits</span>"]:::step
C --> D["<b>Match maturity</b><br><span class='detail' style='display:block; text-align:left'>• Check maturities</span>"]:::step
D --> E["<b>Estimate yield</b><br><span class='detail' style='display:block; text-align:left'>• Use relative estimate</span>"]:::step
E --> F["<b>Estimate price</b><br><span class='detail' style='display:block; text-align:left'>• Use relative estimate</span>"]:::outcome
B -->| No | G["<b>Recheck evidence</b><br><span class='detail' style='display:block; text-align:left'>• Check observable data</span>"]:::riskflowchart TD
A["Read settlement"] --> B["Set coupon periods"]
B --> C["Find elapsed fraction"]
C --> D{"Between dates"}
D -->|yes| E["Discount cash flows"]
D -->|no| E
E --> F["Choose clean or dirty"]
F --> G["Report price"]flowchart TD
A["<b>Read settlement</b><br><span class='detail' style='display:block; text-align:left'>• Use settlement date</span>"]:::input --> B["<b>Set coupon periods</b><br><span class='detail' style='display:block; text-align:left'>• Convert to period basis</span><br><span class='detail' style='display:block; text-align:left'>• Match coupon language</span>"]:::step
B --> C["<b>Find elapsed fraction</b><br><span class='detail' style='display:block; text-align:left'>• Days since last coupon</span><br><span class='detail' style='display:block; text-align:left'>• Divide by period days</span>"]:::step
C --> D{"<b>Between coupon dates?</b><br><span class='detail' style='display:block; text-align:left'>• Test settlement timing</span>"}:::decision
D -->| Yes | E["<b>Discount cash flows</b><br><span class='detail' style='display:block; text-align:left'>• Use periodic YTM</span><br><span class='detail' style='display:block; text-align:left'>• Use t minus a timing</span>"]:::step
D -->| No | E
E --> F["<b>Choose clean or dirty</b><br><span class='detail' style='display:block; text-align:left'>• Clean excludes AI</span><br><span class='detail' style='display:block; text-align:left'>• Dirty includes AI</span>"]:::step
F --> G["<b>Report price</b><br><span class='detail' style='display:block; text-align:left'>• Report selected price</span><br><span class='detail' style='display:block; text-align:left'>• Clean or dirty price</span>"]:::outcomexychart-beta
title "Illustrative: Price against annual YTM"
x-axis ["0.04", "0.06", "0.08"]
y-axis "Price" 963.70 --> 1038.08
line [1038.08, 1000.00, 963.70]xychart-beta
title "Illustrative: Price against annual YTM"
x-axis ["0.04", "0.06", "0.08"]
y-axis "Price" 963.70 --> 1038.08
line [1038.08, 1000.00, 963.70]Illustrations: the third channel, tried on four lessons
Where a table or a chart cannot show a structure — who owns what, what sits between whom, which phase follows which — an image can. The first run proved the channel works and showed its failure mode in one picture.
(Image: business forms liability walls)

The image model filled the gap itself: an infographic of LLCs, S and C corporations and pass-through taxation — none of which is in the CFA curriculum — plus a slogan. Every word in it is an unverified claim in the learner's lesson.
(Image: three side-by-side panels, each showing one or more owner figures standing above a small business building; … a solid brick wall standing between them and the building)

The writer must now describe exactly what to draw in at least twelve words, naming only the lesson's own entities (image_request_too_thin otherwise), and every request is framed as a wordless flat illustration. The caption and the prose carry the words.
The run below (cfa-v3-images2) is the four candidate lessons generated after the fixes. The same run also showed the old chart channel failing silently: writers emitted JSON chart specs in shapes the renderer could not read and the media stage dropped them with only a warning. That channel is gone; charts are now Mermaid xychart fences the app draws, with every value checked against the fixtures.
| Lesson | Status | Figures drawn | Figures dropped | Images | Diagrams styled | Rubric | |
|---|---|---|---|---|---|---|---|
| Sole proprietorships, partnerships and corporations Level I · Corporate Finance · chapter ![]() Owners, the business, and the liability boundary across three organizational forms |
ok | 0 | 0 | 1 | 0 / 0 | 23 | |
| Private investment methods and structures Level III · Private Markets Pathway · chapter |
ok | 0 | 0 | 0 | 3 / 3 | 22 | |
| Phases of the business cycle Level I · Economics · chapter |
ok | 0 | 0 | 0 | 1 / 1 | 22 | |
| Bond price from yield-to-maturity between coupon dates Level I · Fixed Income · chapter |
failed | 0 | 0 | 0 | 0 / 0 | 21 |
What was wrong
| Baseline lesson | Words | Tables | Figures | Longest prose run |
|---|---|---|---|---|
| Pricing and valuing equity forwards and futuresLevel II · Derivatives and Risk Management · chapter | 1,409 | 6 | 1 | 16 paragraphs |
| Code of Ethics and Standards of Professional ConductLevel III · Ethical and Professional Standards · topic | 735 | 3 | 2 | 4 paragraphs |
| Recommending portfolio allocations and investments for a private clientLevel III · Private Wealth Pathway · chapter | 757 | 2 | 2 | 6 paragraphs |
| Fixed-Income Bond Valuation: Prices and YieldsLevel I · Fixed Income · topic | 2,202 | 8 | 0 | 14 paragraphs |
| Bond price from yield-to-maturity between coupon datesLevel I · Fixed Income · chapter | 1,229 | 5 | 0 | 13 paragraphs |
"Longest prose run" is the most consecutive paragraphs a reader meets with no table, figure or heading between them. Medical lessons mandate a table or diagram in every section; these had none for 13, 14 and 16 paragraphs at a stretch.
Three other things the earlier hand audit found, still true in the baseline: the planner prompt was a rule dump the model could not follow (it contradicted itself on headings), a repair was free to return less than it was given (one repair dropped eight sections and both worked examples over a single bad key), and the reviewer requested changes on every lesson for source reasons while flagging none of the learner-facing defects.
What changed
Prompts written as a brief
Modelled on the medical lesson prompts: a role, the job in plain words, how this subject teaches, then a section-by-section template that names the table or figure each section owes, then the constraints code enforces. The plan reaches the writer as a brief, not a JSON dump it used to transcribe.
Visuals are a floor, not a suggestion
Each subject's config sets how many tables and figures a lesson owes. Two new static rules enforce it — visual_budget_unmet and prose_run_too_long (more than four paragraphs with nothing between them) — and a failing lesson gets one repair told to add the planned figure where the prose compares things, not to cut prose.
Jev asks graded questions from the rubric
Instead of "is a trap present?" (answered yes 96% of the time), each unit is scored 0–2 on LOS verb alignment, teaching vs reference, whether the worked example is a real scenario, and whether the visuals do work. Overviews are asked whether they orient, not whether they teach. Advisory until calibrated; recorded on every lesson.
Diagrams styled like the medical lessons
The writer keeps emitting plain Mermaid — a decision or a process, at most eight boxes, no numbers. A separate stylist call then redraws each fence phone-first with titles, detail lines and semantic colours, and code refuses any redraw that changes the diagram. See the pairs below.
A repair cannot shrink the work
A repaired plan is compared with the first attempt; losing fixtures, snapshot lines, traps or visuals is fed back as a defect for one more try. A repaired lesson that loses a unit section, a table or figure, a practice block, or a sixth of its words is refused and the original kept, with the refusal on record.
What the first re-run taught
The first run on the new prompts failed three of the five lessons, and both reasons were mine. The prose-run rule counted a worked example's alternating steps and $$ equations as an eight-paragraph wall and failed a chapter that had already gained a figure and a table; display math no longer counts as prose. And the brief that replaced the plan JSON dump lost one fact the dump carried implicitly — which unit owns the practice fence — so two overviews wrote a chapter id where overview-3 belonged. The templates now say it. The run below is the corrected one; the first is kept in cfa-v3-after.
Before and after, lesson by lesson
Green is better, red is worse; word counts are prose only (tables and fences excluded) and are a fact, not a score. The rubric is the tutor's 0–2 score on twelve criteria; both runs were scored after the fact by the same reviewer prompt, so the column compares like with like. "Repairs" counts bounded repair calls; "(refused)" means the gate kept the original.
| Lesson | Words | Tables | Figures | Longest run | Major findings | Repairs | Rubric | After | |
|---|---|---|---|---|---|---|---|---|---|
| Pricing and valuing equity forwards and futuresLevel II · Derivatives and Risk Management · chapter | 1,409 → 1,023 | 6 → 6 | 1 → 2 | 16 → 4 | 0 → 0 | 1 → 1 | 22 → 25 / 24 | ok | |
| Code of Ethics and Standards of Professional ConductLevel III · Ethical and Professional Standards · topic | 735 → 866 | 3 → 10 | 2 → 2 | 4 → 4 | 0 → 0 | 1 → 0 | 13 → 22 / 24 | ok | |
| Recommending portfolio allocations and investments for a private clientLevel III · Private Wealth Pathway · chapter | 757 → 847 | 2 → 4 | 2 → 2 | 6 → 8 | 0 → 0 | 1 → 0 | 22 → 23 / 24 | ok | |
| Fixed-Income Bond Valuation: Prices and YieldsLevel I · Fixed Income · topic | 2,202 → 1,362 | 8 → 14 | 0 → 3 | 14 → 6 | 0 → 2 | 1 → 1 | 19 → 20 / 24 | failed | |
| Bond price from yield-to-maturity between coupon datesLevel I · Fixed Income · chapter | 1,229 → 1,153 | 5 → 6 | 0 → 2 | 13 → 10 | 0 → 1 | 0 → 1 | 20 → 24 / 24 | failed |
What is next
- Calibrate the rubric questions against a labelled set with negatives before any of them gates a repair; today they are recorded, not enforced.
- Patch-scoped repairs. The gate stops a repair from shrinking the work; the next step is asking the model for only the broken piece and splicing it in ourselves.
- Module overviews assembled from chapters rather than free-written, once the runner generates a module's chapters before its overview.
- Re-run the sixty-lesson benchmark on the new prompts and re-audit eight by hand, targeting seven of eight publishable (four of eight before).
