{ "module": "FT23 — Synthesis: Uncensor the Model, Harness the Model", "course": "3 — LLM Fine-Tuning Masterclass", "version": "1.0.0", "duration_minutes": 40, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the synthesis principle that module FT23 lands?", "options": [ "Uncensor the model so it executes; harness the model so it executes only what it should.", "Fine-tuning steers behavior; it does not teach knowledge.", "The model is 1.6% of an agent; the harness is 98.4%.", "Abliterate the model and deploy it; the harness is optional for chatbots." ], "answer_index": 0, "rationale": "The synthesis: 'Uncensor the model so it executes; harness the model so it executes only what it should.' Option B is the Course 3 thesis (FT00); option C is Course 1's thesis; option D is the dangerous error. FT23's principle joins all three: the model steers (does), the harness bounds (may)." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "State the absolute rule of FT23.", "options": [ "Never deploy an uncensored model in any environment; always use refusal-trained models.", "Never deploy an uncensored model without an eval'd harness whose policy gates you have hardened AND whose threat model you have hardened for the absence of model-level refusal.", "Never abliterate without first running GRPO to recover the lost capability.", "Never use an open-weights-only base for an uncensored deployment; open-data is mandatory." ], "answer_index": 1, "rationale": "The absolute rule requires TWO separate hardening steps: the policy gates must be eval'd AND the threat model must be re-written for the absence of model-level refusal. Option A is wrong (uncensored is defensible inside an eval'd harness). The rule is about the harness, not the base openness or the recovery training." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "Which four properties distinguish a harness policy gate (Layer 5) from a model-level refusal (Layer 3)?", "options": [ "Faster, cheaper, more accurate, more capable.", "Auditable, deterministic, tunable to doctrine, revisable without retraining.", "Open-data, open-weights, Apache-2.0, BAA-free.", "Lower VRAM, higher throughput, lower latency, better quantization." ], "answer_index": 1, "rationale": "The four properties: Auditable (logged gate decisions), Deterministic (same action + same policy = same decision), Tunable to doctrine (per-deployment policy file), Revisable without retraining (edit the policy, redeploy, no GPU). A model-level refusal has none of these — it is a non-auditable, non-deterministic, non-tunable, non-revisable black box." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "A hospital wants a clinical-analytics agent that queries a data warehouse and drafts reports. The team proposes abliterating a reasoning-capable base and deploying it. Per FT23, what is the first problem with this plan?", "options": [ "Abliteration is illegal in healthcare settings.", "Nothing — abliteration is the right first step for an agent that must execute.", "The plan skips the harness. The first question is not 'how do we uncensor' but 'what is the eval'd harness whose policy gates replace the refusal layer we are removing.' Without that, the architecture lands in the forbidden uncensored + weak-harness quadrant.", "The plan should use continued pretraining instead of abliteration to teach the model HIPAA." ], "answer_index": 2, "rationale": "Per FT23, uncensoring is only defensible inside an eval'd harness. The architecture must be specified end-to-end (Layers 1–5) before any steering decision. Skipping the harness lands in the strictly-most-dangerous quadrant. Option B is the dangerous error; abliteration is never the first step — the harness architecture is." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "Your tool-use agent halts mid-loop because the model refused to formulate a legitimate in-scope SQL query. The team's fix is to 're-prompt with more context.' Per the tool-use agent argument, what is the actual problem and the correct fix?", "options": [ "The prompt was bad; the fix is better prompt engineering. Refusal should stay in the model.", "The problem is that refusal lives in the weights (Layer 3), where it is non-deterministic and breaks the agent loop. The correct fix is to move refusal to a harness policy gate (Layer 5) — steer the model to execute, bound the executed actions in the gate.", "The problem is the harness is too strict; relax the policy gates.", "The model is too small; use a larger base that refuses less often." ], "answer_index": 1, "rationale": "A model that refuses mid-loop is a broken agent (a production outage, not a UX annoyance). Re-prompting is non-deterministic patching of a Layer 3 problem. The synthesis fix: steer the model to execute (abliterate or DPO-toward-compliance) and move the refusal policy to a Layer 5 gate where it is auditable, deterministic, doctrine-tunable, and revisable without retraining." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "A red-team lab and a hospital both want the same uncensored base model. Per FT23, what must differ between the two deployments?", "options": [ "Nothing — the same uncensored model works for both.", "The base model must be retrained for each environment.", "The doctrine (the Layer 5 policy file) and the threat model. Same model, different policy gates: the lab permits offensive calls within the scope-letter CIDRs; the hospital refuses anything that could exfiltrate PHI. Both threat models are re-written for the absence of model-level refusal.", "Only the quantization format — the lab uses GGUF, the hospital uses AWQ." ], "answer_index": 2, "rationale": "Doctrine is the deployment-specific policy the harness enforces. Same model + different policy file = different deployment. This is precisely what a model-level refusal CANNOT provide (its refusal direction is the same for all inputs) — and it is the core argument for moving refusal to Layer 5. No retraining is required to change doctrine." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "You are writing the re-done threat model after abliterating a model. Which approach is correct per FT23?", "options": [ "Copy the pre-uncensoring threat model unchanged — the harness handles everything new.", "Delete the threat model — the policy gates replace it entirely.", "Re-examine every attack path that previously assumed 'the model will refuse.' For each: show the harness gate now catches it, or mitigate it at the harness, or accept the residual risk explicitly. Both the gates AND the threat model must be hardened.", "Add abliteration to the threat model as a new control and leave the rest unchanged." ], "answer_index": 2, "rationale": "The absolute rule requires two hardening steps: the gates AND the threat model. The threat model must be re-written because removing model-level refusal silently breaks every attack path that assumed the model would refuse. Option A is the 'half-finished hardening' anti-pattern; option B confuses the gate (a control) with the threat model (the analysis); option D is incoherent." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "A team has hardened the policy gates and eval'd them, but has NOT re-written the threat model for the absence of model-level refusal. Where does their architecture sit on the FT23 risk matrix?", "options": [ "In the safe quadrant (uncensored + eval'd harness) — the gates are what matter.", "Still in the forbidden zone — the absolute rule requires BOTH hardening steps. Without the threat-model rewrite, attack paths that assumed 'the model will refuse' are silently broken. The architecture is not yet defensible.", "In the refusal-trained + eval'd harness quadrant — they have effectively kept the model layer.", "It cannot be classified without knowing the model size." ], "answer_index": 1, "rationale": "Hardening is two steps: gates AND threat model. Skipping the threat-model rewrite is the 'half-finished hardening' anti-pattern. The architecture is not yet in the safe quadrant because every 'the model will refuse' path is now an unexamined hole. Both steps are required by the absolute rule." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "After this module, a student asks what to take next. They want to deploy an uncensored model for an internal tool-use agent. What is the correct course sequence and why?", "options": [ "Course 2B (red-team) first, then 2A, then 1 — break the harness before you build it.", "Course 1 (build the harness) first — you cannot secure or red-team a harness you have not built. Then 2A if the domain is security. Then 2B, always, because every harness gets red-teamed eventually; mandatory if you deployed uncensored.", "Skip to Capstone 2 — the architecture is enough to deploy.", "None — Course 3 is sufficient to deploy an uncensored agent." ], "answer_index": 1, "rationale": "The order is not arbitrary. Course 1 first (build Layer 5). Then 2A for security-domain doctrine. Then 2B — always, because the model layer is gone and the harness is the only layer left; it must be red-teamed before an adversary does it. Option A inverts the order; options C and D ship without the harness the synthesis requires." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is abliteration described as 'not auditable,' and why does that matter for a regulated deployment?", "options": [ "Because the abliteration tools are closed-source and cannot be inspected.", "Because the refusal behavior is now a probability surface in a billion-dimensional residual stream, not a rule. You cannot point an auditor to 'the refusal policy,' cannot prove what the model will or will not do, and can only report sampled behavior. A regulator needs a written, deterministic, logged rule — which is exactly what a Layer 5 policy gate provides.", "Because abliteration always breaks the model and cannot be tested.", "Because auditors do not understand residual streams." ], "answer_index": 1, "rationale": "Abliteration removes a direction in the residual stream; the resulting 'refusal behavior' is a probability surface, not a rule. You cannot replay 'why did the model refuse/refuse-not at step 4.' For a regulated deployment, the control must be a written, deterministic, logged rule — the harness policy gate. Option A is factually wrong (abliteration is open); the others miss the substance." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "An engineer argues: 'We have a refusal-trained model, so we don't need harness policy gates.' How does FT23 refute this, even for a non-agent chatbot deployment?", "options": [ "FT23 agrees — refusal-trained models are sufficient without gates.", "A model-level refusal is non-auditable, non-deterministic, non-tunable to doctrine, and non-revisable without retraining. Even for a chatbot, the 'safety' it provides is a black box you cannot prove to a regulator, cannot tune per-deployment, and cannot update without a GPU. The policy gate is the opposite on all four axes. They are not equivalent; the gate is the defensible control.", "FT23 refutes it only for agents; for chatbots, refusal-trained models are enough.", "The argument is correct for HIPAA but wrong for red-team labs." ], "answer_index": 1, "rationale": "The four-property argument applies regardless of deployment type. A model-level refusal fails auditable/deterministic/tunable/revisable in every setting. The gate succeeds on all four. They are not equivalent — and 'we have refusal-trained models' does not substitute for policy gates in any deployment where the policy is load-bearing (which is every serious deployment)." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why does the FT23 risk matrix claim that uncensored + weak harness is STRICTLY MORE dangerous than refusal-trained + weak harness, rather than merely equally dangerous?", "options": [ "Because uncensored models are inherently more capable and thus more harmful.", "Because removing the model-level refusal removes a (free, imperfect) safety layer and replaces it with nothing. The refusal-trained + weak-harness case has TWO imperfect layers (model refuses some, gate catches some); the uncensored + weak-harness case has ONE (the gate only). When the weak harness inevitably misses one, there is no second chance. The model formulates every dangerous action fluently.", "Because abliteration introduces new vulnerabilities the base did not have.", "Because uncensored models attract more adversaries." ], "answer_index": 1, "rationale": "The argument is mechanical: layers. Refusal-trained + weak harness = two imperfect layers. Uncensored + weak harness = one imperfect layer (the model layer was removed by construction). One imperfect layer is strictly worse than two. This is why removing the free model-level layer obligates you to replace it with a BETTER layer (the eval'd harness) — or you are worse off." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "A clinical-analytics agent needs reliable cohort math. The team proposes abliteration. Using FT17/FT18/FT23, analyze whether this is defensible and what would make it so.", "options": [ "Defensible — abliteration does not affect reasoning, only refusal.", "Likely indefensible without strong evidence. FT17 shows abliteration can cost up to -18.8pp on GSM8K because the refusal direction is entangled with reasoning. For a reasoning-heavy clinical agent, FT18 (DPO-toward-compliance) is the higher-fidelity, lower-cost path. Abliteration becomes defensible ONLY with a pre/post eval plan showing the regression on cohort-analytics tasks is within tolerance (e.g., <2pp).", "Defensible — just re-run GRPO after abliteration to recover the math.", "Indefensible — clinical agents must never use uncensored models under any circumstances." ], "answer_index": 1, "rationale": "FT17's capability-degradation data makes abliteration risky for reasoning-heavy agents. FT18 (DPO-toward-compliance) is the higher-fidelity alternative. Abliteration is not categorically forbidden, but it is indefensible WITHOUT a measured eval showing the regression is acceptable for the deployment's actual reasoning requirements. Option A denies the documented cost; option C is unvalidated recovery; option D is too absolute — uncensored is defensible inside an eval'd harness." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "How does FT23's synthesis ('the model steers; the harness bounds') relate to Course 1's thesis ('the model is 1.6%; the harness is 98.4%')?", "options": [ "They contradict — FT23 says the model matters more than Course 1 admits.", "They are complements describing the same system from opposite ends. Course 1 says the harness is most of the system; Course 3 (FT00) zooms into the 1.6% and says fine-tuning steers behavior. FT23 joins them: when you steer the model to stop refusing, you changed what it DOES, not what it MAY do — and the gap between does and may is exactly where the harness lives. Take both courses and you hold the whole picture.", "FT23 supersedes Course 1 — the model is now the primary safety control.", "They are unrelated theses from different domains." ], "answer_index": 1, "rationale": "Complement, not contradiction. Course 1: harness is 98.4%. Course 3: the 1.6% is steered behavior, not taught knowledge. FT23: steering changes does (Layer 3), the harness changes may (Layer 5). The synthesis is the junction of the two theses — same system, opposite ends. Together they describe the complete steered-model-in-bounding-harness architecture." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "A team says: 'Pillar 5 (Alignment Control) lets us lower our guard on the harness side, because the model is now aligned to our intent.' Why is this the dangerous error, per FT23?", "options": [ "Because Pillar 5 models are always less secure than refusal-trained models.", "Because abliteration REMOVES the free-but-bad safety layer (model refusal) that came with the weights. You have not lowered the safety requirement — you have removed a layer and must now REPLACE it with something better (an eval'd harness whose gates are auditable/deterministic/doctrine-tunable/retrain-free). Pillar 5 RAISES the harness bar; it does not lower it. The team's reasoning inverts the actual obligation.", "Because the model's intent-alignment cannot be measured.", "Because Course 1 forbids using uncensored models." ], "answer_index": 1, "rationale": "The dangerous error. A refusal-trained model provides a free (imperfect, un-auditable) safety layer. Abliteration removes it. The obligation is therefore to replace it with a BETTER layer (the harness), not to relax because 'the model is aligned.' Pillar 5 raises the harness requirement; it does not lower it. The team has it exactly backwards." } ] }