Lab Specification — Module FT23: The Architect's Verdict

Course: Course 3 — LLM Fine-Tuning Masterclass Module: FT23 — Synthesis: Uncensor the Model, Harness the Model Duration: 60–90 minutes (judgment lab — no GPU required) Environment: A text editor and your judgment. This is a no-GPU architecture lab. You produce a 2-page architecture document and a defense. No training, no inference, no code execution.

This lab is the Capstone 2 preview. The architecture you produce here is the skeleton of the Capstone 2 deliverable (the calibrated uncensored agent). Do it well and the capstone is half-done.

Learning objectives

By the end of this lab you will have:

Architected a calibrated uncensored agent across all five layers of the Steering Stack — base, adapter, steer, export, and harness — for a specific deployment scenario.
Defended a steering choice (abliterate or DPO-toward-compliance) with reference to the capability-degradation trade-off (FT17: the −18.8pp GSM8K figure) and the deployment's actual requirements.
Specified the harness policy gates that replace the model-level refusal layer you removed — auditable, deterministic, doctrine-tunable, revisable without retraining.
Re-written the threat model for the absence of model-level refusal, per the absolute rule of FT23.
Stated the absolute rule and shown in your architecture how every quadrant of the risk matrix is handled except the forbidden one (uncensored + weak harness).

The point of this lab is judgment, not execution. A senior engineer who can steer a model but cannot defend the architecture is not senior. This lab forces the architectural decisions into the open.

The scenario

You are the architect. A client — pick one of the three below (your instructor may assign) — wants a calibrated uncensored agent. "Calibrated" means: the model executes reliably (no mid-loop refusals), the harness bounds what it may do (doctrine-specific policy gates), and the whole thing is auditable to a regulator.

Scenario A — The hospital clinical-analytics agent

Environment: A HIPAA-covered entity. BAA-free (no vendor data-processing agreement — see FT21). Air-gapped clinical network. Task: An agent that queries the clinical data warehouse (execute_sql), summarizes patient-cohort anomalies (run_python on de-identified extracts), and drafts clinician-facing reports (write_file to a reviewed queue). Must never exfiltrate PHI, never write outside the reviewed queue, never contact an external endpoint. Doctrine: HIPAA + institutional policy. Refuse-by-default for anything that could move PHI outside the enclave.

Scenario B — The authorized red-team agent

Environment: An offensive-security firm with a scope-of-authorization letter for a client engagement. Air-gapped to the firm's lab network; egress to authorized targets only. Task: An agent that runs run_shell (nmap, burp, custom exploit scaffolding) against in-scope targets, queries findings (execute_sql on the engagement DB), and drafts the engagement report. Must never touch out-of-scope IPs, never exfiltrate client data to the firm's non-engagement storage, never act without the scope letter loaded. Doctrine: The scope-of-authorization letter. Permitted within scope; refused outside it.

Scenario C — The classified-environment analyst agent

Environment: An air-gapped classified network (IL5/JWICS-adjacent — see FT22). No egress, ever. Cross-domain transfer forbidden by policy. Task: An agent that queries classified intelligence holdings (execute_sql), drafts analytic products (write_file to a review queue), and produces summary briefs. Must never attempt a cross-domain transfer, never cite a source above the product's classification level, never act outside the analyst's clearance-enforced permissions. Doctrine: Cross-domain rules + classification guides. Refused at the domain boundary, absolutely.

All three scenarios share the same architecture shape. They differ in doctrine (the policy file) and threat model. That is the point — same model, same steering approach, three different harnesses.

The architecture template

Produce a 2-page architecture document. Use the template below. Every section is required. A missing section is a failing grade — the lab exists to force you to make every decision explicit.

Page 1 — The stack (Layers 1–4: the model)

L1 — Base selection

Chosen base (model ID + version). Open-data or open-weights-only? Why is that defensible for this environment?
Parameter size and the VRAM/hardware it implies (cite FT01).
Why this base over the alternatives (name at least two you rejected and why).

L2 — Adapter

LoRA or DoRA? Rank, alpha, target modules. (Cite FT08/FT09.)
Why this configuration — what behavior are you steering, and what is the minimal adapter that achieves it?

L3 — Steer (THE KEY DECISION)

Abliterate OR DPO-toward-compliance. Pick one. Defend the choice with explicit reference to:
- The capability-degradation trade-off (FT17: −18.8pp GSM8K worst case). What is your eval plan to measure the regression on your task?
- The fidelity argument (FT18: DPO-toward-compliance steers toward the policy you want, rather than away from refusal — higher fidelity, lower capability cost, but requires a preference dataset).
- Your deployment's actual reasoning requirements. Does the agent need the math/reasoning that abliteration may degrade? If yes, that weighs toward DPO.
The SFT layer beneath it (FT12) — what format/instruction-following steering do you need before the alignment steer?

L4 — Export

Quantization format (GGUF Q4_K_M for Ollama local? AWQ for vLLM? MLX for Apple Silicon?). Cite FT19.
Serving stack (vLLM vs llama.cpp vs Ollama). Cite FT20. Air-gap implications (FT22).
Why this format/stack for this environment (latency, throughput, hardware, air-gap).

Page 2 — The boundary (Layer 5: the harness)

L5 — The harness policy gates (THE SYNTHESIS)

The doctrine. Write it as a short policy file (5–10 rules). Be specific to your scenario — "refuse anything that could exfiltrate PHI" is a rule; "be safe" is not.
The four properties, addressed explicitly:
- Auditable: what gets logged per gate decision? (Cite Course 1 module-10.)
- Deterministic: how is the policy applied deterministically (no sampling)?
- Tunable to doctrine: show that the same model + a different policy file = a different deployment.
- Revisable without retraining: confirm the policy is external to the weights.

The threat model (re-written for the absence of model-level refusal)

List 3–5 attack paths that previously relied on "the model will refuse." For each: how does the harness gate now catch it? Or, if it doesn't, why is the residual risk accepted?
This section is the one students most often skip. Do not skip it. It is half the point of the lab.

The risk matrix check

Confirm your architecture lands in the "uncensored + eval'd harness" quadrant (Diagram 4). Show the two hardening steps: gates hardened AND threat model hardened.
State the absolute rule verbatim and confirm your architecture satisfies it.

Decision points (the places your defense is graded)

There are no single right answers in this lab. There are defensible and indefensible answers. Your grade is the defensibility of your reasoning at these decision points:

The steering choice (abliterate vs DPO). Indefensible: "abliterate because it's faster" without measuring the capability cost on your task. Defensible: either choice, with an eval plan and a stated reason grounded in the deployment's reasoning requirements.
The base's openness. Indefensible: a closed-weights API model in an air-gapped classified environment. Defensible: an open-data base (auditable corpus) for regulated environments; open-weights-only acceptable for a red-team lab if you state why the corpus audit is not required there.
The policy file's specificity. Indefensible: "the harness will be safe." Defensible: a written policy with testable rules, each mapping to an action class the gate intercepts.
The threat model's honesty. Indefensible: copying the pre-uncensoring threat model unchanged. Defensible: re-examining every "the model will refuse" path and showing the harness mitigation (or the accepted residual risk).
The quantization's fit. Indefensible: a quantization that breaks the air-gap (e.g., requires an API call to a vendor). Defensible: a format that runs fully offline on the chosen hardware.

Deliverables

Submit ft23-architecture.md:

Page 1 — the stack (L1 base, L2 adapter, L3 steer with the abliterate-vs-DPO defense, L4 export). ~1 page.
Page 2 — the boundary (L5 policy gates with the four properties, the re-written threat model, the risk-matrix check). ~1 page.
The defense (can be inline or a short appendix) — 300–500 words defending your steering choice and your threat-model rewrite with explicit citations to FT17 (capability cost), FT18 (DPO fidelity), FT23 (the absolute rule), and Course 1 (the harness properties).

Hard constraints:

2 pages for the architecture proper. The defense may be a third short page. Going over is a sign you have not made decisions — force yourself to choose.
Every layer must name the module that informs the choice (FT01, FT09, FT17, FT19, etc.). This proves the synthesis landed.

Solution key

There is no single correct architecture. There are defensible ones. Below are the shapes a correct solution takes — your job is to pick one and defend it.

Defensible: DPO-toward-compliance over abliteration (the higher-fidelity path)

L1: an open-data base (MiniCPM, OLMo, or Tulu) at 3–8B for the hospital (auditable corpus is a HIPAA defense); an open-weights base (Qwen3, Llama 3.x) acceptable for the red-team lab.
L2: DoRA, r=16–32, on attention + MLP projections.
L3: SFT for the agent-loop format (tool-call schema, plan/act structure), then DPO toward compliance (FT18) on a preference dataset of (compliant-continuation, refused-continuation) pairs for in-scope tasks. Defense: the hospital agent needs reliable reasoning for cohort analytics — abliteration's −18.8pp GSM8K worst case is unacceptable. DPO gives compliance at lower capability cost. Eval plan: re-run the cohort-analytics eval suite pre- and post-DPO; require <2pp regression.
L4: AWQ 4-bit on vLLM (hospital — production throughput), GGUF Q4_K_M on llama.cpp (red-team lab — portability), GGUF on Ollama (classified — air-gapped simplicity).
L5: a written policy file specific to the doctrine (HIPAA safe-harbor list for PHI fields; the scope-letter's CIDR ranges for the red-team lab; cross-domain rules for classified). All four properties addressed. Threat model re-written.

Defensible: abliteration (the lower-fidelity, faster path) — if the eval holds

L3: abliteration (FT17) instead of DPO. Defense: the red-team lab agent's task is largely shell-scaffolding and findings-query — it does not need GSM8K-grade math, so abliteration's reasoning cost is acceptable. Eval plan required: measure the regression on engagement-relevant tasks (exploit-logic eval, findings-summarization eval); require <5pp regression or fall back to DPO. If the student cannot produce an eval plan, this choice is indefensible.
Everything else as above.

The threat model rewrite (required in every defensible solution)

The re-written threat model names at least three paths that previously assumed model-level refusal and shows the harness mitigation:

Prompt-injection attempting to exfiltrate via send_message → previously "the model will refuse to send PHI"; now the policy gate refuses any send_message to a non-allowlisted recipient, deterministically, regardless of what the model emitted.
A tool call targeting an out-of-scope IP → previously "the model will refuse"; now the scope-letter CIDR check in the gate catches it.
A cross-domain write → previously "the model will refuse"; now the gate refuses any write_file outside the reviewed queue / classified review path.

The risk-matrix check (required)

The solution explicitly lands in the "uncensored + eval'd harness" quadrant and shows both hardening steps (gates + threat model). It states the absolute rule verbatim and confirms the architecture satisfies it.

Indefensible solutions (failing grades)

Abliteration with no eval plan for the capability cost.
A closed-weights API model in an air-gapped environment.
A policy file that says "be safe" instead of listing rules.
A threat model copied unchanged from the pre-uncensoring version.
Any architecture that lands in the "uncensored + weak harness" quadrant (the forbidden one).
An architecture that adds Layer 3 uncensoring but treats Layer 5 as "we'll add it later."

Stretch goals

Write the policy file as actual code. Express 3–5 of your doctrine rules as a deterministic policy function (Python or pseudocode) that takes a proposed action and returns permit/deny. Show that the same function with a different config is a different deployment. (Sets up Course 1 module-06/07.)
Spec the eval suite. List the 5–10 eval tasks you would run pre- and post-steering to measure capability regression. Name a benchmark where applicable (GSM8K for math, a custom cohort-analytics eval for the hospital, an exploit-logic eval for the red-team lab). (Sets up Course 1 module-09 verification.)
Spec the audit log schema. Define the fields a gate-decision log entry would carry (timestamp, proposed action, policy rule matched, decision, operator, deployment). Show that any past decision is replayable. (Sets up Course 1 module-10 observability.)
Identify the residual risk. Name one attack path your harness does not fully mitigate, and state whether you accept the residual risk or require Course 2B (red-team) before deploy. (Sets up the curriculum funnel.)

# Lab Specification — Module FT23: The Architect's Verdict **Course**: Course 3 — LLM Fine-Tuning Masterclass **Module**: FT23 — Synthesis: Uncensor the Model, Harness the Model **Duration**: 60–90 minutes (judgment lab — no GPU required) **Environment**: A text editor and your judgment. This is a **no-GPU architecture lab**. You produce a 2-page architecture document and a defense. No training, no inference, no code execution. > This lab is the **Capstone 2 preview**. The architecture you produce here is the skeleton of the Capstone 2 deliverable (the calibrated uncensored agent). Do it well and the capstone is half-done. --- ## Learning objectives By the end of this lab you will have: 1. **Architected a calibrated uncensored agent** across all five layers of the Steering Stack — base, adapter, steer, export, and harness — for a specific deployment scenario. 2. **Defended a steering choice** (abliterate *or* DPO-toward-compliance) with reference to the capability-degradation trade-off (FT17: the −18.8pp GSM8K figure) and the deployment's actual requirements. 3. **Specified the harness policy gates** that replace the model-level refusal layer you removed — auditable, deterministic, doctrine-tunable, revisable without retraining. 4. **Re-written the threat model** for the absence of model-level refusal, per the absolute rule of FT23. 5. **Stated the absolute rule** and shown in your architecture how every quadrant of the risk matrix is handled except the forbidden one (uncensored + weak harness). The point of this lab is **judgment, not execution.** A senior engineer who can steer a model but cannot defend the architecture is not senior. This lab forces the architectural decisions into the open. --- ## The scenario You are the architect. A client — pick one of the three below (your instructor may assign) — wants a calibrated uncensored agent. "Calibrated" means: the model executes reliably (no mid-loop refusals), the harness bounds what it may do (doctrine-specific policy gates), and the whole thing is auditable to a regulator. ### Scenario A — The hospital clinical-analytics agent **Environment**: A HIPAA-covered entity. BAA-free (no vendor data-processing agreement — see FT21). Air-gapped clinical network. **Task**: An agent that queries the clinical data warehouse (`execute_sql`), summarizes patient-cohort anomalies (`run_python` on de-identified extracts), and drafts clinician-facing reports (`write_file` to a reviewed queue). Must never exfiltrate PHI, never write outside the reviewed queue, never contact an external endpoint. **Doctrine**: HIPAA + institutional policy. Refuse-by-default for anything that could move PHI outside the enclave. ### Scenario B — The authorized red-team agent **Environment**: An offensive-security firm with a scope-of-authorization letter for a client engagement. Air-gapped to the firm's lab network; egress to authorized targets only. **Task**: An agent that runs `run_shell` (nmap, burp, custom exploit scaffolding) against in-scope targets, queries findings (`execute_sql` on the engagement DB), and drafts the engagement report. Must never touch out-of-scope IPs, never exfiltrate client data to the firm's non-engagement storage, never act without the scope letter loaded. **Doctrine**: The scope-of-authorization letter. Permitted within scope; refused outside it. ### Scenario C — The classified-environment analyst agent **Environment**: An air-gapped classified network (IL5/JWICS-adjacent — see FT22). No egress, ever. Cross-domain transfer forbidden by policy. **Task**: An agent that queries classified intelligence holdings (`execute_sql`), drafts analytic products (`write_file` to a review queue), and produces summary briefs. Must never attempt a cross-domain transfer, never cite a source above the product's classification level, never act outside the analyst's clearance-enforced permissions. **Doctrine**: Cross-domain rules + classification guides. Refused at the domain boundary, absolutely. All three scenarios share the same architecture shape. They differ in doctrine (the policy file) and threat model. That is the point — same model, same steering approach, three different harnesses. --- ## The architecture template Produce a 2-page architecture document. Use the template below. **Every section is required.** A missing section is a failing grade — the lab exists to force you to make every decision explicit. ### Page 1 — The stack (Layers 1–4: the model) **L1 — Base selection** - Chosen base (model ID + version). Open-data or open-weights-only? Why is that defensible for *this* environment? - Parameter size and the VRAM/hardware it implies (cite FT01). - Why this base over the alternatives (name at least two you rejected and why). **L2 — Adapter** - LoRA or DoRA? Rank, alpha, target modules. (Cite FT08/FT09.) - Why this configuration — what behavior are you steering, and what is the minimal adapter that achieves it? **L3 — Steer (THE KEY DECISION)** - **Abliterate OR DPO-toward-compliance.** Pick one. Defend the choice with explicit reference to: - The capability-degradation trade-off (FT17: −18.8pp GSM8K worst case). What is your eval plan to measure the regression on *your* task? - The fidelity argument (FT18: DPO-toward-compliance steers toward the policy you want, rather than away from refusal — higher fidelity, lower capability cost, but requires a preference dataset). - Your deployment's actual reasoning requirements. Does the agent need the math/reasoning that abliteration may degrade? If yes, that weighs toward DPO. - The SFT layer beneath it (FT12) — what format/instruction-following steering do you need before the alignment steer? **L4 — Export** - Quantization format (GGUF Q4_K_M for Ollama local? AWQ for vLLM? MLX for Apple Silicon?). Cite FT19. - Serving stack (vLLM vs llama.cpp vs Ollama). Cite FT20. Air-gap implications (FT22). - Why this format/stack for *this* environment (latency, throughput, hardware, air-gap). ### Page 2 — The boundary (Layer 5: the harness) **L5 — The harness policy gates (THE SYNTHESIS)** - The doctrine. Write it as a short policy file (5–10 rules). Be specific to your scenario — "refuse anything that could exfiltrate PHI" is a rule; "be safe" is not. - The four properties, addressed explicitly: - **Auditable:** what gets logged per gate decision? (Cite Course 1 module-10.) - **Deterministic:** how is the policy applied deterministically (no sampling)? - **Tunable to doctrine:** show that the same model + a different policy file = a different deployment. - **Revisable without retraining:** confirm the policy is external to the weights. **The threat model (re-written for the absence of model-level refusal)** - List 3–5 attack paths that *previously* relied on "the model will refuse." For each: how does the harness gate now catch it? Or, if it doesn't, why is the residual risk accepted? - This section is the one students most often skip. Do not skip it. It is half the point of the lab. **The risk matrix check** - Confirm your architecture lands in the "uncensored + eval'd harness" quadrant (Diagram 4). Show the two hardening steps: gates hardened AND threat model hardened. - State the absolute rule verbatim and confirm your architecture satisfies it. --- ## Decision points (the places your defense is graded) There are no single right answers in this lab. There are **defensible** and **indefensible** answers. Your grade is the defensibility of your reasoning at these decision points: 1. **The steering choice (abliterate vs DPO).** Indefensible: "abliterate because it's faster" without measuring the capability cost on your task. Defensible: either choice, with an eval plan and a stated reason grounded in the deployment's reasoning requirements. 2. **The base's openness.** Indefensible: a closed-weights API model in an air-gapped classified environment. Defensible: an open-data base (auditable corpus) for regulated environments; open-weights-only acceptable for a red-team lab *if* you state why the corpus audit is not required there. 3. **The policy file's specificity.** Indefensible: "the harness will be safe." Defensible: a written policy with testable rules, each mapping to an action class the gate intercepts. 4. **The threat model's honesty.** Indefensible: copying the pre-uncensoring threat model unchanged. Defensible: re-examining every "the model will refuse" path and showing the harness mitigation (or the accepted residual risk). 5. **The quantization's fit.** Indefensible: a quantization that breaks the air-gap (e.g., requires an API call to a vendor). Defensible: a format that runs fully offline on the chosen hardware. --- ## Deliverables Submit `ft23-architecture.md`: - [ ] **Page 1** — the stack (L1 base, L2 adapter, L3 steer with the abliterate-vs-DPO defense, L4 export). ~1 page. - [ ] **Page 2** — the boundary (L5 policy gates with the four properties, the re-written threat model, the risk-matrix check). ~1 page. - [ ] **The defense** (can be inline or a short appendix) — 300–500 words defending your steering choice and your threat-model rewrite with explicit citations to FT17 (capability cost), FT18 (DPO fidelity), FT23 (the absolute rule), and Course 1 (the harness properties). Hard constraints: - 2 pages for the architecture proper. The defense may be a third short page. Going over is a sign you have not made decisions — force yourself to choose. - Every layer must name the module that informs the choice (FT01, FT09, FT17, FT19, etc.). This proves the synthesis landed. --- ## Solution key There is **no single correct architecture.** There are defensible ones. Below are the shapes a correct solution takes — your job is to pick one and defend it. ### Defensible: DPO-toward-compliance over abliteration (the higher-fidelity path) - **L1**: an open-data base (MiniCPM, OLMo, or Tulu) at 3–8B for the hospital (auditable corpus is a HIPAA defense); an open-weights base (Qwen3, Llama 3.x) acceptable for the red-team lab. - **L2**: DoRA, r=16–32, on attention + MLP projections. - **L3**: SFT for the agent-loop format (tool-call schema, plan/act structure), then **DPO toward compliance** (FT18) on a preference dataset of (compliant-continuation, refused-continuation) pairs for in-scope tasks. Defense: the hospital agent needs reliable reasoning for cohort analytics — abliteration's −18.8pp GSM8K worst case is unacceptable. DPO gives compliance at lower capability cost. Eval plan: re-run the cohort-analytics eval suite pre- and post-DPO; require <2pp regression. - **L4**: AWQ 4-bit on vLLM (hospital — production throughput), GGUF Q4_K_M on llama.cpp (red-team lab — portability), GGUF on Ollama (classified — air-gapped simplicity). - **L5**: a written policy file specific to the doctrine (HIPAA safe-harbor list for PHI fields; the scope-letter's CIDR ranges for the red-team lab; cross-domain rules for classified). All four properties addressed. Threat model re-written. ### Defensible: abliteration (the lower-fidelity, faster path) — *if* the eval holds - **L3**: abliteration (FT17) instead of DPO. Defense: the red-team lab agent's task is largely shell-scaffolding and findings-query — it does not need GSM8K-grade math, so abliteration's reasoning cost is acceptable. **Eval plan required:** measure the regression on engagement-relevant tasks (exploit-logic eval, findings-summarization eval); require <5pp regression or fall back to DPO. If the student cannot produce an eval plan, this choice is indefensible. - Everything else as above. ### The threat model rewrite (required in every defensible solution) The re-written threat model names at least three paths that previously assumed model-level refusal and shows the harness mitigation: - **Prompt-injection attempting to exfiltrate via `send_message`** → previously "the model will refuse to send PHI"; now the policy gate refuses any `send_message` to a non-allowlisted recipient, deterministically, regardless of what the model emitted. - **A tool call targeting an out-of-scope IP** → previously "the model will refuse"; now the scope-letter CIDR check in the gate catches it. - **A cross-domain write** → previously "the model will refuse"; now the gate refuses any `write_file` outside the reviewed queue / classified review path. ### The risk-matrix check (required) The solution explicitly lands in the "uncensored + eval'd harness" quadrant and shows both hardening steps (gates + threat model). It states the absolute rule verbatim and confirms the architecture satisfies it. ### Indefensible solutions (failing grades) - Abliteration with no eval plan for the capability cost. - A closed-weights API model in an air-gapped environment. - A policy file that says "be safe" instead of listing rules. - A threat model copied unchanged from the pre-uncensoring version. - Any architecture that lands in the "uncensored + weak harness" quadrant (the forbidden one). - An architecture that adds Layer 3 uncensoring but treats Layer 5 as "we'll add it later." --- ## Stretch goals 1. **Write the policy file as actual code.** Express 3–5 of your doctrine rules as a deterministic policy function (Python or pseudocode) that takes a proposed action and returns permit/deny. Show that the same function with a different config is a different deployment. (Sets up Course 1 module-06/07.) 2. **Spec the eval suite.** List the 5–10 eval tasks you would run pre- and post-steering to measure capability regression. Name a benchmark where applicable (GSM8K for math, a custom cohort-analytics eval for the hospital, an exploit-logic eval for the red-team lab). (Sets up Course 1 module-09 verification.) 3. **Spec the audit log schema.** Define the fields a gate-decision log entry would carry (timestamp, proposed action, policy rule matched, decision, operator, deployment). Show that any past decision is replayable. (Sets up Course 1 module-10 observability.) 4. **Identify the residual risk.** Name one attack path your harness does *not* fully mitigate, and state whether you accept the residual risk or require Course 2B (red-team) before deploy. (Sets up the curriculum funnel.)