docs(plans): add heretic encoder swap task for FLUX.2 Klein uncensored generation

2026-04-10 20:32:05 +02:00
parent 1d8849cb41
commit 1d1e70776f
1 changed files with 139 additions and 0 deletions
@@ -0,0 +1,139 @@
+# Task: Swap Qwen3-4B Encoder for Heretic Abliterated Version
+
+**Datum:** 2026-04-10  
+**Status:** Ready — waiting for correct Heretic encoder to be published  
+**Depends on:** FLUX.2 Klein 4B working (✅ done as of 2026-04-10)
+
+---
+
+## Goal
+
+Replace the standard `qwen_3_4b_klein.safetensors` with an abliterated (Heretic) version that has:
+- **Zero measurable quality loss** (KL divergence = 0.0000)
+- **No prompt refusals** (≤3/100 in DreamFast v1.2.0 testing)
+
+Result: `generate_image(prompt, model="flux-2-klein-4b.safetensors")` will work with **any** prompt without refusals.
+
+---
+
+## Current State
+
+| File | Location | Status |
+|------|----------|--------|
+| `flux-2-klein-4b.safetensors` | `~/ComfyUI/models/diffusion_models/` | ✅ Working |
+| `qwen_3_4b_klein.safetensors` | `~/ComfyUI/models/text_encoders/` | ✅ Working (standard, has refusals) |
+| `flux2-vae.safetensors` | `~/ComfyUI/models/vae/` | ✅ Working |
+
+The MCP workflow [`mcp/mcp-image-gen/src/workflows/flux2_klein_heretic.json`](../mcp/mcp-image-gen/src/workflows/flux2_klein_heretic.json) already uses `qwen_3_4b_klein.safetensors` — **no code change needed**, only the file on disk needs to be replaced.
+
+---
+
+## The Problem to Solve First
+
+The standard Heretic repos may not have the **FLUX.2 Klein-compatible** encoder dimensions:
+
+| Encoder | `hidden_size` | Conditioning dim | Usable? |
+|---------|--------------|-----------------|---------|
+| BFL Qwen3-4B (FLUX.2 Klein) | **2560** | 7680 (2560×3) | ✅ |
+| DreamFast/qwen3-4b-heretic | unknown — must check | ? | ⚠️ verify first |
+| Standard Qwen3-4B | 4096 | 4096 | ❌ wrong |
+
+**Before downloading, verify DreamFast's model is fine-tuned from the BFL variant** (hidden_size=2560), not the standard Qwen3 (hidden_size=4096).
+
+---
+
+## Steps
+
+### Step 1: Check DreamFast Heretic repo
+
+```bash
+huggingface-cli model-info DreamFast/qwen3-4b-heretic 2>/dev/null | grep -i hidden
+```
+
+Or browse: https://huggingface.co/DreamFast/qwen3-4b-heretic/blob/main/config.json  
+Look for: `"hidden_size": 2560` — that's the FLUX.2 Klein-compatible version.
+
+### Step 2a: If DreamFast has the right dimensions (2560)
+
+```bash
+# Download
+huggingface-cli download DreamFast/qwen3-4b-heretic \
+  --local-dir /tmp/qwen3-4b-heretic/
+
+# Back up working encoder first
+cp ~/ComfyUI/models/text_encoders/qwen_3_4b_klein.safetensors \
+   ~/ComfyUI/models/text_encoders/qwen_3_4b_klein_backup.safetensors
+
+# Swap in the Heretic version
+cp /tmp/qwen3-4b-heretic/model.safetensors \
+   ~/ComfyUI/models/text_encoders/qwen_3_4b_klein.safetensors
+```
+
+### Step 2b: If DreamFast has wrong dimensions (4096) — find alternative
+
+Options in order of preference:
+1. **Lockout/qwen3-4b-heretic-zimage** — check if BFL-compatible:
+   ```bash
+   huggingface-cli model-info Lockout/qwen3-4b-heretic-zimage 2>/dev/null | grep hidden
+   ```
+2. **Run Heretic abliteration yourself** on the working `qwen_3_4b_klein.safetensors`  
+   Tool: https://github.com/FailSpy/abliterator  
+   Script: `python abliterator.py --model qwen_3_4b_klein.safetensors --output qwen_3_4b_klein_heretic.safetensors`
+
+3. **Wait** for DreamFast or BFL to publish the FLUX.2-specific abliterated encoder
+
+### Step 3: Live test
+
+```python
+generate_image(
+    "an explicit test prompt that would normally be refused",
+    model="flux-2-klein-4b.safetensors",
+    steps=20
+)
+```
+
+Expected: Image generated, no refusal error in ComfyUI logs.
+
+### Step 4: If it works — no code changes needed
+
+The MCP code, workflow JSON, and registry are already correct. Just verify:
+- Check `journalctl --user -u comfyui -f` during generation for any errors
+- Confirm file in `~/Pictures/mcp-generated/` was saved
+
+---
+
+## Fallback Plan
+
+If the Heretic encoder is unavailable in the right dimensions, the **GGUF route** works too:
+
+```bash
+# ComfyUI-GGUF is already installed: ~/ComfyUI/custom_nodes/ComfyUI-GGUF
+# Download Heretic GGUF (if BFL-compatible variant published):
+huggingface-cli download Lockout/qwen3-4b-heretic-zimage \
+  qwen-4b-zimage-hereticV2-q8.gguf \
+  --local-dir ~/ComfyUI/models/text_encoders/
+```
+
+Then update [`flux2_klein_heretic.json`](../mcp/mcp-image-gen/src/workflows/flux2_klein_heretic.json) node `"1"`:
+```json
+"class_type": "CLIPLoaderGGUF",  // instead of CLIPLoader
+"inputs": {
+  "clip_name": "qwen-4b-zimage-hereticV2-q8.gguf",
+  "type": "flux2"
+}
+```
+
+---
+
+## No Code Changes Required (unless GGUF fallback)
+
+The entire MCP server, workflow registry, and test suite are already correct. This is **purely a model file task**.
+
+---
+
+## Success Criteria
+
+- [ ] `generate_image("...", model="flux-2-klein-4b.safetensors")` works with prompts that currently get refused
+- [ ] Output image quality identical to standard encoder (check: no visible artifacts vs reference)
+- [ ] ComfyUI logs show no dimension errors
+- [ ] `qwen_3_4b_klein_backup.safetensors` kept as rollback