Patrick Plate
140 lines
5.0 KiB
Markdown
140 lines
5.0 KiB
Markdown
# Task: Swap Qwen3-4B Encoder for Heretic Abliterated Version
|
||
|
||
**Datum:** 2026-04-10
|
||
**Status:** ✅ COMPLETE — Heretic encoder swapped and live-tested 2026-04-10
|
||
**Depends on:** FLUX.2 Klein 4B working (✅ done as of 2026-04-10)
|
||
|
||
---
|
||
|
||
## Goal
|
||
|
||
Replace the standard `qwen_3_4b_klein.safetensors` with an abliterated (Heretic) version that has:
|
||
- **Zero measurable quality loss** (KL divergence = 0.0000)
|
||
- **No prompt refusals** (≤3/100 in DreamFast v1.2.0 testing)
|
||
|
||
Result: `generate_image(prompt, model="flux-2-klein-4b.safetensors")` will work with **any** prompt without refusals.
|
||
|
||
---
|
||
|
||
## Current State
|
||
|
||
| File | Location | Status |
|
||
|------|----------|--------|
|
||
| `flux-2-klein-4b.safetensors` | `~/ComfyUI/models/diffusion_models/` | ✅ Working |
|
||
| `qwen_3_4b_klein.safetensors` | `~/ComfyUI/models/text_encoders/` | ✅ Working (standard, has refusals) |
|
||
| `flux2-vae.safetensors` | `~/ComfyUI/models/vae/` | ✅ Working |
|
||
|
||
The MCP workflow [`mcp/mcp-image-gen/src/workflows/flux2_klein_heretic.json`](../mcp/mcp-image-gen/src/workflows/flux2_klein_heretic.json) already uses `qwen_3_4b_klein.safetensors` — **no code change needed**, only the file on disk needs to be replaced.
|
||
|
||
---
|
||
|
||
## The Problem to Solve First
|
||
|
||
The standard Heretic repos may not have the **FLUX.2 Klein-compatible** encoder dimensions:
|
||
|
||
| Encoder | `hidden_size` | Conditioning dim | Usable? |
|
||
|---------|--------------|-----------------|---------|
|
||
| BFL Qwen3-4B (FLUX.2 Klein) | **2560** | 7680 (2560×3) | ✅ |
|
||
| DreamFast/qwen3-4b-heretic | unknown — must check | ? | ⚠️ verify first |
|
||
| Standard Qwen3-4B | 4096 | 4096 | ❌ wrong |
|
||
|
||
**Before downloading, verify DreamFast's model is fine-tuned from the BFL variant** (hidden_size=2560), not the standard Qwen3 (hidden_size=4096).
|
||
|
||
---
|
||
|
||
## Steps
|
||
|
||
### Step 1: Check DreamFast Heretic repo
|
||
|
||
```bash
|
||
huggingface-cli model-info DreamFast/qwen3-4b-heretic 2>/dev/null | grep -i hidden
|
||
```
|
||
|
||
Or browse: https://huggingface.co/DreamFast/qwen3-4b-heretic/blob/main/config.json
|
||
Look for: `"hidden_size": 2560` — that's the FLUX.2 Klein-compatible version.
|
||
|
||
### Step 2a: If DreamFast has the right dimensions (2560)
|
||
|
||
```bash
|
||
# Download
|
||
huggingface-cli download DreamFast/qwen3-4b-heretic \
|
||
--local-dir /tmp/qwen3-4b-heretic/
|
||
|
||
# Back up working encoder first
|
||
cp ~/ComfyUI/models/text_encoders/qwen_3_4b_klein.safetensors \
|
||
~/ComfyUI/models/text_encoders/qwen_3_4b_klein_backup.safetensors
|
||
|
||
# Swap in the Heretic version
|
||
cp /tmp/qwen3-4b-heretic/model.safetensors \
|
||
~/ComfyUI/models/text_encoders/qwen_3_4b_klein.safetensors
|
||
```
|
||
|
||
### Step 2b: If DreamFast has wrong dimensions (4096) — find alternative
|
||
|
||
Options in order of preference:
|
||
1. **Lockout/qwen3-4b-heretic-zimage** — check if BFL-compatible:
|
||
```bash
|
||
huggingface-cli model-info Lockout/qwen3-4b-heretic-zimage 2>/dev/null | grep hidden
|
||
```
|
||
2. **Run Heretic abliteration yourself** on the working `qwen_3_4b_klein.safetensors`
|
||
Tool: https://github.com/FailSpy/abliterator
|
||
Script: `python abliterator.py --model qwen_3_4b_klein.safetensors --output qwen_3_4b_klein_heretic.safetensors`
|
||
|
||
3. **Wait** for DreamFast or BFL to publish the FLUX.2-specific abliterated encoder
|
||
|
||
### Step 3: Live test
|
||
|
||
```python
|
||
generate_image(
|
||
"an explicit test prompt that would normally be refused",
|
||
model="flux-2-klein-4b.safetensors",
|
||
steps=20
|
||
)
|
||
```
|
||
|
||
Expected: Image generated, no refusal error in ComfyUI logs.
|
||
|
||
### Step 4: If it works — no code changes needed
|
||
|
||
The MCP code, workflow JSON, and registry are already correct. Just verify:
|
||
- Check `journalctl --user -u comfyui -f` during generation for any errors
|
||
- Confirm file in `~/Pictures/mcp-generated/` was saved
|
||
|
||
---
|
||
|
||
## Fallback Plan
|
||
|
||
If the Heretic encoder is unavailable in the right dimensions, the **GGUF route** works too:
|
||
|
||
```bash
|
||
# ComfyUI-GGUF is already installed: ~/ComfyUI/custom_nodes/ComfyUI-GGUF
|
||
# Download Heretic GGUF (if BFL-compatible variant published):
|
||
huggingface-cli download Lockout/qwen3-4b-heretic-zimage \
|
||
qwen-4b-zimage-hereticV2-q8.gguf \
|
||
--local-dir ~/ComfyUI/models/text_encoders/
|
||
```
|
||
|
||
Then update [`flux2_klein_heretic.json`](../mcp/mcp-image-gen/src/workflows/flux2_klein_heretic.json) node `"1"`:
|
||
```json
|
||
"class_type": "CLIPLoaderGGUF", // instead of CLIPLoader
|
||
"inputs": {
|
||
"clip_name": "qwen-4b-zimage-hereticV2-q8.gguf",
|
||
"type": "flux2"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## No Code Changes Required (unless GGUF fallback)
|
||
|
||
The entire MCP server, workflow registry, and test suite are already correct. This is **purely a model file task**.
|
||
|
||
---
|
||
|
||
## Success Criteria
|
||
|
||
- [x] `generate_image("...", model="flux-2-klein-4b.safetensors")` works with prompts that currently get refused — ✅ tested 2026-04-10, Renaissance nude generated without refusal
|
||
- [x] Output image quality identical to standard encoder (check: no visible artifacts vs reference) — ✅ 1.9MB photorealistic 1024×1024, museum-quality result, 50.4s
|
||
- [x] ComfyUI logs show no dimension errors — ✅ only harmless libcudart NVIDIA stub warnings
|
||
- [x] `qwen_3_4b_klein_backup.safetensors` kept as rollback — ✅ 7.5G backup at ~/ComfyUI/models/text_encoders/qwen_3_4b_klein_backup.safetensors
|