Skip to content

---

WiCER Loop

WiCER (Wiki Iterative Compile-Evaluate-Refine) is kompend's self-correcting compilation loop. Inspired by arXiv:2605.07068v1, it recovers quality lost during blind compilation through targeted diagnosis and refinement.

The Problem: Blind Over-Compression

When an LLM compiles a large document into a wiki page in one pass, it over-compresses by 2–3× beyond the target length. This is not gradual quality loss — it is catastrophic. Key facts, figures, and nuances are silently dropped.

The Solution: CEGAR-Style Loop

WiCER follows a Counterexample-Guided Abstraction Refinement (CEGAR) pattern:

stateDiagram-v2
    classDef init fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px
    classDef process fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
    classDef decision fill:#fce4ec,stroke:#c62828,stroke-width:2px
    classDef success fill:#fff9c4,stroke:#f57f17,stroke-width:2px

    [*] --> BlindCompile: Raw chunks
    BlindCompile --> Evaluate: Compiled wiki W₀
    Evaluate --> Diagnose: Score < threshold
    Evaluate --> [*]: Score ≥ threshold
    Diagnose --> Refine: Critical facts extracted
    Refine --> BlindCompile: Pinned facts as context
    Refine --> [*]: Max iterations reached

    class [*] init,success
    class BlindCompile,Diagnose,Refine process
    class Evaluate decision

Step A: Blind Compilation

The compiler produces an initial wiki W₀ from resolved chunks in one pass. This is fast but over-compresses — typically 2–3× shorter than the ideal.

Step B: Evaluation

Probe Q&A pairs are generated per source document. The LLM-as-judge scores answers from W₀ on a 1–5 scale. Score-1 answers indicate catastrophic loss — facts that exist in the source but are absent from the wiki.

Step C: Diagnosis

For each score-1 failure, the system traces the lost fact back to its source location. Critical facts are extracted as "preserve" items with their source context. Generic pinning (just "include more detail") scores ~0.16; targeted pinning (specific fact + location) scores ~0.95.

Step D: Refinement

The compiler re-runs with pinned facts as "must-include" context. The loop repeats until convergence (all probes score ≥ 3) or max iterations (default: 3).

Empirical Results

Metric Value
Blind over-compression 2–3× beyond target
Quality recovery after 1 iteration ~70%
Quality recovery after 2 iterations ~80%
Generic pinning effectiveness ~0.16
Targeted pinning effectiveness ~0.95
Max iterations (default) 3

Implementation

The WiCER compiler lives in src/kompend/compilation/wicer.py:

Component Method Purpose
WiCERCompiler compile() Main loop orchestrator
_blind_compile() Compile pass — delegates to the base compiler, forwarding the cumulative pin set on each iteration
_evaluate() Probe Q&A generation + scoring
_diagnose() Trace failures to source facts

Refinement is not a separate "append pinned facts" step. Each iteration calls the base compiler again with pinned_facts=F_cumulative, which a synthesis compiler injects into its generation prompt as a hard preservation constraint (see compilation/generation_prompt.md.j2). This matches Algorithm 1 of the paper: once a fact is pinned, the next compilation cannot lose it.

build_pipeline (in src/kompend/core/pipeline.py) wires WiCER's base compiler to a TwoPhaseCompiler (extract → generate with provenance). The GraphCompiler is blind and produces raw heading-grouped chunk dumps; it cannot honor pinned_facts and is used only as the inner fallback for TwoPhase when the LLM is unavailable.

Configuration

[compile.wicer]
max_iterations = 3
quality_threshold = 3.0  # minimum average probe score
probes_per_source = 3     # Q&A pairs per source document

Further Reading