Stop RAG hallucinations: gate the answer on whether retrieval is grounded
A calibrated gate that lets a RAG system answer only when the retrieved context supports it — and abstains instead of hallucinating when it doesn't.
The problem
RAG answers sound equally confident whether the retrieval was relevant or junk. The failure isn't being wrong occasionally — it's confidently answering from context that doesn't support the question. You want it to say "I don't know" exactly when it shouldn't answer.
Is the retrieval grounded enough to answer, or should the system abstain / hand off?
Was the answer right — a thumbs-down, a correction, or an eval verdict.
cell: rag_answer
The code
Zero-dependency SDK. Mint a key on your dashboard and drop this in.
from warmwinter import WarmWinter
ww = WarmWinter(api_key="ww_YOUR_KEY",
base_url="https://api.warmwinter.io")
# Gate: is the retrieval grounded enough to answer?
d = ww.decide(domain="rag", decision_type="rag_answer",
stated_confidence=retrieval_score(hits),
on_ungrounded="abstain") # don't guess off the frontier
answer = generate(q, hits) if d.verdict == "act" else "I don't know that yet."
ww.outcome(d.decision_id, "success" if not user_corrected else "failure")import { WarmWinter } from "./warmwinter";
const ww = new WarmWinter({ apiKey: "ww_YOUR_KEY", baseUrl: "https://api.warmwinter.io" });
// Gate: is the retrieval grounded enough to answer?
const d = await ww.decide({
domain: "rag", decisionType: "rag_answer",
statedConfidence: retrievalScore(hits),
onUngrounded: "abstain", // don't guess off the frontier
});
const answer = d.verdict === "act" ? generate(q, hits) : "I don't know that yet.";
await ww.outcome(d.decisionId, userCorrected ? "failure" : "success");Why it works
The gate returns act only where it has verified — against your own resolved outcomes — that this kind of action is trustworthy; otherwise it escalates or abstains. Stakesraise the bar for riskier actions. Nothing is graded on volume — only on outcomes that actually resolved. There is no global accuracy number; trust is per cell. That calibrated humility is the point: the system tells you where it doesn't know yet.
Wrap this call in about five minutes. Free to start.
Get your API key →