checkFaithfulness()
Verify that an LLM response is grounded in the memories you provided as context.
Method signature
recall.checkFaithfulness(input: CheckFaithfulnessInput, options?: RequestOptions): Promise<FaithfulnessReport>Parameters
input.responsestringrequiredThe LLM-generated response text to verify.
input.contextMemoryIdsstring[]optionalIDs of the memories provided to the LLM as context. Capped at 50. Get these
from recall.context() — the memoryIds field.
input.config.onHighRisk"warn" | "block"optionalWhat to do when faithfulness risk is high. "warn" (default): return the
report normally. "block": throw RecallConflictError with code
FAITHFULNESS_BLOCKED.
input.config.highRiskThresholdnumberoptionalScore below which risk is high. Default 0.5. Range [0, 1].
input.userIdstringoptionalEnd-user ID (sets X-Recall-User-ID header). Inherited from forUser() if set.
Returns
Per-claim verdicts: text, verdict (supported | partial | unsupported),
and evidence (the supporting memory text, if any).
Fraction of supported claims in [0, 1]. Only fully supported claims count
(partial = 0). An empty claims list returns 1.0.
Aggregated risk level based on score.
Opaque trace ID for correlation.
Examples
import { createClient, RecallConflictError } from '@arc-labs/recall';
const recall = createClient({ url: process.env.RECALL_URL, apiKey: process.env.RECALL_KEY });
// After your LLM generates a response using context memories:
const ctx = await recall.forUser(userId).context({ query: userQuery });
const llmResponse = await llm.complete({ system: ctx.promptSection, user: userQuery });
let report;
try {
report = await recall.forUser(userId).checkFaithfulness({
response: llmResponse,
contextMemoryIds: ctx.memoryIds,
config: { onHighRisk: 'warn' },
});
} catch (e) {
if (e instanceof RecallConflictError && e.code === 'FAITHFULNESS_BLOCKED') {
return 'I cannot confirm that response is accurate.';
}
throw e;
}
if (report.risk === 'high') {
// Show a caveat to the user
return `${llmResponse}\n\n⚠️ Some claims could not be verified.`;
}
return llmResponse;Errors
FAITHFULNESS_BLOCKED409fatalonHighRisk: "block" was set and the score fell below highRiskThreshold.
The LLM response was not grounded enough to show.
NOT_FOUND404fatalOne or more contextMemoryIds does not exist in scope.
VALIDATION_ERROR422fatalMore than 50 contextMemoryIds provided, or an ID is not a valid UUID.
Was this page helpful?