AI Feedback Loops: Keep Recommendations Out of Ground Truth

A model can improve the metric it is trained to predict while making that metric less trustworthy. A support prioritiser moves certain cases to the front, those cases receive faster specialist attention, and their resolution rate rises. If the next training set records the improved outcomes without the routing decision, the model learns from a world it helped create. Its recommendation has entered its own evidence.

The operating decision is therefore not simply when to retrain. It is whether an observed outcome describes the underlying case, the intervention chosen for it, or both. Until that distinction is recoverable, production labels should not flow automatically into evaluation or training.

This field guide separates recommendations, policy, actions and outcomes so teams can detect and control decision feedback. It is different from ordinary input drift: the system is not merely observing change; it may be causing which examples are seen, which treatment they receive and which labels can exist. The cited research and standards establish those mechanisms and obligations. The evidence contract, release gates and worked support example below are AIEngine's implementation synthesis.

Draw the loop before measuring the model

Start with the complete path, not a model dashboard:

• a model scores, ranks or recommends;
• a policy turns that output into an offer, route or action;
• a person or automated service may accept, override or modify it;
• the action changes exposure, service or opportunity;
• an outcome is observed for some cases; and
• a label derived from that outcome returns to evaluation or training.

The Performative Prediction paper gives the central mechanism a precise name: a prediction is performative when acting on it influences the target it predicts. The authors study conditions under which repeated retraining converges, but convergence is not the same as an organisation reaching the outcome it intended. A stable system can be stably optimising a policy-shaped target.

This is why the response cannot be “monitor drift and retrain”. The AI drift monitoring guide explains how to identify changes in inputs, concepts, labels and outcomes. A feedback-loop analysis asks the additional causal question: which of those changes occurred because this system or its users acted on an earlier output?

Separate four records that are often collapsed into one label

One event table with a final status is rarely enough. Keep four linked records with separate owners and clocks:

Use stable case and decision identifiers to join the records, while keeping personal data out of identifiers and logs that do not need it. Record both event time and recorded time. A late resolution, corrected fraud decision or reopened ticket must not overwrite the version that an earlier evaluation used. The temporal-correctness guide provides the matching rules for point-in-time reconstruction.

The separation also exposes responsibility. The model owner can explain a score, the policy owner can explain why a threshold applied, operations can confirm the action, and the outcome owner can define when the result is mature. “The model approved it” is not a valid substitute for any of those records.

Mark how every label became observable

Some outcomes exist only after a decision. Loan performance is observed for approved loans, job performance for hired candidates, and response to a treatment for people who received it. The original selective-labels research shows why evaluating a new decision rule on such data is difficult: prior human choices determine which cases have labels, and decision-makers may have used relevant information that was never recorded for the algorithm.

Give each potential label an observation state rather than forcing every row into positive, negative or null:

• independent: observed through a process not materially changed by the recommendation;
• exposed: observable because the case received a particular offer, position or intervention;
• censored: the decision prevented the counterfactual outcome from being observed;
• proxy: a measurable behaviour stands in for the outcome the organisation actually values;
• adjudicated: a competent review resolved conflicting or incomplete evidence; or
• immature: the agreed outcome window has not closed.

Missing is not negative. Unresolved is not unsuccessful. A click proves exposure and interaction, not satisfaction. If label provenance is unknown, the record is ineligible for a release claim until the uncertainty is handled explicitly.

Log the policy that created the data

Google's Rules of Machine Learning warns that position affects interaction: putting an item first can make it more likely to be clicked and then make the system believe it deserved first place. The same guidance identifies feedback between a model and its algorithm as one source of training-serving skew. Google's production-system checklist separately asks teams to determine whether a model affects its own or another model's future inputs.

For each action, preserve the candidate set the policy could have chosen from, the selected action, ordering or intensity, policy version, model output available at that moment, operator intervention and outcome window. For controlled exploration, also retain the assignment mechanism and probability needed by the approved analysis. Do not reconstruct these fields later from the current policy: thresholds, availability and user interfaces change.

Logging a probability does not make an estimate causal. Off-policy and counterfactual methods depend on assumptions about coverage, confounding and the policy that produced the data. A domain statistician or causal-inference specialist should approve their use for consequential decisions. The engineering requirement is narrower and immediate: preserve the path so a qualified analysis remains possible.

Preserve evidence outside the loop

The safest source of comparison depends on consequence and feasibility. Options include a fixed expert-reviewed evaluation set, blind re-review of a risk-based sample, outcomes supplied by an independent system of record, a shadow candidate that cannot alter production, or a controlled experiment where randomisation is ethical and operationally safe.

Do not randomise harmful treatment, deny necessary service or expose protected data merely to improve identifiability. The selective-labels paper notes that acquiring labels experimentally may be infeasible or unethical in consequential settings. Where experimentation is inappropriate, narrow the claim: measure performance only on the population for which outcomes are observable, use bounds or sensitivity analysis, and state what cannot be inferred.

Shadow traffic is useful for response quality, latency and compatibility because the candidate is prevented from changing the user-facing action. The shadow-testing guide explains that boundary. It cannot reveal the long-term consequence of an action the shadow path never took, so a canary or other governed field design may still be necessary after the non-acting evidence passes.

Stop naive retraining from hardening the loop

Treat production feedback as a proposed dataset, not a trusted stream. Before promotion, validate the observation state, policy version, outcome maturity, duplicate and correction handling, consent or lawful-use conditions, subgroup coverage and whether the label was derived from another model's output.

Quarantine records whose action path is missing. Keep model suggestions, reviewer decisions and later outcomes in separate fields. If reviewers see the model output, their judgement is not independent evidence unless the evaluation is designed to account for that influence. Do not convert thumbs-up, completion, silence or an absence of complaints into ground truth without defining the behaviour and its known alternatives.

Repeated retraining can amplify an early policy choice: the system sees more evidence from the cases it already favours and less from cases it suppresses. The EU AI Act's Article 15 text specifically requires high-risk systems that continue learning after deployment to reduce the risk of biased outputs influencing future inputs and to address feedback loops with mitigation. Scope and legal obligations require case-specific advice, but the engineering issue exists beyond regulated high-risk systems and even when updates are made in batches.

Work through a support-priority model

Suppose a model predicts whether a new support case needs a specialist. High-scoring cases go to the front of the specialist queue; low-scoring cases remain with general support. The team later labels “specialist needed” when a specialist resolved the issue quickly.

That label is contaminated in three ways. Queue position changed response time. Specialist exposure changed the chance of resolution. Low-scoring cases that never reached a specialist cannot show what that specialist would have achieved. Retraining on the final status alone will reward the route the current model already favours.

A safer design stores the recommendation and its reasons, the routing policy, actual queue position, transfers, staff overrides, first-response and resolution times, reopenings, customer-confirmed outcome and the final adjudicated issue class. A blind quality team reviews a stratified sample from every score band, including unresolved and overridden cases. Low-risk cases can enter a small controlled exploration lane only if staffing, customer commitments and safeguards permit it.

Evaluation reports results by original score band, action received and observation state. The team compares resolution quality at a fixed service level, not raw resolution rate after unequal attention. A new model cannot claim improvement by sending more cases to scarce specialists or by allowing difficult cases to age out of the label window.

Make the feedback loop a release gate

NIST's 2026 deployed-AI monitoring report identifies human–AI feedback loops as an under-researched monitoring gap and highlights fragmented logging and limited ground truth as practical barriers. The NIST AI RMF Core calls for post-deployment monitoring, appeal and override mechanisms, documented measurement in deployment context, and adjudicated feedback rather than undifferentiated user signals.

Turn those principles into measures owned across product, operations and model teams:

• loop coverage: share of decision paths mapped from recommendation to mature outcome;
• policy traceability: share of outcomes joined to the exact policy and action that produced exposure;
• observation-state coverage: labelled records with a declared independent, exposed, censored, proxy, adjudicated or immature state;
• censored-outcome rate: cases for which the chosen action prevents the target outcome being known;
• policy-stratified performance: quality and harm measures at comparable action and service levels;
• override and disagreement: operator departures from the recommendation, with reason and later outcome;
• independent-sample delta: difference between routine production labels and blind or external assessment; and
• feedback sensitivity: how conclusions change when exposed, proxy or selectively observed labels are excluded or reweighted under an approved method.

Block retraining when the action path is materially incomplete, the label window is still open, one policy version cannot be reconstructed, a subgroup has no comparable exposure, or an apparent gain disappears on independent evidence. Use the evaluation sample-size guide to keep item counts, clusters and repeated generations distinct; a large number of policy-shaped rows does not create an independent sample.

The decision to revisit

A production outcome is evidence, but it is evidence of a case under a particular policy and action. It becomes fit for evaluation or training only when the team can reconstruct what the system recommended, what was actually done, why the label became observable and which claims remain counterfactual.

Revisit that decision when ranking, thresholds, reviewer instructions, capacity, incentives, user interface, outcome windows or downstream models change. If the policy path cannot be recovered, freeze automatic learning and narrow the performance claim. The objective is not to eliminate every feedback loop. It is to stop a system from treating the consequences of its own choices as independent proof that those choices were right.