AI Feature Rollback and the Threshold You Set Too Late

The alert fires at 16:20 on a Friday. Complaints about the new summarisation feature have tripled since Tuesday, three examples in the ticket queue are wrong in a way that will embarrass someone senior, and the one person who can authorise switching it off is on a train. Nobody agreed in advance what number would trigger the decision, so the next ninety minutes go on inventing one while the queue grows.

An AI feature rollback is the decision to withdraw a live model-driven behaviour and return users to the previous one — an earlier model version, the deterministic rule it replaced, or nothing at all. The mechanism is rarely what stops you. What stops you is setting the threshold during the incident, under time pressure, in front of the people who sponsored the launch.

When to roll back an AI feature

Roll back when a guardrail metric breaches an agreed threshold and the cause is not understood inside the agreed window. Roll back immediately, without diagnosis, when the failure is irreversible for the customer — money moved, a decision recorded, a message sent. Everything else can wait for the next deployment.

The delay has a mechanism worth naming, because it repeats across teams and industries. Diagnosis feels like progress and switching off feels like an admission, so the first hour goes on proving the model is at fault rather than removing the exposure. Every minute of that hour writes more records nobody can unwrite. Rollback and diagnosis are not sequential steps — the rollback buys the hours in which the diagnosis happens.

Hold the rollback in three cases, and only three. When the breach is confined to a segment you can exclude at the routing layer. When the previous behaviour is measurably worse than the degraded one, which is the case teams forget: reverting a fraud model to last year's rules can raise losses rather than lower them. And when the metric that moved is one you already knew was unstable below a certain daily volume.

What an AI feature rollback actually reverses

A rollback removes the feature from the path of new requests. It does not remove what the feature already did: records written, emails sent, decisions logged, downstream systems that consumed the output, and users who have changed their behaviour in response. Those are cleanup projects, and they are usually larger than the incident itself.

The specifics are unglamorous and they decide how bad the week gets. A generated summary sitting in a customer relationship management (CRM) free-text field is now indistinguishable from one a human wrote, because the flag marking it as AI-assisted was added to the audit table but never to the field the operations team actually reads. Three weeks of those are already in the system when you switch the feature off.

The second category is worse because it is silent. If the feature's outputs feed anything that learns — a ranking model, a routing heuristic, a fine-tuning set someone assembles quarterly — the regression has already propagated into a system with its own release cycle. Ask during design which downstream consumers read your feature's output, and treat each one as a separate rollback with a separate owner. That list takes an afternoon before launch and cannot be built during an incident.

Rollback routes and how each one fails

Every rollback route trades speed against capability loss, and every one of them has a failure mode that shows up only in production. Choose the route during design, test it in a live release, and write the choice into the acceptance criteria for the story itself rather than into a runbook nobody has opened since onboarding.

Two of these are not really available to you if the model belongs to somebody else. When a provider deprecates a version or ships an update on its own schedule, reverting is a request rather than a deployment, which makes the rollback route part of the build-or-buy decision rather than an operational detail. Pin versions, and read the deprecation notice period as a hard product constraint.

A rollback threshold is a number, an owner and a window

A usable threshold has three parts: the guardrail metric with its trigger value, the single named person who can act on it, and the elapsed time after which the decision defaults. Miss any one and the threshold does not work under pressure, because the missing part is exactly what gets argued about at 16:20 on a Friday.

Pick a guardrail metric that moves within hours. Accuracy against a held-out set is not one — it is an entry condition measured before launch, and your offline evaluation scores say nothing about the live regression you are looking at. Complaint rate per thousand sessions, abandonment at the step the feature sits on, manual-correction rate, and escalation rate all move fast enough to act on.

The window is the part teams skip. Without one, "we are still investigating" is an indefinitely renewable position, and the feature stays live through the whole of it. Write the default direction explicitly: if the named owner cannot be reached within the window, the feature goes off automatically, and turning it back on requires the same authority that would have kept it running. Defaults are what actually execute; everything else is intent.

Guardrail metrics come from controlled experimentation, where Ron Kohavi, Diane Tang and Ya Xu define them in Trustworthy Online Controlled Experiments (Cambridge University Press, 2020) as the measures you watch to catch what a change breaks while it improves something else. The adaptation for an AI feature is that the guardrail must be behavioural rather than a model score, because the model can perform exactly as evaluated while the feature fails the user.

How to agree the rollback threshold before launch

Agreeing a threshold takes one meeting and produces four lines of text. Run it before the release, with the person who will be accountable in the room, and treat the output as a launch artefact of the same standing as the acceptance criteria. Seven steps, in this order.

1. Name one guardrail metric that moves inside a working day, with the exact query or dashboard that produces it. Two metrics are acceptable; five is a way of avoiding the decision.
2. Set the trigger value and the measurement window in one sentence — for example, manual-correction rate above eight per cent measured over any rolling four hours with at least two hundred sessions.
3. Name the person who can call it alone, name a deputy, and write both sets of working hours including time zone. An on-call rota that contains only engineers cannot authorise a decision about customer-visible behaviour.
4. Write the default: what happens if neither can be reached inside the window. Then confirm the automation actually implements the default rather than paging someone about it.
5. Choose one route from the table above, and exercise it in production before launch — the flag flipped off in a live release, timed, with the old path serving real traffic for at least an hour.
6. Draft the customer message now, while nobody is angry, and get it approved by whoever approves customer communications. That approval takes days and it takes them after the incident too.
7. Rehearse once end to end and record the elapsed minutes. That number, not the theoretical one, is your recovery time, and it belongs in the launch pack.

The predictable failure of this procedure is that teams complete steps one to four, skip five and six, and discover mid-incident that the off switch throws an error and the holding statement needs legal review. An untested rollback route is not a plan; it is a hypothesis with a customer impact attached. Five and six are the only steps that cost real time before launch, which is exactly why they get moved to next sprint.

Who makes the call while the regulatory clock runs

In a regulated firm the rollback decision and the reporting decision are separate, run on different clocks, and are usually made by different people. Product decides whether the feature stays live. Compliance or risk decides whether the event is reportable. Neither decision waits for the other, and assuming otherwise is how firms miss a deadline they never knew had started.

The clocks are specific. Under the European Union Artificial Intelligence Act (EU AI Act), Article 73 requires providers of high-risk AI systems to report serious incidents to the market surveillance authority immediately after establishing a causal link with the system or the reasonable likelihood of one, and in any event not later than fifteen days after becoming aware of it — two days for a widespread infringement, ten days where a person has died. Article 20 separately requires immediate corrective action: bring the system into conformity, withdraw it, disable it or recall it, and inform distributors, deployers and importers. The dates on which the high-risk obligations bite have themselves moved during 2026, so check the applicable date rather than the one in your last compliance briefing.

Outside that regime, the expectation is the same in substance. The National Institute of Standards and Technology (NIST) AI Risk Management Framework playbook states under MANAGE 2.4 that mechanisms are in place and applied, and responsibilities assigned and understood, to supersede, disengage or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use. Read as a product requirement rather than a governance principle, that sentence asks for exactly the three things above: a mechanism, a named owner, and a trigger.

The clock starts when someone in your organisation becomes aware, which means your support team's first ticket may have started it eleven days before anyone told you. That is an argument for routing AI-feature complaints to a distinct queue with a named product owner, and it is the sort of plumbing that gets built after the first incident rather than before it. In firms that already run a defined incident classification with the second line, add the AI feature to it explicitly; classification schemes written before 2024 rarely have a category your feature fits.

Terms to agree before the incident, not during it

Six terms cause most of the confusion in a live rollback conversation, usually because product, engineering and risk each use one of them to mean something slightly different. Agree them once, in writing, in the launch pack. A ten-minute argument about what "kill switch" covers is ten minutes of live customer exposure.

• AI feature rollback is the withdrawal of a live model-driven behaviour and the return of users to a defined previous behaviour.
• Kill switch is the mechanism that removes the feature from the request path immediately, without a deployment and without diagnosis.
• Guardrail metric is a behavioural measure watched to detect harm from a change, as distinct from the metric the change was meant to improve.
• Regression threshold is the trigger value of a guardrail metric, stated with its measurement window, above or below which the rollback decision is taken.
• Blast radius is the set of users, records and downstream systems affected between the feature going live and the rollback completing.
• Shadow mode is running the model on live traffic while its output is logged rather than shown, which produces real readings without customer exposure.

With the threshold agreed, the route tested and the owner named, the next question arrives immediately and it is harder: what has to be true for this feature to be finished in the first place. Accuracy against a target is an entry condition, not a definition of done, and the rest of that definition is monitoring, failure user experience and the rollback you have just designed — which is worth settling in the same week you decide whether the feature needed a model at all.

Frequently asked questions

When should you roll back an AI feature?

Roll back when an agreed guardrail metric breaches its threshold and the cause is not understood within the agreed window. Roll back immediately, before diagnosis, when the failure is irreversible for the customer — money moved, a decision recorded, a message sent externally. Reversible degradation in a contained segment can usually wait for the next scheduled deployment.

What is the difference between a rollback and a kill switch?

A kill switch is a mechanism: it removes the feature from the request path immediately, without a deployment. A rollback is a decision that may use a kill switch, or may instead revert to a previous model version, fall back to a deterministic rule, or route traffic to human review. Teams that own a kill switch but no agreed threshold still hesitate for hours.

Who decides to roll back an AI feature?

One named person, agreed before launch, with a named deputy and stated working hours. In practice this is the product manager or the accountable business owner, not the engineer on call, because the decision concerns customer-visible behaviour rather than system health. If nobody can be reached inside the agreed window, a written default should switch the feature off automatically.

Can you roll back an AI feature that has already written to customer records?

You can stop it writing more, which is what a rollback does. The records already written are a separate remediation with its own scope, owner and timeline, and it is usually the larger piece of work. Design for this before launch by tagging every AI-generated field at the point of write, so the affected population can be identified with a query rather than reconstructed from logs.

Do you have to report an AI feature rollback to a regulator?

The rollback itself is not the reportable event; the underlying incident may be. Under the EU AI Act, providers of high-risk AI systems report serious incidents to the market surveillance authority within fifteen days of becoming aware, two days for widespread infringement and ten where a death is involved. Classification sits with compliance or risk, not with the product team, so escalate early.

How do you test a rollback before launch?

Exercise the actual route in production, not in staging: flip the flag off in a live release, serve real traffic on the old path for at least an hour, and time the whole sequence from decision to restored behaviour. Record the elapsed minutes as your recovery time. A route that has only been tested in a lower environment has not been tested.