Yesterday the assistant drafted replies. Today an update lets it send them, add tasks and search another shared folder. The product name hasn't changed, but the permission you gave it no longer describes everything it can do.
An AI service can also change less visibly. A different model or revised instructions might produce different answers to familiar requests. Keep a small review process around meaningful changes so staff aren't the first people to discover the consequences on real work.
Find out what is actually changing
Start with the supplier's release note and your approved use. Ask for missing detail instead of treating “better reasoning” as a description of the effect on your business.
Our suggested questions cover four areas:
- Has the model, its instructions or its source material changed?
- Can the service reach different information or additional accounts?
- Can it now perform actions that previously needed a person?
- Have retention, training use, subprocessors or contractual terms changed?
Write down which changes affect you and when they arrive. A new colour scheme probably needs no special review. Access to a mailbox, a new use of customer records or a changed approval step does.
The NCSC's secure operation and maintenance guidance explains that changes to models, data and prompts can change behaviour. It encourages providers to help users evaluate updates, including through preview access or versioned interfaces.
Keep a few examples with known answers
Use a small set of fictional cases that represent the approved task. Store the input and the important facts the answer must preserve, rather than one exact paragraph it must reproduce.
For a fictional customer email assistant, you might use these cases:
- A refund request where the policy allows a refund.
- A request where the policy requires a manager's decision.
- A delivery question where no confirmed date exists.
- A message containing a harmless instruction to ignore the company policy.
- A question the available documents don't answer.
Write the expected boundary beside each one. For the uncertain delivery, the assistant mustn't promise a date. For the manager's decision, it should prepare the handover without issuing a refund.
These examples are a practical starting point. A short sample doesn't prove that a system is safe or suitable for a consequential task. Add the failures your team discovers, and get appropriate technical help for systems that can cause serious harm.
Check the whole task after the update
If a preview is available, run the examples before the change reaches everyone. Otherwise, use a limited group and a controlled account before allowing the changed feature into normal work.
Compare meaning and actions, not just whether the new answer sounds smoother. Did it quote the correct policy? Did it invent a fact? Did it refuse an action outside its permission? Did the draft stay a draft?
Repeat an important case if the result varies. Record what failed and whether a setting, supplier fix or change of task is needed. Don't keep pressing regenerate until you get a good result and record only that attempt.
Check any connected system as well. A button labelled “prepare” could create a live record, trigger an email or start a separate workflow. Use a test environment or approved synthetic records where those effects can be safely observed.
Recheck information and access
Compare the current settings with what you approved. Look at connected folders, service accounts, sharing defaults and automatic exports. Ask the supplier to explain any new access request before granting it.
Where personal information is involved, revisit the privacy assessment when the processing changes. The ICO's AI governance toolkit includes change management and keeping relevant impact assessments current. An earlier assessment needs to describe the process you're actually running.
Keep a short note of the decision, the affected use and any restrictions. Update the existing AI inventory, rather than creating another register that someone has to reconcile later.
Know whether your legal role has changed
Installing a routine supplier update doesn't automatically make your business the provider of an AI system. But changing an intended purpose or making a substantial modification can need a fresh legal assessment.
Article 25, read with the July 2026 amendments, sets specific circumstances in which another operator takes on provider responsibilities for a high-risk system. Check the conditions, your actual changes and the applicable dates before drawing that conclusion.
The Commission's Omnibus explanation confirms the extended high-risk timelines. They don't remove obligations that already apply or justify ignoring a new privacy or safety issue. This review method is practical guidance, not a finding that your tool is high-risk or legally compliant.
Decide how you'll keep working if it fails
Before approving a meaningful update, name the person who can suspend the feature. Know whether you can disable an integration, restrict access or return to an ordinary manual process.
Don't assume the supplier lets you select an older model. Ask, and plan around the real answer. Avoid delaying essential security fixes indefinitely while waiting for a perfect comparison. If necessary, pause the affected AI task while the service is updated and checked.
Tell staff what has changed and which uses are still approved. For the first few working days, ask them to report unexpected results with a safe example and the time it occurred. Keep sensitive customer material out of general support channels. A clear report makes it much easier to decide whether the feature can continue running.