Shopify AI Support Making Mistakes

Mistakes cluster. QA the cluster, ship one rule.

When to use this list

One invented number is a page bug. Ten is a permissions bug. Do not 'tune the prompt' first. Print it or put it in the ticket you open with the vendor. If they cannot walk it, you are in a demo, not a trial.

The checklist

• Write the definition of ai support making mistakes in one sentence the agent can test against an order.
• List the Shopify fields it must read before it replies (status, tags, tracking, refunds already issued).
• Write the allowed actions and the cap. If there is no action, write the exact escalate condition.
• Add two examples: a yes case and a no case. Agents learn more from the no.
• QA ten live tickets after a week. Change the article or the cap, not the vibe.

What you can skip

A new helpdesk if the agent can sit on the tickets you already have.

Automatic fail

• They will not refund a test order.
• They need a solutions engineer for a 400-ticket store.
• The only metric they show is conversations started.

Load this rule into Relay and stop solving ai support making mistakes by hand at midnight.