Skip to content
Guides

12 questions to ask before you pick an AI agent for your online store

Ask the vendor to show live order lookup on your own store, to explain exactly when a person takes over, and to put the price of your busiest month in writing. The twelve questions below cover those three and nine more, grouped so you can run them in a single demo or trial.

By Kylix Agent team ·

The 12 questions

Order lookup and handoff (questions 1 to 5)

This is where support tools succeed or fail. Run three real order questions in the demo: where is my order, can I change my address, and I want a refund. Watch for the vendor reaching for a canned answer instead of your store's data. Then ask to see what your team sees when the AI hands over.

Channels and languages (questions 6 and 7)

Match channels to plans before you compare prices, because a cheaper plan that excludes WhatsApp or Instagram is not a like-for-like quote. If a channel is run through Meta, the platform's own rules apply to everyone, so ask how the vendor handles them. For Instagram, see our guide to Meta's DM rules.

Data (questions 8 and 9)

If an app reads Shopify customer records, Shopify requires it to request protected customer data access, process only the minimum data needed, set retention limits and encrypt data at rest and in transit. Ask whether the vendor has gone through that process. For WhatsApp, Meta's Business Solution Terms restrict letting third parties use WhatsApp data to build or improve AI models, with an exception for fine-tuning a model for your exclusive use. Ask the vendor to confirm in writing that your conversations do not train models used for other customers.

A one-hour demo script

  1. Ask the vendor to connect to your store, or a copy of it, before the call starts.
  2. Give it three order lookups with real order numbers, one of them delayed.
  3. Ask a product question whose answer is on your site but not in the product title.
  4. Ask a question you want it to refuse or pass on, such as a refund outside your policy.
  5. Watch the handoff on the other side: who is notified, and what they see.
  6. Close by asking for the price at your busiest month's volume, in writing.

Pricing traps (questions 10 to 12)

  • A per-resolution meter that counts chats the customer abandoned.
  • Per-seat fees that grow when you add a part-time teammate.
  • A low headline plan that excludes the channel you actually need.
  • Annual-only discounts that lock you in before you have tested real traffic.
  • Channel fees from Meta that the quote leaves out. Our cost guide works through them at three volumes.

Frequently asked questions

How long should I trial an AI agent?

Long enough to cover a normal week and one spike, such as a promotion. Fourteen days on live channels is enough to see real order, shipping and returns questions.

What should I test in a trial?

Write ten questions from your own inbox: three order status, two shipping, two returns, two product, one that needs a person. Run them on each channel you plan to use and compare the answers to what you would send.

Should I trust a vendor's resolution-rate claim?

Ask how it is measured, over what period and for which kinds of stores. A rate without a definition of resolved cannot be compared across vendors.

Do I need a developer to set it up?

For Kylix, no. Setup is self-serve: create an account, connect your store, let the agent learn your site and documents, and switch it on. It takes about an hour.

What if I want to leave?

Check that cancellation is self-serve and that your conversation history and knowledge can be exported. Ask before you sign.

See it on your own store

Connect your store and test the agent on real questions. Every plan starts with a 14-day free trial.

Start your 14-day free trial

Sources

  1. Shopify, Work with protected customer data, accessed 11 November 2026
  2. WhatsApp, Business Solution Terms, accessed 11 November 2026

Written by the Kylix Agent team. Meta changes its rules and rates often, so check the linked pages before you rely on a figure.

12 questions to ask before choosing an AI agent — Kylix Agent