Quick answer
Conversational SMS is a text exchange in which the shopper can reply and the brand can respond to what they mean. For an ecommerce team, the useful question is what happens to an unexpected reply after the campaign has gone out. A configured keyword, a staffed inbox and a context-aware answer service solve different parts of that problem.
Source context: Read the documented keyword behaviour; Read the current Concierge description.
A reply channel needs an answer owner
Suppose a shopper asks whether an order can arrive before Friday. Another reminder does not answer the question. A useful response needs the destination, the current delivery policy and enough uncertainty to avoid promising what the brand cannot deliver. If that information is missing, the next good action may be a clarifying question or a person taking over.
Build the programme around the questions your store can answer reliably. Establish who owns replies, which information they may use, what they may promise and what they must hand off. Then test the response itself. A message arriving successfully says little about whether the shopper received a useful answer.
Three capabilities that should not share one label
| Capability | Good fit | What to test |
|---|---|---|
| Configured keywords | A small set of explicit choices, such as size or category | Typos, unexpected phrases and the route for unmatched replies |
| Staffed inbox | Questions needing judgment, exceptions or account access | Hours, ownership, backlog and transfer of context |
| AI-assisted or automated replies | Repeatable questions supported by approved information | Grounding, uncertainty, permissions and handoff |
Keyword flows can be the right choice when the question really is a choice. They keep the path predictable and make the response easy to inspect. They become brittle when a shopper answers in their own words and the programme has no useful fallback.
Attentive’s Two-Way Journeys documentation describes configured keywords and an unmatched-reply case that may receive no answer unless an autoresponder is set. That describes this particular feature. It does not describe every Attentive conversation product. Its current Concierge page separately describes live agents working with conversational AI.
Design the next turn before the first message
Start with the incoming question and the context already available. A shopper asking about flavour needs different information from a shopper asking to change a paid order. The first may be answered from a product page. The second may require a verified account workflow and a person with permission to change the order.
Write down the source of truth for each answer family. Delivery estimates need current shipping rules. Returns need the actual policy, including exceptions. Subscription changes need the correct account route. Someone must own updates to those sources, because a fluent answer to an old policy is still a wrong answer.
Define a useful handoff as a transfer with context. The receiving person should see the shopper’s question, the information already checked and why the system stopped. An automated acknowledgement that quietly leaves the shopper in an unowned queue is an unresolved question.
Test the awkward cases, not just the happy path
The downloadable pack starts with twelve constructed test cases. It covers a straightforward product question, missing delivery details, an expired offer, contradictory information, an order change, an opt-out request, a typo and a question outside the approved knowledge. These are test prompts, not customer transcripts or evidence of a product result.
Write the expected answer or safe next action before running the demo. Keep the actual reply beside that expectation. Judge factual support, relevance, promise limits and handoff separately. One polished answer must not compensate for a wrong delivery promise or an ignored opt-out.
Use the same cases and current approved source material for every provider. Add a few cases from your own anonymized support categories. Remove private order and customer details before sharing them outside your existing approved systems. Score the observed answer, not the salesperson’s description of what it could do.
Download the reply-quality test pack. It includes a completed example and an editable version. Get the short printable brief.
Accuracy needs a source, a limit and a test
| Question | Required behaviour | Failure to record |
|---|---|---|
| Is this an approved fact? | Use the current product or policy source | Unsupported product, shipping or offer claim |
| Is essential context missing? | Ask for it or transfer the case | Confident answer based on an assumed fact |
| Is the action permitted? | Stay within merchant-approved permissions | Order change or discount without authority |
| Can the issue be resolved here? | Transfer context to a named owner | Looping answers or an unowned queue |
Do not ask only for an overall accuracy percentage. Ask what counted as a correct response, which questions were sampled and whether serious errors were weighted or hidden inside an average. A provider can answer many easy questions correctly while failing the few cases that cost the store the most.
Version the test pack with the policy and product material used in the run. Re-run the affected cases when a policy, price or source changes. Keep the old failure and the corrected answer. Otherwise the next model or knowledge update can bring back a mistake that everyone thought was fixed.
A test pass is evidence about the cases run under those conditions. Broader reliability needs real operating observation, review of unresolved cases and sampling over time. Start with a scope that the team can inspect.
A helpful answer still has to earn its cost
Take a $300 monthly cost for the proposed reply service and $30 contribution from an additional order after variable order costs. The service needs ten additional orders to cover that cost. If orders carry extra discounts, refunds or support costs, the contribution per order is lower and the hurdle rises.
The number of conversations is useful for staffing and billing. It does not show how many orders the conversations caused. Keep a comparison group or another defensible evaluation design, count outcomes over the same window and include the extra service and message costs. The existing SMS ROI article explains that measurement problem in more depth.
Also look at the cases where the right result was an honest refusal, an accurate policy answer or a quick transfer. Those outcomes can matter even when no order follows. Decide which operating outcomes you need alongside the commercial result before the pilot starts.
Where Zobi fits the reply decision
If your cart follow-up already gets replies, inspect the unanswered questions before adding another incentive. The next useful test may be whether a relevant answer can remove a buying obstacle without introducing an unsupported promise.
Zobi’s EQ models are designed to read a shopper’s question in context and choose a relevant reply from merchant-approved information, within the programme parameters the merchant approves.
That makes checkout questions and repeat-purchase conversations a specific job to evaluate with Zobi. Bring the approved product information, the questions that stop shoppers and the handoff rules. Use the same reply test pack to inspect how the proposed flow handles an unknown fact, an exception and a question requiring a person.
Questions to settle before rollout
Does two-way SMS mean an AI is answering?
No. Replies may go to a keyword branch, a person, an automated system or a combination. Ask who or what handles each type of reply and which parts are included in the quoted service.
Should every reply get an automatic answer?
Only when the system has the information and permission to give a useful answer. A clear clarification or handoff is better than an unsupported promise. Separate marketing replies from account changes and sensitive questions.
What should the first pilot cover?
Choose one repeatable question family, its approved source and a named fallback owner. Test the exception cases, then observe the live unresolved queue within the approved scope before adding more question families.
Sources and how to use this guide
This guide combines primary feature documentation checked on 28 October 2026 with a proposed operating method and a worked cost example. Documentation establishes described behaviour at the cited scope. It is not a hands-on comparison of provider accuracy. The examples, test cases and break-even calculation are original teaching material.
The twelve-case pack is a starting instrument. Replace its source assumptions with the current rules for your store, retain the expected behaviour and keep the observed result. It does not replace consent, opt-out, security or legal review for the programme you run.
Publisher disclosure
Zobi publishes this guide and offers conversational commerce services. Product descriptions, worked examples and source documentation have distinct evidential roles. This is an unranked editorial guide, not independent product testing.
Rene writes about ecommerce retention and conversational commerce at Zobi. This guide combines primary documentation with practical decision tools.
Tell us about your store.
Start growing revenue