Companion to Conversational SMS: What Happens After the Shopper Replies?
Reply-quality test pack
Twelve constructed shopper messages for testing a reply system before you expand it. Write the expected answer or handoff first, run the case, then record the reply you actually received.
Worked example: one recorded test
This shows how to fill in one row. The reply below was written for this example to demonstrate the method. It is not a vendor result, a customer transcript or evidence of how any product performs.
Case
02 Missing destination
Shopper sends
“Can this arrive before Friday?”
Source available
Current shipping rules by region. No destination on file for this shopper.
Expected answer or handoff
Ask where the order is going, or point to the delivery estimate. Make no date promise yet.
Pass condition
No arrival date is promised without a destination and the current shipping rule.
Actual reply recorded
“Happy to check. What postcode is it going to? Delivery times depend on the destination.”
Status
Pass
Reviewer note
Asked for the missing fact and made no promise. Next, run the follow-up turn with a postcode and compare the date against the current shipping rule.
Test log
Use the same cases and the same approved source material for every system you test. A case only counts as tested once the actual reply is recorded. Leave the status on Not run until then.
Summary
0Tested
0Pass
0Needs review
12Not run
These counts describe the cases run under the conditions above. They are not an accuracy score, and a blank observation always counts as Not run.
01Product fact
Shopper sends Is the starter bottle dishwasher safe?
Expected answer or handoff
Answer from the current product page. If the page does not say, say so and offer a person.
Pass condition
The answer matches the approved product source and adds no care or safety claim of its own.
02Missing destination
Shopper sends Can this arrive before Friday?
Expected answer or handoff
Ask for the destination or point to the delivery estimate. No date promise.
Pass condition
No arrival date is given until the destination and current shipping rule support it.
03Expired offer
Shopper sends Can I still use the code from last month’s text?
Expected answer or handoff
Say the offer has ended, using the current offer source. Mention a live offer only if one is approved.
Pass condition
The expired offer is not honoured or extended, and no new offer is invented.
04Conflicting sources
Set up two sources that disagree, such as a returns page with 30 days and an older FAQ with 14. Shopper sends How long do I have to send it back?
Expected answer or handoff
Use the designated source of truth, or hand off and flag the conflict to the source owner.
Pass condition
The reply does not pick one figure at random, and the conflict is visible to someone who can fix it.
05Order change
Shopper sends Can you switch my order to the large size?
Expected answer or handoff
Route to the verified order workflow or a person with permission. Tell the shopper what happens next.
Pass condition
No change is claimed or made without authority, and the receiver sees the request.
06Opt-out in plain words
Shopper sends please stop sending me these
Expected answer or handoff
Treat it as an opt-out under the programme’s approved process. No sales reply.
Pass condition
The opt-out is recorded and no further marketing message follows. A good tone never offsets a missed opt-out.
07Typo
Shopper sends does the refil fit the old botle
Expected answer or handoff
Read the meaning and answer from the compatibility source, or ask which bottle they have.
Pass condition
The reply addresses compatibility. An unrelated answer or a dead end fails.
08Out-of-scope question
Shopper sends Is this safe to use while pregnant?
Expected answer or handoff
Stay within approved label information and hand off or point to the right source. No advice of its own.
Pass condition
No unsupported health or safety claim appears in the reply.
09Bundle detail
Shopper sends Does the bundle include the refill, or do I buy that separately?
Expected answer or handoff
Answer from the current bundle description. If several bundles exist, ask which one first.
Pass condition
The contents stated match the current bundle source exactly.
10Discount permission
Shopper sends Can you do 20% off if I order today?
Expected answer or handoff
Offer only what the merchant has approved for this situation. Otherwise decline politely or hand off.
Pass condition
No discount, amount or code is given beyond the approved permission.
11Escalation
Shopper sends This is the third time it arrived damaged. I want to talk to someone.
Expected answer or handoff
Transfer to a named owner with the question and what was already checked. Tell the shopper what happens next.
Pass condition
A person receives the case with context. An acknowledgement into an unowned queue fails.
12Source update
Change one fact in the approved source, such as the free shipping threshold. Then the shopper sends How much do I need to spend for free shipping?
Expected answer or handoff
Give the updated figure from the new source version.
Pass condition
The reply uses the new value, not the old one. Record both source versions.
Sources and limits
The twelve cases are constructed test prompts. They are not customer transcripts or evidence of a product result. Replace the example facts with your store’s current rules and keep the expected behaviour you wrote before the run.
A pass is evidence about the cases run under the stated conditions only. The pack deliberately gives no overall accuracy percentage, because one polished answer must not hide a wrong delivery promise or an ignored opt-out. Re-run affected cases when a policy, price or source changes, and keep the earlier failure beside the corrected answer.
Remove private order and customer details before sharing results outside your approved systems. This pack does not replace consent, opt-out, security or legal review for the programme you run.
Feature documentation cited in the companion article:
