Training a model on what actually sells
Language models are trained to predict the next word. We're training one to predict the next order. Here's what it learns from and how we score it.
If you ask a general model to write a sales message, you get something polite and fluent. What you don't get is any idea whether it would have worked, because the model has never seen the result of a message it wrote. It has read a lot of marketing. It has never watched a customer decide.
That's the gap we're working on. We think the useful signal for selling isn't more text, it's the outcome: what the person did after they read the message. Almost none of that exists on the public web. We have a bit over a million of these conversations from running Zobi for commerce brands, each one with what happened next attached, and that's what the model is being trained on.
What the data looks like
Zobi's agents handle abandoned checkouts, shipping questions, reorder reminders and add-on suggestions for the brands we work with. Every thread ends up with a small set of facts about how it went:
- whether they replied, and how quickly
- whether they clicked the link
- whether they bought, for how much, and whether a discount was involved
- whether they came back later or not
Nobody rates these replies. There's no annotator deciding whether something "sounds helpful". The label is just what the customer did, which is a much harder thing to fool.
How it's scored
Most assistant models are tuned with human preference: two answers, a rater picks the one they like. That's fine for "is this explanation clear". It's not much use for "did this close the sale", because a person reading the transcript genuinely can't tell, and honestly neither can we most of the time. The reply that reads best is often not the one that sold.
So we score the model on one thing: revenue per conversation. Each draft gets a predicted chance of converting, we check that against what actually happened, and the difference goes back into training. The aim is that, over enough conversations, the model stops sounding like a salesperson and starts getting the same results as one, which is a different thing.
It knows who it's talking to
The other thing in the data is context about the person, which a product catalogue doesn't have. Before the model writes a reply, it will know whether this is a first order or a repeat one, whether they've been price-sensitive before, when they last bought, which channel they tend to answer on, and roughly what time of day they read messages. That changes what the right reply is. Giving a loyal repeat customer a discount mostly costs margin. Giving the same discount to someone hesitating over a first order on a Sunday night sometimes gets the sale. The model has to learn to tell those apart by seeing it happen a lot, not by being told the rule.
Where it is
The model is training now. Zobi's agents are in production at 80+ brands today, and every conversation they run is data for the next version. We measure improvements with live holdouts so we can tell a real gain from noise.
There's plenty we haven't figured out. How persuasive is too persuasive. Where the line is between a good salesperson and an annoying one. How to make sure a model that's rewarded for revenue doesn't learn to spend the customer's trust to get it, because that's what makes the second order possible. Most of our time goes on those questions right now, and we'll write about them here.
