86% of brands think their AI works. Only 51% of customers agree.

86% of brands think their AI works. Only 51% of customers agree.

Ask a company if their customers are happy with its AI support, and 86% will say yes.

Ask their customers, and only 51% agree.

Hmm…what’s going on here? Both can’t be right, can they?

Businesses don’t usually throw numbers around willy-nilly, so they probably have data to back it up. Customers are having a very different experience, though (a 35-point gap experience).

Odds are brands aren’t over-optimistic. It's more likely they have dashboards telling them they're right. But those dashboards are wrong. Here’s why.

Why brands and customers don’t agree on AI satisfaction

Most companies are measuring their AI performance, but they’re not looking at the right metrics. That’s because there’s inherent bias behind the answers (and who actually answers them):

• Completed conversations are the only ones that answer surveys. A post-interaction CSAT survey only reaches people who stayed to the end. Someone who gave up at minute two and called a competitor never gets it. Twilio research found 71% of consumers will abandon a conversation entirely if an AI fails to recognize who they are. Almost none of those abandonments show up in a satisfaction score.

• Containment is interpreted as success. A conversation the AI handled without escalating counts as contained. It also counts as contained when the customer gave up and stopped replying. Those two outcomes look identical in the data, but they’re definitely not the same in your customer's eyes.

• Brands measure interactions, while customers measure progress. Your dashboard scores a session, but the customer scores based on whether the thing they needed got done. And that includes the three previous attempts they made on other channels. Twilio’s Inside the Conversational AI Revolution found 49% of consumers say AI never resolved their issue.

Other data points fill out the gaps in the story

This isn’t a one-off disagreement. AI perception from brands and consumers just isn’t quite aligned (yet). There are other places where this shows up.

73% of brands think their customers want AI to exhibit more human-like personalities. Yet, only 54% of consumers agree. Nearly half are neutral or don't care whether the assistant sounds human—they just want it to help them.

No, that’s not a massive discrepancy, but it is another example of businesses missing the mark with customers.

You don’t have to keep guessing what your customers want, though. They actually told us.

What customers said they wanted

When consumers were asked what matters most in customer support, the answers were boringly basic and bland.:

• Immediate issue resolution: 40%

• Speed of assistance: 40%

• Accuracy of information: 38%

• 24/7 availability: 38%

Nope, they don’t care about how human or empathetic your AI is. They want practicality. Four operational attributes, all of which are measurable, but none of which a satisfaction survey of completed conversations will tell you much about.

Here’s what to measure instead

You don’t need to replace your CSAT scores. Those are genuinely helpful, just not in this exact use case. Instead, layer on these other measurements:

• Abandonment, tracked by stage: Where in the conversation people stop replying. This is the population your satisfaction score can't see, and it's the one with the strongest opinions.

• Resolution rate, measured on the errand: Whether the customer's underlying issue closed, including across sessions and channels. A contained conversation that produced a callback tomorrow doesn’t count.

• Repeat contact within seven days: The cleanest proxy for something the first interaction failed to fix. It requires no survey and nobody can game it.

• Containment split by outcome: Separate conversations the AI resolved from conversations the customer exited. Those aren’t the same.

• Escalation quality: How often a transferred customer has to re-explain their situation. 76% say the human agent they reach has little or no context about them and their issue.

Close the gap before you defend the score

The 86% isn't a lie your team told you. They’re not gaming the metrics. It's the honest output of a measurement system that only sees the customers who stuck around.

Now, you need to find out how many didn't. Pull abandonment by stage for one month and compare it against your satisfaction sample. The distance between those two populations is probably the distance between 86 and 51, and it's the part of the customer experience nobody on your team has read.

Want the research behind these numbers? Read the Customer Insights Series guide on why customers value empathy over imitation, or see how context and trust interact in How to Scale Automation Without Losing Customer Confidence. Ready to build? Start with Twilio for free.

Frequently asked questions

Are customers satisfied with AI customer service? 

Twilio research found a 35-point gap. 86% of companies believe their customers are satisfied with AI-powered service, while only 51% of consumers agree.

Why is my AI CSAT score higher than customer sentiment? 

Post-interaction surveys reach customers who completed a conversation. Twilio research found 71% of consumers abandon a conversation when an AI fails to recognize them, and those exits rarely appear in satisfaction data.

Is containment rate a good measure of AI success? 

On its own, no. A contained conversation looks identical whether the AI resolved the issue or the customer gave up. Split containment by outcome and track abandonment by stage alongside it.

What should I measure for AI customer service? 

Look at abandonment by conversation stage, resolution measured on the customer's underlying issue, repeat contact within seven days, containment split by outcome, and how often escalated customers are forced to re-explain their problem.

Do customers want AI to sound more human? 

Less than brands assume. Twilio research found 73% of brands believe customers want more human-like AI personalities, while 54% of consumers agree. Consumers ranked resolution and speed far above personality.