How Duolingo uses Twilio Verify WhatsApp with SMS Fallback

At SIGNAL 2022, Duolingo shared how they deployed Twilio's Verify application to authenticate users and connect them with their friends. Furthering our relationship, Twilio invited Duolingo and Nico Sacher, Principal Product Manager, to run an experiment in select countries using WhatsApp as a primary verification channel with SMS as a fallback for One-time Passcodes (OTPs). This blog post will describe the results of that experiment.

Duolingo saw the following results from sending OTPs with WhatsApp and SMS as a fallback channel:

• Improved OTP verification conversion/success rate

• Lower OTP costs

• No code implementation

Our research shows that the experimental WhatsApp with SMS fallback treatment performed better than the SMS-only control treatment, demonstrating a 12% improvement in verification success rate while reducing cost by 87%*. Twilio and Duolingo saw that it took fewer OTP messages to verify a phone number going through WhatsApp with SMS fallback treatment.

Using WhatsApp as an OTP channel

What's the best channel to send One-time Passcodes to your end users? Email? SMS? An Over-the-top (OTT) messaging application like WhatsApp? The answer can sometimes be more complex, with varying customer preferences and regional nuances.

What if your OTPs for multi-factor authentication were delivered to users based on the most cost-effective channel or the channel most likely to convert? While SMS verification is commonly used for OTPs, SMS costs vary by country, and not all users have reliable cellular coverage.

WhatsApp is one of the most popular messaging apps, with over 2 billion users across 180 countries. WhatsApp usage is prevalent in some countries and can be much less expensive for businesses than sending an SMS (up to 90%). On the other hand, consumers are less accustomed to receiving OTPs from WhatsApp, and some don't have the app.

At Twilio, we wanted to explore using WhatsApp as an OTP delivery channel with SMS as a fallback to improve conversion rates and decrease customer costs.

Experiment design

During the experiment, we enabled a new Twilio Verify feature called “WhatsApp with SMS Fallback” for OTP message delivery in select countries. When Duolingo made the Verify API call to send an OTP via SMS, Verify attempted to send an OTP via WhatsApp instead.

Verify identified if the WhatsApp message attempt resulted in delivery to the end user's handset or a failure. If the WhatsApp OTP failed because the recipient didn’t have a WhatsApp account, Verify automatically sent an SMS in under 3 seconds (P99). The end-user received the same code from WhatsApp or SMS to input into Duolingo’s existing UI.

If Duolingo made another Verify API call to send an OTP via SMS to the same mobile number (end-user) in the same verification session (ten-minute window), Verify sent the OTP via SMS and did not attempt a send via WhatsApp.

Experiment results

The A/B experiment ran for two weeks, where 50% of Duolingo’s verification volume used the feature, and 50% was unaffected. In that time, Duolingo saw improved OTP conversion rates and lower costs per OTP.

• Improved verification conversion rates:
• Out of 12 countries tested, nine countries showed higher success rates and lower costs when WhatsApp was used, compared to SMS-only

Conclusions

• We saw improved OTP conversion in some markets where WhatsApp is a preferred communication channel
• We saw decreased OTP costs by sending via WhatsApp instead of SMS in some markets
• We recommend implementing an omnichannel fallback system

Along with Duolingo, Twilio Verify has conducted similar experiments across other customers, countries, end-user platforms, and messaging channels, powering an algorithm to predict which channel is optimal for OTP delivery. Cross-channel fallback is also useful when one channel has an outage.

Twilio Verify offers omnichannel user verification across a growing selection of channels. Stay tuned for more from the Twilio Verify team on intelligently sending OTPs to your end-users on their preferred channels.

Learn more about Twilio’s Verify WhatsApp and contact Sales to enable the feature.