Deflection rate is the easiest metric to game and the least useful one alone

A chatbot can "deflect" a high share of conversations simply by answering vaguely or closing the thread early - that looks good on a dashboard and does nothing for the business. The number worth tracking is resolution rate: the share of conversations that ended with the customer's actual problem solved, not just a reply sent.

For a US SMB, a realistic first-quarter resolution rate on a well-scoped support chatbot sits well below the marketing claims of 90%+ - somewhere in the 40-60% range for genuinely varied queries is a solid start, with the rest routed cleanly to a human rather than looped.

If a chatbot vendor's only stated metric is deflection rate, ask for the resolution rate and the escalation-quality rate next - what happens to the conversations it didn't handle matters as much as the ones it did.

The metrics that actually predict payback

Cost per resolved query - support labor cost saved divided by queries the bot genuinely closed - is the number that ties directly to payback period. Lead capture rate matters just as much for a business-development use case: a chatbot on a marketing site should be measured by qualified leads captured, not conversations started.

Response latency and containment on the first message also matter more than most SMBs realize - a slow or looping bot pushes customers to abandon the chat before it gets the chance to help at all, which shows up as a silent loss nowhere in the deflection number.

Track cost per resolved query

Support labor saved divided by queries genuinely closed - this is the number that ties to actual payback, not deflection percentage.

Track qualified leads captured

For a marketing-site chatbot, leads captured and qualified matters more than conversation volume.

Set the baseline before you build, not after

The single biggest reason US SMBs can't prove chatbot ROI six months in is that they never measured the baseline - average support cost per ticket, average time-to-first-response, current lead conversion rate - before deployment. Without that number, every post-launch metric is a guess at improvement rather than a measured one.

Scoping a pilot against your own historical support or lead data, before a single line of the bot's logic is written, is the difference between a defensible ROI case and an anecdote.

Want a chatbot ROI baseline scoped against your own support data before you commit to building? Talk to us about a pilot.

Key takeaways

  • Resolution rate, not deflection rate, is the metric that reflects whether the chatbot actually helped - track what happened to the conversations it didn't fully resolve.
  • Cost per resolved query ties directly to payback period; qualified leads captured matters more than conversation volume for a business-development bot.
  • Measure your baseline - current support cost per ticket, current lead conversion - before deployment, or you'll have no way to prove the improvement afterward.
  • A realistic first-quarter resolution rate for varied queries is well below vendor marketing claims; scope your pilot expectations accordingly.