How to Measure Chatbot ROI: The Metrics That Actually Matter

Most chatbot reports are full of numbers that feel good and mean little. "4,000 messages handled this month" sounds impressive until you ask how many of those turned into an order, or how many customers left because the answer was wrong.

If you are paying for a chatbot, you want to know one thing: is it earning more than it costs? This guide walks through how to answer that honestly, with numbers you can actually collect.

Start with a baseline, or you are guessing

Before you switch anything on, write down how things work today. Two weeks of rough numbers is enough:

• How many messages arrive per day, across every channel
• How long a customer usually waits for the first reply, during the day and after hours
• How many orders or bookings came from DMs and WhatsApp
• How many hours your team spends in the inbox each day

You don't need software for this. A notebook or a shared spreadsheet works. Say you run a clothing page in Kathmandu. Spend a week noting the time each DM arrived and when someone replied. You will probably find that daytime replies are fine and evening ones wait until the next morning. That gap is exactly where a bot earns its keep, and without the baseline you can't prove it later.

The metrics that matter

Time to first reply

This is the easiest number to move and the one customers notice first. A bot should bring it down to seconds at any hour. Track it separately for business hours and after hours, because the after-hours number is usually where the big change shows up. We wrote more on how fast you need to reply.

Resolution rate

The share of conversations the bot fully handled without a person. Be strict about "fully". If the customer asked the price, got it, and then ordered, that counts. If they asked three times and gave up, that doesn't.

Don't chase a very high number. Some conversations should go to a person, and a bot that refuses to hand over will look good on this metric while quietly annoying people.

Leads and orders captured in chat

This is the number that pays for the tool. Count names and phone numbers collected, orders placed, bookings made or demo requests submitted through chat. If your bot collects order details, tag those conversations so you can count them at month end.

Handoff quality

When the bot passes a chat to your team, does the agent have what they need? Read a sample of handoffs each week. If staff keep asking customers to repeat themselves, the handoff is broken. Our guide on chatbot human handoff covers what a good one looks like.

Wrong answers

Nobody puts this on a dashboard, but it is the metric that protects your reputation. Read a random set of bot conversations every week and mark any reply that was wrong or misleading. The target is close to zero, and every mistake should lead to a fix in your knowledge base.

Metrics to ignore, or at least not celebrate

• Total messages sent by the bot. A chatty bot sends more messages. That is not success.
• Conversations started. Useful for volume planning, meaningless as a result.
• Average conversation length. Longer can mean engaged, or it can mean confused. On its own it tells you nothing.
• Button click rates in isolation. Fine for tuning a flow, but they don't tell you whether anyone bought.

Working out the actual return

Keep the maths simple. You need three things.

1. What it costs. The setup fee spread over a year, plus the monthly fee. With Orbit, for example, you pay a one-time implementation fee and then a monthly plan based on unique customers. The pricing page has the current numbers.
2. What it saves. Hours of inbox work your team no longer does, multiplied by what those hours cost you. Be realistic. If staff still check the inbox, the saving is smaller than "one full salary".
3. What it earns. Orders or leads from chat that you would likely have lost. The fairest way to estimate this is to look at after-hours and weekend conversations that converted, since those are the ones that previously waited until morning.

Say you run a trekking agency in Thamel. Before the bot, enquiries from Europe arrived overnight and got answered at 9am Nepal time, by which point some travellers had already booked elsewhere. After the bot, those enquiries get an instant reply with trip dates and prices, and a lead is captured. If even a few of those turn into bookings in a month, compare that margin to the monthly fee. That comparison is your ROI, and it is much more convincing than a chart of messages.

Common mistakes when measuring

• Judging too early. The first two or three weeks are mostly about finding gaps. Measure the settled bot, not the first draft.
• Not tagging chat-sourced sales. If your team takes the order on the phone after a chat, the chat never gets credit. Ask staff to mark the source.
• Comparing different seasons. Dashain or a big sale week will inflate everything. Compare like with like.
• Ignoring one channel. If WhatsApp and Instagram are measured separately in different places, you will miss the full picture. A unified inbox makes this much easier because everything is in one report.

A simple monthly review

Once a month, spend thirty minutes on this:

1. Check reply time, resolution rate and chat-sourced orders against last month.
2. Read twenty random bot conversations and twenty handoffs.
3. List the questions the bot couldn't answer, and add those answers.
4. Decide one thing to change next month, such as a new keyword trigger or a clearer order flow.

That routine is what makes the numbers improve. A chatbot is not a one-time install. It gets better as you feed it what customers actually ask.

If you would like to see what these numbers could look like for your own page, book a demo. We'll look at your current inbox with you and set Orbit up so the metrics above are easy to track from the start.