How to Tell Whether Your AI Connection Is Actually Working

Written by

in

Every vendor in this space promises something measurable, and we are no exception: the point of this work is a measurable channel, not a science project.

Far fewer explain what actually gets measured. So here is the specific version: what to count, what to ignore, and what the first ninety days genuinely look like.

The step almost everyone skips

Measure before you build.

It sounds obvious. It is skipped constantly, and it is the reason so many businesses end up unable to say whether something worked.

If you do not know how many enquiries you received last month, how many converted, and how many arrived outside working hours, then in three months you will have a number with nothing to compare it to. You will be left arguing from impressions, and impressions favour whoever is most enthusiastic in the room.

Establishing that baseline is part of the audit, before anything is built. It is usually the single most valuable half-hour of the whole engagement, and it costs nothing.

The numbers that actually matter

They depend on which connection you started with. Pick the small set that maps to your constraint, and ignore the rest.

If you connected enquiries: enquiries captured outside business hours · how many arrive already qualified · time from enquiry to first response · how many convert compared with your baseline.

If you connected bookings: bookings completed without a person involved · reduction in back-and-forth before a booking is confirmed · fill rate on slots that historically went empty · no-shows, which sometimes shift once confirmations become automatic.

If you connected product discovery: conversion rate for guided sessions versus ordinary browsing · average order value · how often customers land on products they would not have found by searching · where people abandon.

If you connected support: questions resolved without a person · your team’s time returned, in hours · what proportion escalate, and whether that proportion is falling as the rules improve.

What to ignore

Conversation volume. A big number that says nothing about whether anything was achieved.

“Engagement”. If it cannot be traced to an enquiry, a booking, a sale or an hour saved, it is not a business result.

Response speed on its own. Fast wrong answers are worse than slow right ones. Speed only counts alongside resolution.

Any dashboard that leads with these is a dashboard designed to look good rather than to inform a decision.

The honest problem: attribution

Some of the value is genuinely hard to attribute.

If a customer asks an assistant about you, gets a useful answer, and books three weeks later by phone, that booking will not appear in any AI report. It was still caused by the connection.

We would rather say that plainly than present precision we cannot support. The practical approach is to track what is directly measurable, watch your overall numbers move alongside it, and treat the gap as upside rather than pretending to have measured it.

Be sceptical of anyone claiming to attribute all of it. They are either measuring something narrow or being creative.

What the first ninety days actually look like

Days 1 to 30: small and noisy. Real volume is low. Some rules turn out too tight, some too loose. Interesting rather than conclusive, and this is where most of the tuning happens. Judging success here is like judging a shop on its opening morning.

Days 31 to 60: the shape appears. Enough volume for patterns. You will usually see one thing working better than expected and one worse. This is where the connection earns its keep, because now you are adjusting from evidence.

Days 61 to 90: a number you can trust. Comparable to your baseline, stable enough to decide from: widen the connection, add a second one, or stop.

Anyone promising a clear return in week two is selling you week two.

What “not working” looks like, and what we do

Sometimes the honest answer is that it is not delivering. That usually shows up as one of:

  • Volume is there, but escalations stay high: the rules are too narrow, or the underlying

information is not good enough to answer from.

  • Volume never materialises: you connected a journey your customers do not actually use.
  • It works, but the value is smaller than the effort: real, and worth saying out loud.

The first is fixable and normal. The second means we chose the wrong connection, and the fix is to move to the right one. The third means you should not extend it, and we will tell you so.

A connection that quietly underperforms while nobody looks is the worst outcome available. That is what the review rhythm exists to prevent.

What you get from us

A monthly report in your numbers, not conversation counts, but enquiries, bookings, resolutions and hours. What changed, what we adjusted, and what we recommend next.

Reviews at 30, 60 and 90 days, then quarterly. Rules reviewed on a set cadence, because the ones that were right at launch rarely stay right.

Where to begin

The agent-readiness audit establishes your baseline before anything is built, which is what makes any of this measurable later.

You get a written report including the payback maths in your own figures: what a customer is worth to you, how many you are losing, and how many the connection would need to recover to cover itself. If those numbers do not work, the report will say so.

Free worksheet: Your first three WebMCP tools (PDF). Pick the three jobs on your website to make ready for AI agents first. No email needed.

WebMCP news, without the jargon

One email a month: what changed in WebMCP, what it means for your website and your clients, and one practical step. For business owners, agencies and developers in Africa.

WebMCP Africa, PO Box 35305, Somerset West 7130, South Africa. We never sell your details. Privacy policy

Comments

One response to “How to Tell Whether Your AI Connection Is Actually Working”

  1. […] convert. Without this, nobody can tell you in three months whether anything improved. Our post on how to tell whether your AI connection is working explains why this step matters so […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Share with