Skip to content
← All articles

Measure the finished job.

Replies, clicks, and activity tell part of the story. Measure what changed for the customer and the team.

A billing reminder went out. The customer opened it. The account is still overdue. All three facts can be true. When you measure AI at work, the last fact matters as much as the first two.

Separate activity from the business result.

For an illustrative utility billing journey, track reminders delivered, customers who requested help, payment arrangements recorded, and payments posted. These events answer different questions. A click is useful engagement information; it is not money collected.

Choose the source of truth for each measure. The messaging provider can confirm delivery. The billing system confirms a posted payment. When those sources disagree or a result is still pending, keep that uncertainty visible.

Keep the denominator honest.

If you report a completion rate, define which requests count. Show eligible requests, attempted requests, and completed requests separately. Otherwise, excluding difficult cases can make a weak process look healthy.

Record how many jobs needed employee help and why. An appropriate handoff is often the right outcome, especially when someone disputes a charge or needs an exception. Measure whether the handoff reached an owner with enough information to act.

Look for work that came back.

A closed conversation can reopen because the underlying request was never resolved. Review repeat contacts and corrected records alongside initial completion. Use a review window that suits the job: a booking change and a monthly billing cycle have different timelines.

For a DTC order, check whether the requested change appeared in the order system. For a hotel, check whether the guest’s request reached the responsible team and was fulfilled. For a property repair, distinguish scheduled, attended, and resolved.

Put time saved next to time spent.

Compare the employee effort before and after introducing the workflow. Include reviewing drafts, answering exceptions, and maintaining the setup. Use a comparable sample of work, rather than multiplying a best-case demonstration by every request the company receives.

Then decide what that extra capacity is for: faster follow-ups, more account coverage, or fewer backlogs. Finn’s operating history and business records can support that review, but the outcome belongs to your business. Set the baseline, agree on the measure, and review it with the people doing the work.