The question their data couldn't answer: which provider?

Alex Barnett

CEO

Tools

A growing online marketplace came to us with a simple request. They have more than a thousand independent providers on their platform, and leadership wanted to know which ones were generating the most customer complaints. They wanted names, so they could coach the few who were struggling and part ways with the ones who couldn't be.

It's the exact kind of question you assume your support data can answer. Theirs couldn't. And the reason is a trap a lot of support and ops teams are sitting in without knowing it.


The blind spot in their help desk

Their support system records the person who submits the ticket. When a customer writes in to complain about a provider, it captures the customer's name, the customer's account, the customer's words. It never captures the provider, because the provider isn't the one writing in. The customer is.

So "which provider" was not a field anywhere in their system. No column to group by, no filter to apply, no report to run. The one thing leadership most needed to see wasn't in the database at all. On the surface, it looked like they didn't just lack a report. They lacked the data to build one.


The answer was sitting in the text

It wasn't missing. It was buried in the words of every ticket. When a customer is angry at a provider, they name them, just never cleanly. A nickname, a misspelling, a screen name, a half-remembered handle.

The Signal Engine read every ticket, pulled those names out of the prose, and matched each one back to the company's roster of providers.

That single step rebuilt the dimension their system never stored. Complaints could finally be grouped by provider, because the provider's identity had been recovered from what the customer actually wrote.


Why ranking by complaint count would have backfired

Once you can group tickets by provider, the instinct is to rank everyone by total complaints and go after the top of the list. For this marketplace, that ranking would have pointed management straight at the wrong people.

A provider only shows up in this data when an upset customer names them. So a raw count mostly measures how much business a provider does and how exposed they are, not how badly they perform. On top of that, each provider had only a handful of tickets to be judged on, somewhere between 9 and 18. That's far too few to call anyone your worst on the numbers alone.

So the Signal Engine didn't stop at the raw count. It weighed each provider against how little data there was, and against the fact that they only landed in this data when a customer was already upset. The picture changed completely:


Ranking by raw complaint count

After accounting for volume and tiny samples

Who looks like a problem

15 providers flagged

3 genuinely elevated

What you'd do about it

Coach or suspend 15 people

Targeted fix for 2 providers, plus 1 product bug


The part the raw numbers hid

One of those three wasn't a provider problem at all. It was a product bug. The platform's chat kept freezing mid-session. Customers lived that as the provider going silent, and wrote it up in their tickets as a bad provider. The complaint was real. The cause belonged to engineering, not the contractor.

For the two real cases, the text showed the exact behavior driving the complaints. One provider cut sessions short over and over, the same move in 13 of their tickets. That's the difference between vague and actionable. "This person generates complaints" tells you there's a problem. "Here's exactly what they do, and how often" tells you how to fix it.


The dimension your help desk forgets

Step back from the marketplace, because the lesson isn't about providers. The thing you most need to measure is usually the one your help desk never stored as a field. Which feature a complaint is about. Which policy. Which person. What the customer was trying to do when it broke. None of those are tidy checkboxes in your ticketing tool. Almost all of them are sitting in the words your customers already wrote.

If it's in the text, it can be measured. This marketplace was sure they needed to start collecting new data. They didn't. They needed someone to read the data they already had. The questions they'd written off as unanswerable were answerable the whole time.


See what your tickets can answer

This is what the Signal Engine is built for. Point it at your support data and it reads the tickets you already have, recovers the dimensions your help desk never stored as fields (which feature, which policy, which person, what the customer was trying to do), and ranks them by what's actually driving complaints.

If there's a question about your customers you've been told your system can't answer, it's worth checking whether the answer is already sitting in the text. [Book a 20-minute walkthrough] and the Signal Engine will read a slice of your real tickets and show you what's in there.

Share on social media