PingPilot

The 2016 patent promised its rating bonuses would share customers evenly across staff; years later, a pre-registered simulation of those rules failed the promise in 71 of 81 situations.

One feature, start to finish The routing claim in the patent

  • Research research not shown here
  • Design design not shown here
  • Measurement measured what happened

Measure only. The rules were written into the patent, not drawn as a screen, and no research tested them. The simulation tests the patent's words, not real calls.

Read the full study →

Context

Why did this work matter?

  1. C
  2. A
  3. R
  4. D
  5. I
  6. O

PingPilot let customers reach a business from wherever they were, and reach the same person each time. As Chief Product Officer, I worked out the call routing with Brandon Brumage. It became US patent 10,868,915, filed November 9, 2016.

Assumptions

What did we believe going in?

  1. C
  2. A
  3. R
  4. D
  5. I
  6. O

The system picks a staff member by rating, plus two fixed bonuses. One patent line says those bonuses will "provide load balancing amongst the agents". We wrote that before we had proof.

Figure 3 from US patent 10,868,915: a flowchart headed Search Signal Agent Pool. It branches on whether any agent is online, removes agents the customer has banned, compares feedback scores to pick the highest rated, then breaks ties on the number of customers an agent already has before assigning the customer to an agent.
Figure 3 of the patent: how it picks a staff member. This is what the simulation tested.

Intervention

What changed because of it?

  1. C
  2. A
  3. R
  4. D
  5. I
  6. O

Years later I simulated 81 situations, varying how close staff ratings sat, team size and customer volume, 200 runs each. I fixed the pass mark first. As one of three named inventors, I was testing against my own interest.

Outcome

What was the impact?

  1. C
  2. A
  3. R
  4. D
  5. I
  6. O

The promise failed in 71 of 81 situations, passing only when ratings sat close together. Sending a returning customer back to the same person did hold up. The even-sharing promise is what I would write differently.

Campaign Team sheet: seven sales staff, each a toggle and a rating. Arthur Frazier 4.98 switched on, Wallace Evans 4.94 on, Victoria Duncan 4.99 off, Calvin Howard 4.98 off. The screens are from my later app design, which doesn't use the patented routing; the names are made up.
Everything a manager can change sits on one sheet: a switch and one score per person. The top scorer is switched off, so the next customer goes to the best person online.

Also on this project

Read the full study →

  • 6

    ways to reach the team from one link: video, chat, text, call, email and booking a time.

  • 42

    campaign links still saved in the Internet Archive, across 12 Signals and all six ways in.

  • HMU

    The mobile app and responsive web app, rebranded in 2020.

enesru