Machine Learning

Using phone in a digital world. A Data Science story.

Contributors: Vladimir Iglovikov, Sophie Benbenek, and Richard Yeung

It is Wednesday afternoon and the Data Science team at TrueAccord is arguing vociferously. The white board is covered in unintelligible hand writing and fancy looking diagrams. We’re in the middle of a heated debate about something the collections industry has had a fairly developed playbook on for decades: how to use the phone for collections.

Why are we so passionately discussing something so basic? As it turns out, phone is a deceptively deep topic when you are re-inventing recoveries and placing phone in the context of a multi-channel strategy.


Solving Attribution of Impact

The complexity of phone within a multi-channel strategy is revealed when you ask a simple question: “What was the impact of this phone call to Bob?”

In a world with only one channel, this question is easy. We call a thousand people and measure what percentage of them pay. But in a multi-channel setting where these people are also getting emails, SMS and letters, there is an attribution problem. If Bob pays after the phone call, we do not know if he would have paid without the phone call.

To complicate matters further, our experiments have shown that phone has two components of impact:

  1. The direct effect — the payments that happen on the call.
  2. The halo effect — the remaining impact of phone; for example seeing a missed call from us and going back to an email from us to click and pay.

To solve the attribution problem and capture both components of impact, we define the concept of incremental benefit as:


Intuitively, the incremental benefit of a phone call is the additional expected value from that customer due to the phone call. For example, assume Bob has a 5% chance of paying his $100 debt. If we know that by calling him, the probability of him paying increases to 7%, then the incremental benefit is $2 (100 * (0.07 – 0.05)).


How we calculate incremental benefit

Consider the incremental benefit equation in the last section. It requires us to predict the probability of Bob paying for each scenario where we call him and do not call him.

Hence we created models that predict the probability of a customer paying. These models take as inputs everything we know about the customer, including:

  • Debt features: debt amount, days since charge-off, client, prior agencies worked, etc
  • Behavioral features: entire email history, entire pageview history, interactions with agents, phone history, etc
  • Temporal features: time of the day, day of the week, day of the month, etc

The output of the model is the probability of payment by the customer given all of this information. We then have the same model output two predictions: probability of payment with the current event history, and probability of payment if we add one more outbound phone call to the event history.

Back to our example of Bob, the model would output the probabilities of 7% and 5% chance of paying with and without an additional phone call respectively.

This diagram is a simplification that omits many variables and the actual architecture of our models


Optimal Call Allocation

The last step of the problem is choosing who to call, and when. The topic of timing optimization deserves its own write-up, so we will close with discussing who we call.

Without loss of generality, assume that we would only ever call a customer once. The diagram below has the percentage of customers called on the x-axis. And the y-axis is in dollars with 2 curves:

  • Incremental Benefit — this curve shows the marginal incremental benefit of calling the customer with the next highest IB
  • Avg cost — this horizontal curve shows the average cost of an outbound call


There are two very interesting points to discuss:

  • Profit max — calling everyone to the left of the intersection of incremental benefit and avg cost is the allocation that maximizes profit. Every one of these calls brings in more revenue than cost.
  • Conversion max — notice that incremental benefit dips below zero. This is especially true when you remove the assumption that we only call each customer once. The point that maximizes conversion for the client is to call everyone to the left of where incremental benefit intersects with zero.

Our default strategy is to call all customers to the left of the profit maximizing intercept. Interestingly, an intuitive investigation of the types of customers selected reveals customers at two extremes: we end up calling both very high value customers that have shown a lot of intent to pay (e.g. dropped off from signup after selecting a payment plan) and customers where email has been ineffectual (e.g. keeps opening emails with no clicks or no email opens.)



The world has become increasingly digital, and a multi-channel strategy is the right response. Bringing the traditional tool of phone, as just one channel within this strategy, forced us to rethink a lot of assumptions and see where the problem led us. We began by replacing the traditional “propensity to pay” phone metric with incremental benefit, found ways to predict this value, and implemented a phone allocation strategy that maximizes profits for the business.

Come Work With Us!

We combine a social mission, huge market, a proven team and early traction + revenue. We're early enough for you to have a huge influence on the company's future (and upside) but late enough so we already reached first funding and product market fit - the perfect timing to join. If you want to learn a lot while your actions have a positive impact on the world - come work with us.

Stay up to date! Subscribe for updates on TrueAccord

SF Scala: Nadav Samet, ScalaPB: Working with Protocol Buffers in Scala