Customer Analytics Gone Wrong – Ten Common Mistakes To Avoid When Designing Customer Analytics Models

The potential that customer analytics models hold within them are extensive for the companies that choose to utilize them to better their marketing and sales activities. But even best-in-class companies get it wrong sometimes by making mistakes in how they design their models, and in how they utilize them once they have been designed. Herein we present ten mistakes to avoid in designing customer analytics models… 

From HSBC to Target, Vodafone to American Airlines, companies in various sectors the world over are tapping into the benefits of customer analytics. It is the rare enterprise these days that has not already set up a business intelligence unit within their corporate structure to support marketing and sales optimize their performance. In today’s ever-competitive business environment, companies are more and more looking to gain a competitive advantage through the utilization of various customer analytics models in their day to day practices. From knowing ahead of time which customer is likely to leave, to being able to know before-hand which specific product or service a customer is next likely to purchase, these models are having a direct and ever-growing impact on the bottom line of companies.

That said, the development and deployment of a customer analytics model can be difficult. An analysis we recently conducted of companies in Turkey has shown that over 65% of the models they have developed (in-house or had a consulting company develop for them) have not been deployed. Often behind this “failure to launch” are mistakes – either in how the models were designed or in how companies tried to deploy them, we have seen over and over a set of common mistakes made that has prevented the benefits the models hold within them from being tapped into. 

Generally speaking, there are two areas around which mistakes are made – the first, around designing models, the second, around deploying the models. In this article, we focus on those made around designing models. 

Ten Common Mistakes Made in Designing Customer Analytics Models 

Customer Segmentation Models

1. Likely the most commonly designed model for companies, it is also the least actionable. Generally speaking, this model, when completed, only places customers into macro-segments (usually around value, sometimes around behavior). The problem with such segmentation is that though needed, it does not drive a company to take tactical ongoing actions – rather, it helps companies redesign themselves around the customer segments and develop offerings for them.

The mistake made here is that segmentation is done only on a macro-level, and not also on a micro-level. Micro-level segmentation allows companies to take actions on all the little various behaviors of its customers; the outputs of a segmentation conducted in this manner are much more actionable, much more clear. For example, for a telecom, micro-level segmentation would show how each specific subscriber acts around paying, around recharging, around what time of the day they talk, what day of the week they use data the most, around whether they are inbound subscribers receiving calls most of the time or are call makers calling others most of the time. Looking at subscribers across a variety of micro-segments then allows companies to take actual actions to change behaviors (for example, shifting the traditional channel bill paying micro-segment to the online channel bill paying micro-segment).

The key learning here is that companies looking to conduct customer segmentation need not only make macro-level segmentation the focus – they also need to conduct micro-level segmentation.

Retention Models

2. The second most designed / utilized customer analytics model is also the one most incorrectly designed (particularly in the telecommunications and financial services industries). The single most commonly made mistake is around the valuation of customers – most models we have seen being used by companies value customers based on their current / recent value.

According to the defined value of the customer via the retention model, a relevantly lucrative offer is made to retain them – the model tags someone as low value with high churn risk, so a low value offer is made to retain them. The problem here is that by looking at only the recent value of the customer rather than their historically high value to the company, the wrong offer is being made. What the model should do is identify the value the customer once used to generate, and then recommend an offer that is of relevant value to the customer (so that the customer can return to their historically high levels). Accordingly, a currently low value but historically high value customer will be made a high value offer in hopes of not only retaining them but bringing them back to their historical highs (in terms of value generated).

The second aspect around customer valuation that is not reflected in retention models is the “social network” value of the customer. Though difficult to calculate, every customer has a social network value – meaning the additional revenues from the customer’s social network that will be lost should the customer leave. A simple example to make this point clear – when the head of a household indicates he wants to close his mobile line, it’s strongly likely that his wife, children, maybe even parents will follow in step. As such, this individual’s value is more than just the profits he directly generates; it’s also to some degree the profits those around him generate. Companies need to examine conducting social network analysis modeling to identify the relationships their customers have with each other, then update their retention models to reflect the true value each customers represents.

3. Another commonly made mistake around the design of retention models is that they are created in such a way as to allow for action taking only once the model has been re-run at the beginning of each month. While a model that predicts which customers will churn over the coming month is needed, also needed is one that identifies which customers will churn immediately following some type of issue – “churn triggers” is the term we use for this.

The point here is that while a majority of churn happens slowly (and thus can be predicted ahead of time, allowing for actions to be taken as soon as the model which predicts this churn is run each month), an important part of it happens immediately – a model that re-runs each month is not able to help a company prevent this type of churn. The telecom subscriber who gets a massive bill, the banking customer who has a major complaint that has taken forever to resolve, the resident who has had a power outage twice in the past month – each of these types of customers who go through such incidents may churn immediately.

As such, retention models need to be developed so as to have two outputs to them – one, the traditional monthly churn prediction output (predicting which customers will churn in the coming months based on recent behavior), two, the identification of incidents (churn triggers) which drive customers to close their accounts / cut off their business immediately. With the identification of the incidents that drive churn, companies can then take immediate action on customers who have a churn-triggering experience (i.e. as soon as a subscriber lodges a specific type of complaint within a certain amount of time following a specific type of transaction – which has been identified as a churn triggering incident of events, the company can immediately contact the customer and make the appropriate offer to ensure churn is addressed before it’s too late).

4. A third mistake made around this model is not taking seasonality into account, and using just one single retention model year round. The problem with using just one single model is that it is unable to separate the behavior of a customer who seems to be churning but in reality is not – rather, he or she is shifting into “vacation mode;” the decreased spending / erratic behavior the customer is displaying is in fact not the kind which precipitates churn, but rather is a normal one for this type of customer who acts different in different seasons.

Of particular importance in countries / markets where there is a significant migration to vacation homes or abroad (i.e. Dubai residents going abroad for the summer, USA East Coast elder population shifting to Florida for the winter), a second seasonal churn model needs to be utilized by companies in place of their traditional churn model during such seasons. With this model in place, no longer will non-churners who are migrating for a given period of time be labeled incorrectly as such; rather, the decrease in spending they exhibit can be correctly labeled as a seasonal one, with no action taken towards preventing churn. The seasonal model will essentially be able to look at two separate customers who exhibit the same kind of decreased spend but correctly be able to label one as churn likely, the other as not (due to identifying this individual as shifting into their seasonal pattern).

Channel Migration Models

5. A model that has recently been increasing in utilization, channel migration models allow companies to identify which customers to shift to which channels (retail, dealer, online, mobile, etc.). Traditionally the key benefit of deploying such models has been around reducing operational costs – as such, cost per transaction type per channel per customer is at the source of the model.

The most common mistake made in designing such a model is that it does not take into account the level of service or effectiveness in sales a given channel has. As such, the decision on which channel to shift a customer / transactions to is on a cost-only level (the lower cost channel the winner). In fact, the impact that a channel has in driving up sales, in retaining customers, is often more important than the cost of the channel; excluding such factors from the model essentially makes it useless.

An example to make this clearer – a channel migration model will recommend that a bank shift its loan applicants to conduct the application process via the bank’s website, rather than through the branch network. What the model does not take into account, however, is the added revenues generated through cross-sales in the branch during the application process. The lost revenues through the lost cross-sales opportunities may more than outweigh the reduction in costs, and as such, ultimately harm the bank more than benefit it. Around service-related transactions a similar issue may arise, in that receiving service through certain channels may drive up customer churn, also ultimately harming the bank more than benefiting it.

Next Best Activity Models

6. A rather popular model as of late, this allows for companies to identify which action should be next taken on a customer by customer level, from a marketing / sales perspective. Too often, however, this model is focused exclusively on revenue generating actions – “which product or service is the customer most likely to buy, and if they do so, what kind of revenues can I expect” as the objective, with a sales roadmap designed for every customer. While this is a good start, it neglects to take every type of action that can be taken by a company on a customer level into consideration – selling products and services is just one category.

A properly designed model would take into consideration all possible activities, and then develop a roadmap for a given customer – this could include:

  • Migrating a customer to an online channel
  • Collecting personal information from the customer (missing data records)
  • Seeking permission from the customer (to allow for contacts through SMS, for example)
  • Seeking a referral from the customer
  • Leaving the customer alone (taking no action at all this month)

What is important here is that all the types of actions that can be taken on a given customer be considered, assessed from a revenue / retention / satisfaction impact perspective, and be put into the mix of actions that should be analyzed for determining the proper action roadmap for the customer. Sometimes reducing costs, increasing satisfaction, etc., may take priority over driving up revenues.

Across the Board Mistakes

7. A mistake that touches on all customer analytics models, this is around not getting proper insight and input from business units / end users. No model in and of itself is an out of the box solution; rather, well designed models require extensive input from internal stakeholders. Designing such models independently within the business intelligence unit is a recipe for failure, as no model designer can have an understanding of the business the way end users do.

As such, it is critical that marketing / sales / customer care employees (based on the model the stakeholders will change) from various levels be consulted as to:

  • Their needs from such models
  • Limitations that should be factored in to the models
  • Definitions around core concepts for use in the models

The input such stakeholders will provide will be critical in the proper design of the models, so that there is minimal resistance once they are deployed; having disagreements on, for example, the definition of churn, the threshold to be used around revenue for a high value customer, what type of activities should be considered when designing a next best activity model, etc., are all factors which can undermine the designed models.

8. Often neglected by analytical models are new customers; in so doing, companies fail in tapping into a great deal of potential, particularly around growing them out of the gate, and in driving down churn – new clients traditionally are more prone to churn than average clients, and also are more receptive to offers. A recent study we conducted at one company we engage with found that the churn rate of their clients with less than six months tenure was a full 40% higher than the rest of their client base.

In designing models, a traditionally relied upon approach is to exclude new clients (usually defined as those with the company less than 3 or 6 months), as the belief is that their behaviors are not set yet, and, they behave differently than the mainstream client base for a period of time. While this approach is the correct one, it does not forego the fact that such clients need to be acted on as well, to capitalize on opportunities they present. As such, companies need to develop newcomer models, and run these in parallel to their mainstream models. The methodology used in preparing these models will, of course, be somewhat different than that used in preparing mainstream models – for example, the factors that trigger their churn will likely be different than those that trigger mainstream client churn. We recommend companies pay particular attention to replicating their retention models for new clients, and, if possible, cross and up-sell related models.

9. Also touching all analytics models, this mistake is in regards to refreshing them; moreover, the failure to do so – few is the company that regularly revisits their analytical models to see if they need to be re-designed. Analytical models are no different than a company’s strategies, its products, it services – they require revisits and revisions as market conditions change, as competitors come and go, as consumers begin behaving differently.

Over time, analytical models will begin to become less effective in predicting / assessing accurately; at the core of these models are a variety of assumptions. As the environment within which a company operates evolves over time, these assumptions can become null; for example, the factors which precipitated churn several years ago may not be the ones precipitating churn today. Additionally, a rapidly changing consumer portfolio may make models inaccurate as well; for example, the consumer segments a company had several years ago may not be the ones they have today.

Within an ever-changing and evolving business environment, companies need to revisit and possibly re-design their customer analytics models every two to three years, to ensure they are as accurate and effective as the day they were originally designed. In a sense, such a revisit is a check-up, to ensure all the bells and whistles are working properly.

10. Finally, one additional mistake we see often made around the design of models is related to their testing; in particular, around examining their stability over time. What we mean here is that a model needs to appear stable for it to gain traction within the company, stable around its outputs. A segmentation model, for example, that states 20% of a company’s customers are in Segment A in the month of January, then the following month when re-run sees 45% of the company’s customers in Segment A is bound for failure, as end users will have no trust for it. Month over month there is expected to be change – in the number of potential churners, in the number of customers who are highly likely to purchase a given product, in the number of people belonging to a certain segment. But this change should be subtle, not drastic.

To ensure such a situation is avoided, any and all models should be tested before they are deployed; by going back one year’s time and rerunning the model backwards, the integrity and stability of the models can be tested. If drastic swings are observed, then the models’ parameters / rules will need to be revisited. If there appears to be a per-month maximum of 10% shift in the size of a given segment, the size of the highly likely to churn customer group, then the models can be safely deployed.

We believe that companies need to utilize customer analytics model extensively, to get the most out of their customer base. That said, we also recommend they be careful to avoid the above listed mistakes when designing them.

Tags : analytics, business intelligence, channel migration, churn, consulting, customer analytics, customer segmentation, data mining, fact-based, marketing, next best activity, retention