Around 2 years ago, our sales team was given a daunting task by our CEO – win an incremental 15 points of share in a market segment where we have less than 25% share today. The task seemed even more insurmountable given the fact that this customer segment had an entrenched competitor who had more than 50% share. As the strategy professional for the team, I was tasked with building the approach that the sales team could execute to achieve this goal. And straight to work I went – I built an end to end go-to-market framework which had three pillars: 1) targeting the right customers, 2) playbooks to win those customers and 3) revenue optimizers to harness the maximum potential from these customers. A key part of our strategy was targeting the most valuable customers. Our frontline sales team was facing significant challenges with poor conversion rates which in turn significantly reduced their efficiency metrics. We had to fix that – and as someone who has always believed that math models were better than humans at decision making, I turned to analytics to solve the problem. We worked for nearly 3 months building and refining a model to prioritize the customers that our frontline sales should spend most of their time with. The model on the surface is seemingly simple: it works through a mountain of data to identify potential targets based on two dimensions – 1) high likelihood of us winning the customer and 2) high revenue potential if we win the customer. Under the hood were various regressions, time series forecasting and decision trees that worked through hundreds of variables – from both proprietary and public data sets – to come up with these recommendations. These prioritized customers were then communicated to our frontline through our CRM tool. The model worked swimmingly well. In the first three months of implementation, our conversion rate went up threefold. We were looking at winning 3 pts of share in the first year, an unprecedented pace of share gain. However, when the share data came at the end of the first quarter the tool was deployed, the numbers did not look rosy at all. Instead of a share gain, it showed that our market share had remained relatively flat. Given that we were winning customers at an exceptionally fast pace, these numbers did not make sense. We started dissecting the share-data and came to a pretty uncomfortable realization – while we were winning customers on one end, we were losing customers at nearly the same rate on the other end. Our competitor had recognized our recent push in the market and was poaching our existing customers. On top of it, our frontline sales team, too busy with winning new customers, were unable to invest the required time and effort to retain our current customers. To address the issue, I expanded our go-to-market strategy to include a fourth pillar - mechanisms to retain existing customers. We stood up a dedicated team exclusively for managing existing customers. This team’s responsibility was neither hunting for new customers, nor farming to increase revenue with existing customers but simply do what is required to retain existing customers. While our churn rate did drop, the difference was not substantial. The key challenge we faced was that our retention team had to manage tens of thousands of customers and they had no way to identify which ones deserved attention, and which ones did not. That prompted me to ask two questions ·       Which customers deserve special attention from the retention team? ·       How do we identify and prioritize them? Back to analytics I turned. We built a model that went through customer history and publicly available information and predicted the customer’s ‘likelihood of churn’. The model’s prediction ability was spectacularly impressive – it was able to tell us a customer is likely to churn within 90 days with 50-70% accuracy. We were elated - now we had a tool that could predict the future, and tell us exactly where we should concentrate our efforts. But soon we realized that good model prediction does not translate to good outcomes. Despite deploying the model to the retention team and the frontline sales team, the churn rate remained flat. This was a perplexing outcome, and I had to get to the bottom of it. I interviewed each member of the retention team. I went to the field and spent hours with the sales team to understand why they couldn’t stem the flow even when they were warned of it. These interactions helped me understand the problem: You can tell the team that they are going to lose a customer. But then what? They did not have any actionable insights. What we were doing was giving the team some metrics. E.g., the model outcome told them that there is a 70% probability of a customer because their volume has reduced 30% in the last 5 months. That did not help them take any mitigation measures, only told them the problem. As the model was not proving effective in the field, the sales team was losing confidence in it. This significantly reduced model adoption as well. After the first two months of deployment, the model adoption rate was an abysmal 20%. You can tell the team that they are going to lose a customer. But then what? We had to fix this. The solution was to not just communicate to the retention team and the frontline just the symptoms, they needed remedial actions. But doing this using human intervention was cumbersome and expensive. For each customer, we had troves of data. If a customer was flagged as likely to churn, going through the data to find the root cause and then recommending a solution required automation. I turned to AI to do that. AI had two significant advantages – 1) It could go through large amounts of data to come up with insights much faster than humans and 2) It makes the feedback loop between field outcomes and model refinement seamless. After nearly two months of training, our AI model was ready for deployment. It was extremely good at combining model output and customer data to provide actionable advice. Here is a real example (edited to maintain confidentiality) of how AI insights were different from raw model outputs Original model output communicated to field Analysis Report: Risk ID #8842 - Account Attrition Modeling Analysis Report: Risk ID #8842 - Account Attrition Modeling Churn Probability Score: 72.4% (High Priority) Churn Probability Score: 72.4% (High Priority) Churn Probability Score: 72.4% (High Priority) Key Data Insights: Key Data Insights: Volume Deceleration: Time-series analysis identifies a 30% reduction in procurement volume over a 5-month rolling window. This trend deviates significantly from the account’s 3-year seasonal baseline.
Market Share Shift: Competitive intelligence and transactional metadata indicate the recent onboarding of [Competitor Name] SKUs. Cross-referencing logistics data suggests the competitor gained a 15% shelf-share within the last 60 days. Volume Deceleration: Time-series analysis identifies a 30% reduction in procurement volume over a 5-month rolling window. This trend deviates significantly from the account’s 3-year seasonal baseline. Volume Deceleration: Time-series analysis identifies a 30% reduction in procurement volume over a 5-month rolling window. This trend deviates significantly from the account’s 3-year seasonal baseline. Market Share Shift: Competitive intelligence and transactional metadata indicate the recent onboarding of [Competitor Name] SKUs. Cross-referencing logistics data suggests the competitor gained a 15% shelf-share within the last 60 days. Market Share Shift: Competitive intelligence and transactional metadata indicate the recent onboarding of [Competitor Name] SKUs. Cross-referencing logistics data suggests the competitor gained a 15% shelf-share within the last 60 days. Additional recommended actions that were recommended using AI Intervention Strategy

Tactical Target

Financial Framework



Defensive Pricing (Tier 1)

Reclaim 12% of lost volume

Apply a 4.5% rebate on all orders exceeding $xx per month for the next 90 days.



Inventory Saturation(Tier 1)

Achieve 85% "Wallet Share"

Offer a "Buy 10, Get 1" pallet incentive on SKUs <redacted> and limit [competitor] shelf space.



Competitor Displacement(Tier 2)

Displacement of [Competitor]

Authorize a one-time $xx "Conversion Credit" specifically tied to the phased removal of competitor-equivalent products.



Contractual Lock-in(Tier 2)

24-Month Extension

Proposed xx% fixed-price ceiling in exchange for an exclusive 2-year supply agreement (MSA). Intervention Strategy

Tactical Target

Financial Framework



Defensive Pricing (Tier 1)

Reclaim 12% of lost volume

Apply a 4.5% rebate on all orders exceeding $xx per month for the next 90 days.



Inventory Saturation(Tier 1)

Achieve 85% "Wallet Share"

Offer a "Buy 10, Get 1" pallet incentive on SKUs <redacted> and limit [competitor] shelf space.



Competitor Displacement(Tier 2)

Displacement of [Competitor]

Authorize a one-time $xx "Conversion Credit" specifically tied to the phased removal of competitor-equivalent products.



Contractual Lock-in(Tier 2)

24-Month Extension

Proposed xx% fixed-price ceiling in exchange for an exclusive 2-year supply agreement (MSA). Intervention Strategy

Tactical Target

Financial Framework Intervention Strategy Intervention Strategy Intervention Strategy Tactical Target Tactical Target Tactical Target Financial Framework Financial Framework Financial Framework Defensive Pricing (Tier 1)

Reclaim 12% of lost volume

Apply a 4.5% rebate on all orders exceeding $xx per month for the next 90 days. Defensive Pricing (Tier 1) Defensive Pricing (Tier 1) Defensive Pricing (Tier 1) Reclaim 12% of lost volume Reclaim 12% of lost volume Reclaim 12% of lost volume Apply a 4.5% rebate on all orders exceeding $xx per month for the next 90 days. Apply a 4.5% rebate on all orders exceeding $xx per month for the next 90 days. Apply a 4.5% rebate on all orders exceeding $xx per month for the next 90 days. Inventory Saturation(Tier 1)

Achieve 85% "Wallet Share"

Offer a "Buy 10, Get 1" pallet incentive on SKUs <redacted> and limit [competitor] shelf space. Inventory Saturation(Tier 1) Inventory Saturation(Tier 1) Inventory Saturation(Tier 1) Achieve 85% "Wallet Share" Achieve 85% "Wallet Share" Achieve 85% "Wallet Share" Offer a "Buy 10, Get 1" pallet incentive on SKUs <redacted> and limit [competitor] shelf space. Offer a "Buy 10, Get 1" pallet incentive on SKUs <redacted> and limit [competitor] shelf space. Offer a "Buy 10, Get 1" pallet incentive on SKUs <redacted> and limit [competitor] shelf space. Competitor Displacement(Tier 2)

Displacement of [Competitor]

Authorize a one-time $xx "Conversion Credit" specifically tied to the phased removal of competitor-equivalent products. Competitor Displacement(Tier 2) Competitor Displacement(Tier 2) Competitor Displacement(Tier 2) Displacement of [Competitor] Displacement of [Competitor] Displacement of [Competitor] Authorize a one-time $xx "Conversion Credit" specifically tied to the phased removal of competitor-equivalent products. Authorize a one-time $xx "Conversion Credit" specifically tied to the phased removal of competitor-equivalent products. Authorize a one-time $xx "Conversion Credit" specifically tied to the phased removal of competitor-equivalent products. Contractual Lock-in(Tier 2)

24-Month Extension

Proposed xx% fixed-price ceiling in exchange for an exclusive 2-year supply agreement (MSA). Contractual Lock-in(Tier 2) Contractual Lock-in(Tier 2) Contractual Lock-in(Tier 2) 24-Month Extension 24-Month Extension 24-Month Extension Proposed xx% fixed-price ceiling in exchange for an exclusive 2-year supply agreement (MSA). Proposed xx% fixed-price ceiling in exchange for an exclusive 2-year supply agreement (MSA). Proposed xx% fixed-price ceiling in exchange for an exclusive 2-year supply agreement (MSA). These numbers were highly tailored for the specific customer that the model specifically worked through our historic contracts (unstructured data) and pricing playbooks (structured data) among other sources. Within the first two months of AI deployment, model adoption reached nearly 100%. Churn rate has reduced by 10% and we target achieving a 25% churn rate reduction by the first half of 2026. Key learnings Key learnings Key learnings While this experience might seem like a paean to AI models, it goes beyond that. It was a learning experience for a strategist on how best to leverage data, analytics, AI and most importantly people to create valuable business impact. My three key learnings to date from this on-going journey that started two years ago would be: Data is meaningless without actionable insights: If we want real world outcomes, we need real actions. And to take real actions we need insights and direction. Having large amounts of high-quality data is just the first step of the process. Parsing them to come up with actionable intelligence and taking those actions is critical Use AI for large datasets: Automation creates efficiency and efficiency creates high RoI. AI is the most powerful tool we have today to automate insights. While humans are good at building the rubrics, AI is the best way to deploy it in a repeatable and sustainable manner AI can augment humans, not replace them: Customer service at the end is a relationship game. AI cannot build meaningful emotional relationships. We need humans for that. Furthermore, AI is a capability, not something that can take accountability for actions and results. AI helped our frontlines sales team but could never replace them.

Target

new customers

Why My "Spectacular" Churn Model Failed—And How AI Saved It

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Mastering the Moves: A Comprehensive Study on 4 Go-To-Market Strategies (Part I)

A Developer’s Guide to Fixing Computer Vision Annotations

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

Mastering the Moves: A Comprehensive Study on 4 Go-To-Market Strategies (Part I)

A Developer’s Guide to Fixing Computer Vision Annotations

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps