AI Case Study
ASOS.com researchers demonstrate improved customer lifetime value predictions using neural networks and automatic feature selection but do not advise implementation due to increased cost
ASOS researchers demonstrate how the currently used customer lifetime value prediction system can be improved through the use of automatic feature selection. These predictions are used in business operations such as marketing for customising and targeting retention strategies. However, due to the increase cost associated with running the best performing system, it is not at this time considered a commercially viable solution.
Industry
Consumer Goods And Services
Retail General
Project Overview
The research team looks at both calibrating the existing random forests model and implementing a hybrid model using the current random-forest system and incorporates a deep neural network.
"The deployed ASOS CLTV system uses the state-of-the-art architecture, which is a Random Forest (RF) regression model with 132 handcrafted features. ...We showed that learning an embedding of a rich source of data (products viewed by a customer) in an unsupervised setting can improve the performance over using only handcrafted features and plan to incorporate this embedding in the live system as well as apply this same approach to other types of events (e.g. products bought by a customer)... The model is retrained every day using customer data from the past two years. Labels are the net customer spend over the previous year. Model parameters are learned in the training
period and used to predict CLTV from new features in the
live system."
Reported Results
Despite the research demonstrating performance improvements, the researchers state that currently "the monetary cost required to perform such training outweighs the benefit of gain in performance. The cost rises exponentially with increasing number of neurons, indicating that a hybrid model that outperformed our calibrated RF model is not be practical on cost grounds."
Technology
"We experimented with learning representations directly from data using two different approaches: (1) by training a feedforward neural network on the handcrafted features in a supervised setting; (2) By augmenting the RF feature set with unsupervised
customer embeddings learnt from web and app browsing data. The novel customer embeddings are shown to improve CLTV prediction performance significantly compared with our benchmark. Incorporating embeddings into long-term prediction models is challenging because, unlike handcrafted features, the features are not easily identifiable.
In the training period we learn parameters for each feature. However, as the features are not labelled and their order randomly permutes from training to test time, it is not possible to map the training parameters to the test features. We describe how this problem can be solved within the neural embedding framework using a form of warm start for the test period embeddings.
A handcrafted feature generator on Apache Spark and an experimental customer embedding generator on GPU machines
running Tensorflow, which uses only the web/app sessions as
input. The model is trained in two stages using an Apache Spark
ML pipeline. The first stage pre-processes the features and trains
random forests for churn classification and CLTV regression on
percentiles. The second stage performs calibration and maps percentiles to real values. Finally the predictions are presented to a range of business systems that trigger personalised engagement
strategies with ASOS customers."
Function
Strategy
Strategic Planning
Background
ASOS.com is an online global fashion retailer. "Customer lifetime value (CLTV) prediction is an important problem in e-commerce where an accurate estimate of future value allows retailers to effectively allocate marketing spend, identify and nurture high value customers and mitigate exposure to losses. CLTV system addresses two tightly coupled problems: CLTV and churn prediction." A customer's CLTV is the purchases minus returns they make in a year, and is considered "churned" if they have not ordered in a year. CLTV predictions are fed to other business operations to inform marketing efforts and create custom retention strategies.
Benefits
Data
Proprietary customer purchasing data from over the past two years along with web and app browsing data. "Customer data is collected and pre-processed by our data warehouse and stored on Microsoft Azure blob storage. The processed data is used to generate handcrafted features in Spark clusters, with web/app sessions additionally used to produce experimental customer embeddings in Tensorflow."