AI Case Study

ASOS.com researchers demonstrate improved customer lifetime value predictions using neural networks and automatic feature selection but do not advise implementation due to increased cost

ASOS researchers demonstrate how the currently used customer lifetime value prediction system can be improved through the use of automatic feature selection. These predictions are used in business operations such as marketing for customising and targeting retention strategies. However, due to the increase cost associated with running the best performing system, it is not at this time considered a commercially viable solution.

Industry

Consumer Goods And Services

Retail General

Project Overview

The research team looks at both calibrating the existing random forests model and implementing a hybrid model using the current random-forest system and incorporates a deep neural network.
"ThŒe deployed ASOS CLTV system uses the state-of-the-art architecture, which is a Random Forest (RF) regression model with 132 handcra‰fted features. ...We showed that learning an embedding of a rich source of data (products viewed by a customer) in an unsupervised settŠing can improve the performance over using only handcraft‰ed features and plan to incorporate this embedding in the live system as well as apply this same approach to other types of events (e.g. products bought by a customer)... Th‡e model is retrained every day using customer data from the past two years. Labels are the net customer spend over the previous year. Model parameters are learned in the training
period and used to predict CLTV from new features in the
live system."

Reported Results

Despite the research demonstrating performance improvements, the researchers state that currently "the monetary cost required to perform such training outweighs the benefi€t of gain in performance. The cost rises exponentially with increasing number of neurons, indicating that a hybrid model that outperformed our calibrated RF model is not be practical on cost grounds."

Technology

"We experimented with learning representations directly from data using two di‚fferent approaches: (1) by training a feedforward neural network on the handcraft‰ed features in a supervised seŠtting; (2) By augmenting the RF feature set with unsupervised
customer embeddings learnt from web and app browsing data. ThŒe novel customer embeddings are shown to improve CLTV prediction performance signifi€cantly compared with our benchmark. Incorporating embeddings into long-term prediction models is challenging because, unlike handcraft‰ed features, the features are not easily identifi€able.

In the training period we learn parameters for each feature. However, as the features are not labelled and their order randomly permutes from training to test time, it is not possible to map the training parameters to the test features. We describe how this problem can be solved within the neural embedding framework using a form of warm start for the test period embeddings.

A handcraft‰ed feature generator on Apache Spark and an experimental customer embedding generator on GPU machines
running Tensorflƒow, which uses only the web/app sessions as
input. ThŒe model is trained in two stages using an Apache Spark
ML pipeline. ŒThe fi€rst stage pre-processes the features and trains
random forests for churn classi€fication and CLTV regression on
percentiles. ThŒe second stage performs calibration and maps percentiles to real values. Finally the predictions are presented to a range of business systems that trigger personalised engagement
strategies with ASOS customers."

Function

Strategy

Strategic Planning

Background

ASOS.com is an online global fashion retailer. "Customer lifetime value (CLTV) prediction is an important problem in e-commerce where an accurate estimate of future value allows retailers to eff‚ectively allocate marketing spend, identify and nurture high value customers and mitigate exposure to losses. ŒCLTV system addresses two tightly coupled problems: CLTV and churn prediction." A customer's CLTV is the purchases minus returns they make in a year, and is considered "churned" if they have not ordered in a year. CLTV predictions are fed to other business operations to inform marketing efforts and create custom retention strategies.

Benefits

Data

Proprietary customer purchasing data from over the past two years along with web and app browsing data. "Customer data is collected and pre-processed by our data warehouse and stored on Microsoft Azure blob storage. ‡The processed data is used to generate handcrafted features in Spark clusters, with web/app sessions additionally used to produce experimental customer embeddings in Tensorƒflow."