AI Case Study

AirBnB achieves gains in bookings using machine learning models to improve property search results

AirBnB has experimented with different machine learning models to improve the property search results for its users and currently uses a combination of neural networks with gradient boosted decision trees resulting in relative gains in bookings.

Industry

Consumer Goods And Services

Travel And Leisure

Project Overview

"ŒThe search ranking model under discussion is part of an ecosystem of models, all of which contribute towards deciding which listings to present to the guest. ThŒese include models that predict the likelihood of a host accepting the guest’s request for booking, models that predict the probability the guest will rate the on trip... It is common for guests to do multiple searches, clicking through some of the listings to view their details page. Successful sessions end with the guest booking a listing."

Reported Results

"Over time we realized that moving to deep learning is not a drop-in model replacement at all; rather it’s about scaling the system. As a result, it required rethinking the entire system surrounding the model. Con€ned to smaller scales, models like GBDT are arguably at par in performance and easier to handle, and we continue to use them for focused medium sized problems.
So would we recommend deep learning to others? ŒThat would be
a wholehearted Yes. And it’s not only because of the strong gains
in the online performance of the model. Part of it has to do with
how deep learning has transformed our roadmap ahead. Earlier
the focus was largely on feature engineering, but a‰fter the move to
deep learning, trying to do beŠtter math on the features manually
has lost its luster. ThŒis has freed us up to investigate problems at a
higher level, like how can we improve our optimization objective,
and are we accurately representing all our users?

As a side note, while DNNs have achieved human level performance on certain image applications, it is very hard for us to judge where we stand for a similar comparison. Part of the problem is that it’s unclear how to de€fine human level performance. Going through the logs, it’s quite challenging for us to identify which listing was booked. We €find no objective truth in the logs, only tradeoff‚s highly conditional upon the budget and tastes of the guest which remain mostly hidden."

Technology

"Typical con€figuration of the network: an input layer
with a total of 195 features aft‰er expanding categorical features to
embeddings, feeding the fi€rst hidden layer with 127 fully connected ReLUs, and then the second hidden layer with 83 fully connected ReLUs.
Inspired by NN [neural net] architectures the new model was an aŠttempt to combine the strengths of all three models. For the FM [factorization machine] model we took the fi€nal prediction as a feature into the NN. From the GBDT [gradient boosted decision tree] model, we took the index of the leaf node activated per tree as a categorical feature.

Function

Marketing

Merchandising

Background

"ŒThe home sharing platform at Airbnb is a two sided marketplace for hosts to rent out their spaces, referred to as listings, to be booked by prospective guests from all around the world. A typical booking starts with the guest issuing a search at airbnb.com for homes available in a particular geographic location. ThŒe task of search ranking is to respond to the guest with an ordered list of a handful of listings from the thousands available in the inventory".

Benefits

Data

"ŒThe features feeding the DNN were mostly simple properties of the listings such as price, amenities, historical booking count, etc, fed directly with minimal feature engineering.

ThŒe searches performed by a guest and their interactions are logged. While training, a new model has access to the logs of the current and previous models used in production. ŒThe new model is trained to learn a scoring function that assigns impressions of the booked listings in the logs at as high a rank as possible... ThŒe new model is then tested online in an A/B testing framework to see if it can achieve a statistically signifi€cant increase in conversions compared to the current model."