Supervisor

Suitable for

Abstract

"Recommender systems are a classic field for the application of Machine Learning techniques. Approaches developed in this area over the 10+ years elapsed since the Netflix Prize have relied almost exclusively on collaborative filtering models, which learn a latent representation of products and customers that best reconstruct observed customer-product interactions.

We believe that recently developed Transformer architectures represent a unique opportunity to revisit the domain, as these models, applied directly on purchase data, may enable to learn a more powerful contextual representation of products. In a recent project, we have successfully demonstrated the capabilities of this type of approach in an application related to product substitution based on product embeddings generated by a BERT -like model. We propose to explore this approach in the context of a different application: autocompleting the content of a shopping cart in the manner of a text generation task.

Objectives

The student will extend our existing Transformer-based product embedding package with a new model architecture and / or training method enabling the generation of relevant and non-trivial complementary products given a customer basket.

Depending on the student’s interest and progress, directions of research may include the implementation of transformer architectures better suited for generation (e.g., GPT-like), the implementation of fine-tuning methods better suited to generate basket complements and the implementation of loss functions encouraging the prediction of less frequent and more niche products complementing specific items within a basket.

The student will then study the limitations of the proposed model to assess whether it could be used for real world applications and to identify possible areas of further improvements. Experiments will be run on publicly available data such as the Instacart dataset as well as a much larger dataset belonging to a major retail brand, client of J2-Reliance. Computational resources will be available for model training and tuning on Google Cloud Platform.

Prerequisites

The project requires a good mastery of object-oriented programming (preferably in Python), solid foundations in Deep Learning and a basic understanding of Transformer models.

Learning objectives and skills involved

The student will enhance their creativity in Machine Learning and their capacity to solve original and concrete problems. They will also acquire an advanced understanding of Transformer models and their application to representation learning outside of the NLP realm. Finally, they will strengthen their coding skills and be exposed to the challenges related to productionising Machine Learning models.

Supervision

Company Supervisor: Damien Arnol, PhD – LinkedIn profile – damien.arnol@j2reliance.co.uk. Damien has a PhD in Machine Learning and Bioinformatics from the University of Cambridge and an expertise in unsupervised representation learning in contexts such as dimensionality reduction and recommender systems. In the past three years, he has been leading multiple R&D projects for big groups in the retail industry and advised digital and tech start-ups developing AI and data-driven products.

Weekly meetings will be organised between the industry supervisor and the student, either remotely or in-person depending on the student’s preference. The student will also be added to the company’s Slack group and encouraged to interact as much as needed with the company’s engineers to receive guidance both from a ML perspective and from a coding perspective.

Company: J2-Reliance Ltd.

Founded by two machine learning experts (Cambridge PhD and Mila PhD) and a former industry executive, J2-Reliance is an R&D consultancy with an obsession for transforming data and technologies into business value. We are experts in identifying what and how technology can solve a specific business problem and we develop the corresponding solutions end-to-end.

Our philosophy is to leverage cutting-edge technologies in an ambitious and creative way and rely on technical insights to adapt innovations from various research fields to solve hard business problems. In the retail space, we have previously used Language Modelling techniques (Transformers) to analyse customer behaviours, learn meaningful product representations and as a result develop powerful recommender systems.

Bidirectional Encoder Representations from Transformers (Devlin et al 2018) For example, avoiding the trap of recommending frequently purchased products irrespective of the content of the input basket "