Adaptive Quality-Diversity Trade-offs for Large-Scale Batch Recommendation

A core research question in recommender systems is to propose batches of highly relevant and diverse items, that is, items personalized to the user’s preferences, but which also might get the user out of their comfort zone. This diversity might induce properties of serendipidity and novelty which might increase user engagement or revenue. However, many real-life problems arise in that case: e.g., avoiding to recommend distinct but too similar items to reduce the churn risk, and computational cost for large item libraries, up to millions of items. First, we consider the case when the user feedback model is perfectly observed and known in advance, and introduce an efficient algorithm called B-DivRec combining determinantal point processes and a fuzzy denuding procedure to adjust the degree of item diversity. This helps enforcing a quality-diversity trade-off throughout the user history. Second, we propose an approach to adaptively tailor the quality-diversity trade-off to the user, so that diversity in recommendations can be enhanced if it leads to positive feedback, and vice-versa. Finally, we illustrate the performance and versatility of B-DivRec in the two settings on synthetic and real-life data sets on movie recommendation and drug repurposing.

💡 Research Summary

The paper tackles the long‑standing challenge of balancing relevance (quality) and diversity in batch recommendation, especially when the item catalog contains millions of candidates. It assumes that a feedback model qΘ(ϕ, h) – which predicts the expected user response for an item embedding ϕ given a user context h – is available offline. Within this setting the authors introduce B‑DivRec, a novel algorithm that merges Determinantal Point Processes (DPP) with a “fuzzy denuding” procedure to control how much the recommended batch should differ from the user’s past interactions.

The core technical contribution is the definition of a parameterised likelihood matrix Lλ,f for a user h, a history H and a candidate subset S: Lλ,f(S; h) = (Q_h,S)^{2λ} · f(k,S,H)^{2(1‑λ)} · (Q_h,S)^{2(1‑λ)}. Here Q_h,S is a diagonal matrix whose entries are the predicted feedback scores qΘ(ϕ_i, h) for items i∈S, λ∈