ParVecMF: A Paragraph Vector-based Matrix Factorization Recommender System

Review-based recommender systems have gained noticeable ground in recent years. In addition to the rating scores, those systems are enriched with textual evaluations of items by the users. Neural language processing models, on the other hand, have already found application in recommender systems, mainly as a means of encoding user preference data, with the actual textual description of items serving only as side information. In this paper, a novel approach to incorporating the aforementioned models into the recommendation process is presented. Initially, a neural language processing model and more specifically the paragraph vector model is used to encode textual user reviews of variable length into feature vectors of fixed length. Subsequently this information is fused along with the rating scores in a probabilistic matrix factorization algorithm, based on maximum a-posteriori estimation. The resulting system, ParVecMF, is compared to a ratings’ matrix factorization approach on a reference dataset. The obtained preliminary results on a set of two metrics are encouraging and may stimulate further research in this area.

💡 Research Summary

The paper introduces ParVecMF, a novel recommendation framework that integrates textual review information with rating data through a two‑stage approach. In the first stage, user‑ and item‑level reviews are encoded using the Paragraph Vector (PV‑DM) model, which learns distributed representations for both words and whole documents (paragraphs). This yields fixed‑length dense vectors (embeddings) that capture semantic meaning and limited word order, even when trained on unlabeled data. The authors apply PV‑DM directly to the review texts of the Amazon FineFoods dataset, setting the minimum word frequency to one to include all terms despite the dataset’s sparsity.

In the second stage, these embeddings are fused into a probabilistic matrix factorization (PMF) model. Traditional PMF assumes that each observed rating r_ij is drawn from a normal distribution with mean equal to the inner product of latent user and item vectors (u_i·v_j) and a precision parameter c_ij. The latent vectors themselves are given Gaussian priors with zero mean. ParVecMF modifies this prior by adding the learned review embeddings (θ_i for users, θ_j for items) as the mean of the prior, while retaining a zero‑mean Gaussian offset (ε_i, ε_j) with precision λ_u and λ_v respectively: u_i = θ_i + ε_i, v_j = θ_j + ε_j. This formulation treats the textual embeddings as informative priors that guide the latent factors, especially useful when rating data are scarce.

Maximum a‑posteriori (MAP) estimation is employed to learn the model parameters. The authors derive a log‑posterior objective L that combines three terms: (1) a regularization term penalizing deviation of u_i from θ_i weighted by λ_u, (2) a similar term for v_j and θ_j weighted by λ_v, and (3) a reconstruction error term for observed ratings weighted by c_ij. Setting the gradient of L to zero yields closed‑form update rules for each dimension of u_i and v_j, which resemble standard PMF gradient updates but with an additive bias of the corresponding text embedding.

Experiments are conducted on the FineFoods dataset, comprising 568,454 reviews from 256,059 users on 74,258 products. The dataset is highly sparse; most users and items have only a few reviews, and the median review length is 56 words. The authors perform 5‑fold cross‑validation, evaluating recommendation quality with two ranking metrics: Mean Average Precision at 5 (MAP@5) and Mean Reciprocal Rank at 5 (MRR@5). Relevance is defined as a rating of four or five stars. Two baselines are compared: (a) the proposed ParVecMF model (ratings + review embeddings) and (b) a conventional SVD‑based collaborative‑filtering MF that uses only the rating matrix.

Results show that ParVecMF modestly outperforms the SVD baseline, achieving a MAP improvement of approximately 0.012 and an MRR improvement of about 0.009. While the gains are modest, they demonstrate that incorporating review semantics via Paragraph Vectors can positively influence recommendation performance, confirming the feasibility of a unified probabilistic model that jointly leverages textual and rating signals.

The paper’s contributions are threefold: (1) employing Paragraph Vectors to convert variable‑length reviews into dense, semantically rich embeddings; (2) integrating these embeddings as informative priors within a probabilistic matrix factorization framework using MAP estimation; and (3) providing empirical evidence on a real‑world, large‑scale e‑commerce dataset that the combined approach can surpass a pure rating‑based MF method.

Limitations are acknowledged. The review embeddings are pre‑trained and fixed during MF learning, preventing end‑to‑end fine‑tuning that might further align the embeddings with the recommendation task. The dataset’s extreme sparsity and short review length may limit the quality of the Paragraph Vectors, and the evaluation is confined to a single domain, leaving open questions about generalizability. Future work could explore joint training of the language model and matrix factorization, incorporate additional modalities (e.g., product images, category hierarchies), and test the approach across diverse domains and larger recommendation list sizes.