Performance Comparison of Algorithms for Movie Rating Estimation

In this paper, our goal is to compare performances of three different algorithms to predict the ratings that will be given to movies by potential users where we are given a user-movie rating matrix based on the past observations. To this end, we evaluate User-Based Collaborative Filtering, Iterative Matrix Factorization and Yehuda Koren’s Integrated model using neighborhood and factorization where we use root mean square error (RMSE) as the performance evaluation metric. In short, we do not observe significant differences between performances, especially when the complexity increase is considered. We can conclude that Iterative Matrix Factorization performs fairly well despite its simplicity.

💡 Research Summary

The paper presents an empirical comparison of three well‑known recommendation algorithms applied to the task of predicting movie ratings: User‑Based Collaborative Filtering (UB‑CF), an Iterative Matrix Factorization (IMF) approach, and Yehuda Koren’s Integrated Model that combines neighborhood methods with latent factor models. The authors construct a proprietary dataset consisting of 10,000 users and 1,000 movies, yielding 1,176,192 observed ratings out of a possible 10 million (approximately 11.7 % density). The data are randomly split into 90 % for training and 10 % for validation; the validation set is used exclusively for RMSE calculation and hyper‑parameter tuning.

User‑Based Collaborative Filtering
Two similarity metrics are examined: Pearson correlation and cosine similarity. For each metric the number of nearest neighbors (N) is varied among {5, 10, 25, 50, 100}. The prediction formula incorporates a bias correction term (user average) and weights neighbor contributions by similarity. Experiments show that cosine similarity consistently outperforms Pearson, achieving the lowest RMSE of 1.01 when 100 neighbors are used. The authors note the classic drawbacks of UB‑CF: cold‑start for new users/items, sensitivity to data sparsity, and diminishing returns when increasing N beyond 50.

Iterative Matrix Factorization
The IMF method treats the rating matrix as a low‑rank approximation. After subtracting each movie’s mean rating (centering) and filling missing entries with zeros, a singular value decomposition (SVD) is performed iteratively. After each low‑rank reconstruction the known training entries are re‑inserted, and the process repeats until convergence. The authors explore the latent dimension K (rank) and the number of iterations. Results indicate that rank 3 and 20 iterations yield the best performance, with an RMSE of 0.9908—about 2 % lower than UB‑CF. Higher ranks lead to over‑fitting and increased error, underscoring the importance of careful rank selection.

Koren’s Integrated Model
The third approach follows Koren’s 2008 formulation, which adds a baseline term (global mean µ, user bias b_u, item bias b_i) to a latent factor inner product (q_iᵀ p_u) and augments it with two neighborhood components: (i) a weighted sum of deviations of explicitly rated neighbor items, and (ii) a similar term for implicit feedback (here defined as a binary indicator of whether a rating exists). The model requires learning biases, latent vectors (p_u, q_i, y_j), and neighborhood weights w_it, c_it, as well as determining the neighborhood size k. The paper does not report a final RMSE for this method, but states that “no significant differences” were observed among the three techniques, implying that the integrated model’s performance is comparable to the other two.

Discussion and Conclusions
The authors conclude that, despite the theoretical advantages of the integrated model, the simpler IMF approach achieves the lowest RMSE while being easier to implement. They argue that the marginal gains of more complex models do not justify the additional computational overhead, especially when considering scalability and real‑world deployment constraints. Limitations are acknowledged: the dataset is not a publicly available benchmark (e.g., MovieLens), hyper‑parameter search spaces are relatively narrow, and crucial implementation details for the integrated model (learning rates, regularization strengths, convergence criteria) are omitted, which hampers reproducibility. The paper suggests future work should involve testing on diverse public datasets, incorporating side information (genres, timestamps), and exploring deep learning‑based alternatives.

Overall, the study provides a concise, data‑driven comparison that highlights the practical trade‑offs between algorithmic complexity and predictive accuracy in collaborative recommendation systems.