Users’ claimed willingness to interact with novel and diverse items doesn’t always match the recommendations they accept. While users may express a desire for novelty and diversity in recommendations, their actual choices often gravitate towards relevance. This key finding challenges the conventional approach in multi-objective recommender system design, emphasizing the necessity of aligning system objectives more closely with the nuanced behavior and preferences of users.

In a recent paper with Patrik Dokoupil and Ladislav Peska, and published in the proceedings of RecSys ’23, we present the results of a user study in the context of multi-objective recommender systems (MORS), which are designed to provide suggestions to users based on multiple, potentially conflicting, goals. Specifically, we observe users’ self-proclaimed propensities towards relevance, novelty, and diversity objectives and analyze if these propensities are reflected in the recommendations they accept.

This paper presents a user study that examines how users interact with recommended items and their self-proclaimed propensities towards relevance, novelty, and diversity objectives.

Study Design

The research involved an in-depth user study using the MovieLens-Latest dataset. Participants were exposed to both a single-objective relevance-based recommender system (RS) and MORS, with sessions allowing for the adjustment of their preferences towards relevance, novelty, and diversity.

Key Observations

1. User Behavior vs. System Objectives. Despite the ability to adjust for novelty and diversity, users predominantly chose items recommended based on relevance. This behavior was consistent even though MORS could be fine-tuned for novelty and diversity.
2. Long-Term Satisfaction. The presence of MORS-based recommendations was crucial for long-term user satisfaction. Interestingly, the early sessions featuring MORS had a significant positive impact on user satisfaction in later stages.
3. Discrepancy in User Propensity and Choices. There was a notable gap between users’ claimed willingness to interact with novel and diverse items and their actual choices. This suggests a need for systems that better facilitate users’ understanding of the recommendations.

Implications and Future Directions

Our findings point towards several implications:

• User Understanding and RS Design: There’s a need for recommender systems that help users understand why certain recommendations are made. This could involve better explanations or visualizations of how recommended items fulfill various objectives.
• Long-Term User Studies: Future studies should consider longer periods and varied settings to understand how user preferences evolve and how they interact with multi-objective recommendations over time.

Conclusion

Our work sheds light on the complex dynamics between user preferences and recommender system objectives. It underscores the need for recommender systems that are not only accurate but also align well with the nuanced preferences of users. The study’s insights are crucial for advancing the field of multi-objective recommendation systems.