In the presence of data imbalances, where some demographic groups of providers are represented more than others, the items of all the demographic groups that are not the majority group are under-recommended. A mitigation that accounts for the representation of each demographic group allows to introduce equity in the recommendation process, without having an impact in terms of effectiveness.

In a paper published in the Information Processing & Management journal (Elsevier), with Elizabeth Gómez and Maria Salamó, we study how the providers of different continents are impacted by the way recommender systems generate their results. While in our ECIR 2021 paper we considered a setting where we had two demographic groups, in a majority-versus-rest setting, in this study we show that this is not enough to mitigate unfairness. For this reason, we propose a new algorithm to generate equity for the providers of different continents, in terms of visibility and exposure.

This paper gained the first author, Elizabeth Gómez, one of the Women in RecSys Awards in 2022.

Assessing provider fairness from a binary perspective

We characterize disparate impact that occurs in the presence of multiple demographic groups, by considering two metrics:

• Disparate visibility, which measures the difference between the share of recommendations for items of a demographic group and the representation of that group (where representation is either the percentage of items or ratings associated with that group);
• Disparate exposure, which measures the difference between the exposure obtained by a demographic group in the recommendation lists (i.e., in which positions the items of that group appear) and the representation of that group.

We assessed the behavior of these two metrics in the movie and book domains, by considering the MovieLens-1M and BookCrossing datasets, and by studying four models, namely Most Popular, Random Guess, UserKNN, ItemKNN, BPR, BiasedMF, and SVD++. Specifically, we evaluated both the behavior of the original models and the results obtained with our mitigation for binary groups.

The paper contains the detailed results, but here we report a few takeaways from our assessment:

• Both datasets have a strong geographic imbalance towards North America, which is the most represented group from both item- and rating-based perspectives. The imbalance is strengthened when considering the rating-based representation, meaning that the largest groups attract a share of ratings that is even higher than the amount of items it offers. This clearly has a price for the smaller groups, which are able to attract a percentage of ratings that is lower than the amount of items they offer. Hence, user–item interactions favor the largest group and exacerbate imbalances that already existed in the item offer, even before we run a recommendation algorithm.
• Recommendation models better adjust to the rating distribution than to the item offer associated with a group. Factorization-based approaches are able to account for the needs of smaller groups, unless the imbalance in the input data is too severe. Even though we are working in a multi-group setting, recommender systems mostly operate as if two big groups existed; when one group is favored, the other is affected, both in terms of visibility and exposure. A mitigation for binary groups helps reducing disparities, but is not enough to introduce fairness for groups shaped at the continent level. Indeed, integrating more recommendations of the items from the minority group does not ensure that these recommendations are equally distributed among the different continents, so disparities still emerge.

Mitigating disparate impact

Our mitigation algorithm is based on the idea of promoting in the recommendation list the item, that considering all the users, minimizes the loss in prediction. To achieve this goal, our re-ranking approach works in two passes:

1. We start by targeting the desired visibility, to make sure the items of the disadvantaged groups (i.e., those that are currently under-recommended) are recommended enough times.
2. Once the target visibility is reached (i.e., the items of the minority groups are recommended enough times), we move items up inside the recommendation list to reach the target exposure. This allows the items of the minority groups to appear in higher positions in the list.

The paper contains the details of our approach, including its pseudo-code.

Impact of mitigation

The full paper contains the detailed results of each algorithm. Here are the main outcomes that can be observed:

• When providing a re-ranking based on minimal predicted loss, the effectiveness remains stable, but disparate visibility and disparate exposure are mitigated. The most effective approach remains the best one after the mitigation.
• Introducing provider fairness requires interventions at the recommendation-list level. Mitigating by boosting predicted relevance for the disadvantaged groups, as done by the state-of-the-art approaches, does not provide guarantees of equity of visibility and exposure are fully mitigated. Disparities are only partially mitigated.