Ranking systems that account for the reputation of the users can be biased towards different demographic groups, especially when considering multiple sensitive attributes (e.g., gender and age). Providing guarantees of reputation independence can lead to unbiased and effective rankings. Moreover, these rankings are also robust to attacks.

In a study, published by the Machine Learning journal (Springer) and conducted with Guilherme Ramos and Mirko Marras, we propose a novel approach to introduce reputation independence for multiple sensitive attributes simultaneously. We then analyze the extent to which our approach impacts on discrimination and other important properties of the ranking system, such as its quality and robustness against attacks.

This study extends the advances we made in this topic in our CIKM 2021 paper. We remind the readers to that blog post and to our paper, for our approach to reputation independence for multiple sensitive attributes. In this post, we’ll focus just on the robustness analysis. A preprint of our study is also available.

The results for the random spamming (subfigure a) show us that the robustness for mitigation methods in Eqs. (1) and (3) is comparable, whereas both of them led to a slight improvement concerning the Arithmetic Average (AA).

Subfigure b depicts the results under the love/hate attack. Notice that, by mitigating disparate reputation, the attack is less effective for both methods. Indeed, robustness is significantly higher than the one obtained with the AA. In subfigure c, the hate/love scenario leads to similar observations.