Reputation-based ranking systems can be biased towards the sensitive attributes of the users, meaning that certain demographic groups have systematically lower reputation scores. Nevertheless, if we unbias the reputation scores considering one sensitive attribute, bias still occurs when considering different sensitive attributes. For this reason, reputation scores should be unbiased independently of any sensitive attribute that characterizes the users.

In a CIKM 2021 paper, with Guilherme Ramos and Mirko Marras, we study reputation independence from multiple sensitive attributes of the users, so as to produce unbiased item rankings.

Reputation-based ranking systems score the users to decide how to rank item. In our SIGIR 2020 study, we have shown that reputation scores are based on users’ sensitive attributes, leading to minority demographic groups (such as females) being deemed as less relevant, thus accounting for their preferences less when computing items’ rankings (disparate reputation). To overcome disparities, the concept of reputation independence (RI) was introduced, ensuring that the reputation scores of users belonging to different legally-protected groups are statistically indistinguishable. However, we made interventions on individual sensitive attributes and this does not provide guarantees to groups obtained considering multiple sensitive attributes.

Introducing multi-attribute reputation independence

We design a strategy that, given a set of users’ sensitive attributes, mitigates the user reputations’ bias against user groups characterized by different combinations of those attributes. The method partitions users according to more than one attribute,

jointly. Specifically, we consider a set of K sensitive attributes, where each attribute has a set of classes. Now, we consider all the 𝐾-tuples of classes, and to each 𝐾-tuple, we associate the set of users, which is the set of users characterized by that set of classes. Given these K-tuples, we propose a method that reconciles reputation’s distributions for each 𝐾 -tuple of attributes classes so that the reputations of each 𝐾-tuple of classes are “statistically indistinguishable”. This leads to the targeted multi-attribute RI. Please, refer to the paper for the technical details of our approach and for its theoretical properties.

Evaluation