Date of Award
5-2024
Document Type
Dissertation
Publisher
Santa Clara : Santa Clara University, 2024
Degree Name
Doctor of Philosophy (PhD)
Department
Computer Science and Engineering
First Advisor
Yi Fang
Abstract
Recent advancements in Information Retrieval (IR) and machine learning have significantly improved ranking and search system performance. However, these data-driven approaches often suffer from inherent biases present in training datasets, leading to unfair treatment of certain demographic groups and contributing to systematic discrimination based on race, gender, or geographic location. This research aims to address the fairness and bias issue in ranking and search systems by proposing innovative frameworks that mitigate data bias and ensure equitable representation and exposure across diverse groups.
We introduce two novel frameworks: the Meta-learning based Fair Ranking (MFR) model and the Meta Curriculum-based Fair Ranking (MCFR) framework, both designed to alleviate dataset bias through automatically-weighted loss functions and curriculum learning strategies, respectively. These approaches utilize meta-learning to adjust ranking loss, focusing particularly on improving the fairness metrics for minority groups while maintaining competitive ranking performance. Additionally, we conduct an empirical evaluation of Large Language Models (LLMs) in text-ranking tasks, revealing biases in handling queries and documents related to binary protected attributes. Our analysis offers a benchmark for assessing LLMs’ fairness and highlights the necessity for equitable representation in search outcomes.
Furthermore, we explore the challenge of data selection bias in multi-stage recommendation systems, particularly in online advertising contexts like Pinterest’s multi-cascade ads ranking system. Through comprehensive experiments, we assess various state-of-the-art methods, and our findings demonstrate the effectiveness of a modified version of unsupervised domain adaptation (MUDA) in mitigating selection bias.
Collectively, our work contributes to the development of fairer ranking and search systems. By addressing bias at its source and employing meta-learning and curriculum learning techniques, we pave the way for more equitable and transparent IR systems that serve diverse user bases without discrimination.
Recommended Citation
Wang, Yuan, "Fairness and Bias of Machine Learning in Search and Ranking" (2024). Engineering Ph.D. Theses. 55.
https://scholarcommons.scu.edu/eng_phd_theses/55