Date of Award

4-2026

Document Type

Dissertation

Publisher

Santa Clara : Santa Clara University, 2026

Degree Name

Doctor of Philosophy (PhD)

Department

Computer Science and Engineering

First Advisor

Yi Fang

Abstract

Search systems traditionally balance efficiency and effectiveness by decomposing the retrieval process into a pipeline of query understanding, retrieval, and reranking to surface relevant documents. The rapid progress of large language models (LLMs) is reshaping this paradigm: modern systems increasingly aim not only to retrieve information but to produce direct answers, while remaining reliable, efficient, and scalable. This dissertation, Enhancing Search Systems with Large Language Models, studies how LLMs can improve each stage of the search stack and how new evaluation resources and efficiency measures are needed to deploy LLM-powered search in practice.

First, for query understanding in large-scale e-commerce search, we propose an entityaware multi-task learning framework that jointly optimizes multiple query understanding tasks while incorporating entities mined from engagement data as contextual signals. This approach reduces development and maintenance overhead compared to task-specific models, and yields substantial gains in both offline and online evaluations on metrics at industrial scale. Second, for dense retrieval in domains with limited supervision, we introduce soft prompt tuning for augmentation, in which task-specific soft prompts are learned from small labeled sets and used to guide open-source LLMs to generate weak queries for unlabeled documents. The resulting synthetic query–document pairs enable effective training of domain-adapted dense retrievers, outperforming unsupervised baselines and prior LLM-based augmentation strategies. Third, for LLM-based reranking, we investigate the efficiency–effectiveness trade-off and propose hardware-agnostic metrics, ranking quality per PetaFLOP and queries per PetaFLOP, together with an interpretable FLOPs estimator that enables principled comparisons across model sizes and inference configurations. Fourth, we develop MemRerank, a preference-memory module for product reranking in LLM-powered shopping assistants. This module distills a user’s purchase history into structured within-category and cross-category preference signals and uses reinforcement learning to train an LLM-based extractor that directly optimizes setwise reranking; on a benchmark of purchase histories, candidate sets and relevance labels, MemRerank yields significant personalization gains over baselines that inject raw history or employ untrained extractors. Finally, as search systems evolve toward retrieval-augmented question answering, we study the critical failure mode of out-of-scope questions, where retrieved passages appear relevant but do not contain sufficient evidence, often triggering hallucinated answers. We present ELOQ, a dataset and generation framework for out-of-scope questions, and demonstrate improved methods for detecting and responding appropriately when evidence is insufficient.

Collectively, this dissertation provides methods, metrics, and resources that advance LLM-enhanced search across the full pipeline, from query understanding and representation learning to efficient reranking and personalised memory-based reranking and reliable question answering, bridging the gap between research progress and practical deployment in real-world search systems.

Download

Share

COinS